• Increase font size
  • Default font size
  • Decrease font size
Home Building Software RPM and Yum Building RPM and YUM on AIX 5.2

Building RPM and YUM on AIX 5.2

E-mail Print
Read : 9,732 times
(3 votes, average 4.33 out of 5)



1. Introduction
     1.1. Intended audience
     1.2. Getting the source packages
     1.3. Software dependencies
     1.4. Notational conventions
     1.5. Context
          1.5.1. Software
2. Building
     2.1. Environment
     2.2. Creating the target structure
     2.3. Embedding a private libgcc_s library
     2.4. Compiling the stdc++ library (4.1.1)
     2.5. Compiling zlib (1.2.3)
     2.6. Compiling the bzip2 library (1.0.1)
     2.7. Compiling the OpenSSL library (0.9.8j)
     2.8. Compiling the iconv library (1.12)
     2.9. Compiling the beecrypt library (4.1.2)
     2.10. Compiling the expat library (2.0.1)
     2.11. Compiling the ncurses library (5.7)
     2.12. Compiling pkg-config (0.23)
     2.13. Compiling gettext (0.17)
     2.14. Compiling the pcre library (7.8)
     2.15. Compiling glib (2.14.5)
     2.16. Compiling the popt library (1.13)
     2.17. Compiling libffi (3.0.4)
     2.18. Compiling the sqlite3 library (3.5.7)
     2.19. Compiling gnupg (1.4.8)
     2.20. Compiling libgpg-error (1.6)
     2.21. Compiling gpgme (1.1.4)
     2.22. Compiling the gdbm library (1.8.3)
     2.23. Compiling python (2.5.2)
     2.24. Compiling the xml2 library (2.7.2)
     2.25. Compiling urlgrabber (3.0.0)
     2.26. Compiling pygpgme (0.1)
     2.27. Compiling yum-metadata-parser (1.1.2)
     2.28. Compiling intltool (0.34.2)
     2.29. Compiling yum (3.2.13)
     2.30. Compiling createrepo (0.9.5)
     2.31. Compiling RPM (5.0.2)
     2.32. Creating the RPM database
     2.33. Initializing the database
3. Checking
4. Packaging
     4.1. Tar/gzip
          4.1.1. Creating the package
          4.1.2. Installing the package
5. The end

 

Document history
V 1.0-1 OCT 2006
  • First release
  • HP-UX 11.11 only
V 1.0-4 NOV 2006
  • Build native SD-UX package on HP-UX
  • Added HP-UX 11.00 version
V 2.0-1 MAR 2008
  • Switch to RPM 5.0.2
  • Added native Solaris package
V 2.1-1 JUL 2008
  • Added yum & createrepo
  • Added AIX version
v 2.2-1 FEB 2009
  • Component upgrade to the latest stable version

 

1 - Introduction

Why would you want to use RPM on a non-Linux system ? It may seem strange as all these environments already provide a native package format.

Let me just explain how we got such an exotic idea.

I am a Unix system engineer. The platform I am working on contains some HP-UX, Solaris, AIX, and Linux servers. Historically, on this platform, the developers delivered their software in the form of a gzipped tar file. But this format is very poor as it just allows to gather a set of files into a single file, and the need of a smarter package format quickly arose.

The question was then to choose the package format(s) we would use. Each OS provides its native package format, which are incompatible. So, using the native package format for each OS would have forced the development and system teams to learn how to build and use all these formats, which is quite impractical. Having eliminated this option, we searched for a format to be used everywhere. Only two of the native package formats exist outside of their native OS : SD-UX (HP-UX) and RPM (RPM is not really available in precompiled form on every non-Linux systems but, as it is open-source, it can be compiled). SD-UX was discarded for several reasons, the main ones being that it is closed-source, expensive, and its future is not clear.

RPM remained alone with several benefits :

  • It is easy to teach to the development and system administration teams,
  • It is widely used and known. Most people we hire already know it,
  • Combined with yum, it makes installation procedures very easy.

:note We didn't consider other Linux package formats, like apt, because the only Linux distribution we use is RedHat's RHEL.

Once the decision is made, we 'just' had to build RPM/YUM packages for all the environments. Although RPM is quite portable since version 5 (version 4 was very Linux-specific), this is not the case for YUM, whose port took a lot of time. This is due to several factors like poor documentation, complex dependencies, the need for a dedicated python interpreter... Actually, I wouldn't be surprised if it was the first time somebody builds it on a non-Linux system.

Well, the work is over now. Just follow the rest of this document and you too will be able to provide unified package installation/management procedures on all your Linux and non-Linux environments.

It will bring you the admiration of your {boss|colleagues|neighbours|wife|children|dog}, and, who knows, an appreciable boost in your career ;-).

1.1 - Intended audience

This document can be used in two different ways :

  • In order to apply the procedure and build the software exactly as it is described here, you don't need to be an expert : it just requires some basic Unix and shell knowledge, such as being able to download the source packages, uncompress/untar them, and create files with a text editor.
    At this level, the procedure given below can be splitted in two main steps :
    • you first set a limited number of environment variables (essentially the installation directory)
    • and the rest is just a suite of copy/paste operations from the document to your shell environment.
  • Now, if you want to understand everything that is explained below, or if you want to adapt it to your specific needs, it requires of course more knowledge on different domains, mostly :
    • build mechanisms in general,
    • compile and link operations (with a strong focus on shared libraries),
    • makefiles,
    • the Gnu autoconf-based configure/make process,
    • and even, when it is really needed, the ability to understand and adapt some C source code.

So, to resume, most people will just apply the procedure described below, which is relatively easy and just requires a basic working knowledge of the Unix shell environment.

:note Don't hesitate to post comments on issues you may have with this document. Each time you do it, you help me and the community in general to make the procedure more robust and reliable. I promise to reply to every question asked in comments. Please prefer comments to direct mail as they may benefit others.

1.2 - Getting the source packages

Some readers sometimes ask why I don't provide copies of the source packages I list below. Sure, it would be easier for you, but I won't do it as you MUST get your source packages from a repository you can trust ! You don't know me, you CANNOT trust source code downloaded from my site (except for my own projects).

So, once  again, always get your software from a trusted source and, when a package signature is provided, check it to make sure you get the official code.

I insist on this because compiling some modified source code or installing some corrupted precompiled binaries would give hackers everything they need to take control on your site (remember that most software we compile here runs as root).

In every chapter below, you will find a link to the best place to get the source package from.

So, remain a good, slightly paranoid, system administrator, and everything will go well.

1.3 - Software dependencies

The following diagram displays the libraries RPM depends upon and their relationships.

Image

yum has a very complex dependency diagram, that you'll find in the table below. If someone builds a readable graphical representation from this, I'll be glad to include it :

Module...  ...uses / depends on...
zlib <none>
bzip2 <none>
openssl zlib, iconv
iconv <none>
beecrypt stdc++
expat <none>
ncurses <none>
gettext iconv
glib gettext, iconv, pcre
popt gettext, iconv
libffi <none>
sqlite3 <none>
gnupg bzip2, zlib, gettext, iconv
libgpg-error gettext, iconv
gpgme gnupg, libgpg-error
gdbm <none>
python libffi, sqlite3, bz2, ssl, glib, pygpgme, iconv, gettext, ncurses, xml2, zlib, rpm
xml2 iconv, zlib
urlgrabber python
pygpgme python, gpgme
yum-metadata-parser python
intltool <none>
yum python, rpm, sqlite3, yum-metadata-parser, urlgrabber, pygpgme, gettext, zlib, intltool

1.4 - Notational conventions

Everything written in black on grey can be copy/pasted without any modification.

Everything written in red and between '<' and '>' symbols must be replaced by a value corresponding to your environment.

Example : when you read this :

dir=<$BASE>

you must replace <$BASE> with your base directory path in the line you will actually type.

but, when you read this :

dir=$BASE

you copy and paste it asis into your environment.

1.5 - Context

Host model : PL 820
OS : AIX 5.2
Hardware platform : Power PC
Shell : /bin/ksh93

1.5.1 - Software

Product Version Source
gcc (C & C++) 4.1.1

http://aixpdslib.seas.ucla.edu/index.html

or

http://www-03.ibm.com/systems/p/os/aix/linux/toolbox/download.html

GNU make 3.81
GNU tar any
bash 3.00.16
expat 1.95.7-4
expat-devel 1.95.7-4
XML::Parser perl module any http://search.cpan.org/~msergeant/XML-Parser-2.36/Parser.pm

2 - Building

2.1 - Environment

First, we define the install (target) base directory :

export BASE=<your install directory>

We ensure that the LIBPATH, LD_LIBRARY_PATH, and SHLIB_PATH variables are unset. Only one of them applies to this system but it is simpler to unset them all on every OS :

unset LIBPATH LD_LIBRARY_PATH SHLIB_PATH  || :

If the shell environment contains aliases for cp, rm, or mv, it is time to unalias them :

unalias cp rm mv 2>/dev/null || :

Then, we set the variables we use for every software and on every OS. Among others, we ensure that /usr/local/bin is in the PATH :

export BLIB=$BASE/libs
export PKG_CONFIG_PATH="$BLIB/common/pkgconfig"

export CC=gcc
export CPPFLAGS="-I$BLIB/common/include"
export CF_OPTS="--enable-shared --disable-static"
export PATH="$BLIB/common/bin:$PATH:/usr/local/bin:/usr/ccs/bin"

:note C compilers can have such different behaviors that I strongly recommend using gcc. If you are using another compiler, please don't ask for support.

Now, we set the 'make' commands we'll use in the rest of the document. The difference between $MK and $SMK is that, in $MK, you can add options for a 'parallel make' ('-j', '-l', ...). $SMK stands for 'serial make' and is used where parallel builds are known to fail.

Mileage varies concerning parallel make. Depending on your host characteristics and the options you set, the build can be faster or slower. Personnally, after trying several options, I did not observe a noticeable gain in performance and reverted to the default 'serial' make.

:note Please note that parallel make can cause compilation failures. So, if you get unexpected results while parallel make options are active, the first thing to do is to retry the same without the parallel options. For more info about parallel make, see the GNU make manual.

:note GNU make is mandatory as several packages absolutely require it. On Linux, GNU make is native but, on non-Linux systems, you are strongly advised not to use the native make.

export MAKE='gmake'
export SMK="$MAKE"
export MK="$MAKE"

We also un-localize the shell environment :

unset LANG LC_MONETARY LC_TIME LC_MESSAGES LC_CTYPE \
    LC_COLLATE LC_NUMERIC || :

We now set the variables specific to this operating system :

If the -brtl option is not set, the linker (ld) ignores the files with a '.so' suffix (and searches for '.a' files only).

export LDFLAGS="-L$BLIB/common/lib -Wl,-brtl"

On AIX, using /bin/sh to execute the configure scripts can be very slow, due to the huge number of useless temporary files that get created during the execution. To work around this problem, we will use bash :

export CONFIG_SHELL=`which bash`

2.2 - Creating the target structure

Every libraries and  auxiliary software are installed under '$BASE/libs'. There, each software/library is assigned a subdirectory, and we populate a 'common' subdirectory with symbolic links to the actual file locations. Using a common directory populated with symbolic links brings several benefits :

  • it allows to set a single path in the dynamic lib search path (LDFLAGS).
  • it also allows to set a single path in the include file search path (CPPFLAGS)
  • Putting a common 'bin' subdirectory as first element of the PATH ensures that anything we put in this directory will be found first when searching for an executable command.
  • Last, it allows to set a single path in the PKG_CONFIG_PATH variable.

Thanks to this directory structure, we don't have to modify the environment variables each time we add a library to the package. It becomes also easier to ensure that the compilation will reference our embedded library/include files, and not the ones that can be installed elsewhere in the system.

First, we create the base common directory.

mkdir -p $BLIB/common

Then, we create the script to populate the common directories. Every time we build and install a library we need to reference in a subsequent build, we call this script to register the files in the common structure. This is done through the $LINKDIR environment variable.

:note Note that the link creation script is not in the common PATH. This way, it cannot conflict with another file.

#-- Declare the variable we use to call the script

export LINKDIR=$BLIB/common/linkdir

#-- Create the script

echo 'cd $BLIB/$1

#-- If second argument is unset, default to the last element of target dir (usual case)

cdir=$2
[ -z "$2" ] && cdir=`basename "$1"`

#-- Directory to contain links. Create common subdirectory when needed

tdir=$BLIB/common/$cdir
[ -d $tdir ] || mkdir -p $tdir

for i in *
    do
    [ "X$i" = "X*" ] && break    #-- Empty directory
    [ "X$i" = Xpkgconfig ] && continue    #-- pkgconfig is a special case
    [ -h $tdir/$i ] && continue    # No override
    echo "Linking $i"
    ln -s ../../$1/$i $tdir/$i
done

#-- If a pkgconfig subdir exists, link it, but in the pkgconfig common subdir

[ -d pkgconfig ] && $LINKDIR $1/pkgconfig
exit 0
' >$LINKDIR
chmod +x $LINKDIR

2.3 - Embedding a private libgcc_s library

Some files we generate have a runtime dependency on the libgcc_s shared library. It means that this library must be present on the target host. In accordance with our 'black box' approach, we embed a libgcc_s library in the package and we force everyone to reference this copy.

mkdir -p $BLIB/gcc/lib
cp -p /usr/local/lib/libgcc_s.a $BLIB/gcc/lib
$LINKDIR gcc/lib

:note If you don't have a libgcc_s shared library on your system, the best solution is to rebuild gcc (gmake bootstrap builds the shared gcc lib we need).

2.4 - Compiling the stdc++ library (4.1.1)

Source : http://gcc.gnu.org

Here, you need to download gcc-core and gcc-g++. Uncompress/untar them in the same directory. Note that you will need GNU tar to read the gcc-core source package.

:note If you get errors in this part, and if you use a pre-built gcc compiler, you should first try to rebuild your gcc compiler from source code. It is not guaranteed to solve your problems but it generally did for me.

On AIX, we must compile in two steps, with different values for the OBJECT_MODE variable (the first step ends with an error).

We also run configure through bash, because the default /bin/sh is excessively slow here.

cd libstdc++-v3

#-- Work in a build sub-directory

mkdir build
cd build

#-- Configure, build, install

CXX='gcc' \
CPPFLAGS='' \
    bash ../configure --prefix=$BLIB/g++ $CF_OPTS

$MK || :    # Ignore error
OBJECT_MODE=64 $MK

$MK install "INSTALL = $PWD/../../install-sh -c"

cd $BLIB/g++
/bin/rm -rf  include

#-- Remove the 64-bit code and the pthread version

rm -rf lib/ppc64 lib/pthread

$LINKDIR g++/lib
cd ../..

2.5 - Compiling zlib (1.2.3)

Source : http://www.zlib.net/

The zlib library provides the gzip (GNU zip) compression/decompression features.

./configure --shared --prefix=$BLIB/z

$MK "LDSHARED = gcc -shared $LDFLAGS" test install

 #-- Cleanup

rm -rf $BLIB/z/share $BLIB/z/lib/*.a

$LINKDIR z/lib
$LINKDIR z/include

2.6 - Compiling the bzip2 library (1.0.1)

Source : http://www.digistar.com/bzip2/

Before starting the compile phase, we need to remove some linux-specific options from the makefile :

cp Makefile-libbz2_so Makefile-libbz2_so.orig
sed 's/-Wl,-soname -Wl,libbz2.so.1.0//' <Makefile-libbz2_so.orig >Makefile-libbz2_so

As the current makefile ignores our LDFLAGS settings, we link the shared library with our own command.

We also must install the files ourselves as the Makefile is unable to do it.

$MK -f Makefile-libbz2_so
gcc -shared -o libbz2.so.1.0.1 $LDFLAGS *.o

#-- Install the shared library and the include file

mkdir -p $BLIB/bz2/include $BLIB/bz2/lib
cp bzlib.h $BLIB/bz2/include
cp libbz2.so.1.0.1 $BLIB/bz2/lib/libbz2.so.1.0.1
ln -s libbz2.so.1.0.1 $BLIB/bz2/lib/libbz2.so.1.0
ln -s  libbz2.so.1.0 $BLIB/bz2/lib/libbz2.so.1
ln -s libbz2.so.1 $BLIB/bz2/lib/libbz2.so

$LINKDIR bz2/lib
$LINKDIR bz2/include

2.7 - Compiling the OpenSSL library (0.9.8j)

Source : http://www.openssl.org/source/

Web site : http://www.openssl.org/

We must add a '-fPIC' option to the compile options, so that the compiler produces some 'position independant code', which is mandatory when the code will be included in a shared library. We modify the Configure script :

cp Configure Configure.orig
sed 's/:aix-shared::/:aix-shared:-fPIC:/' <Configure.orig >Configure

:note As the configuration process ignores the LDFLAGS environment variable, we modify several variables related to the way the libraries will be linked.

./Configure --openssldir=$BLIB/ssl shared zlib aix-gcc

$MK "SHARED_LDFLAGS = -shared $LDFLAGS" \
    "EX_LIBS = $LDFLAGS -ldl -lz -Wl,-blibpath:$BLIB/common/lib:/usr/lib:/lib" \
    "SHAREDFLAGS = $LDFLAGS -Wl,-G,-bE:lib\$\$i.exp"

$MK install_sw

#-- Cleanup

rm -rf  $BLIB/ssl/man $BLIB/ssl/lib/*.a  $BLIB/ssl/misc $BLIB/ssl/private \
     $BLIB/ssl/certs

$LINKDIR ssl/lib
$LINKDIR ssl/include
$LINKDIR ssl/bin

# Skip the tests if the 'NO_TEST' variable is defined

[ -z "$NO_TEST" ] && $MK test "EX_LIBS = $LDFLAGS -ldl -lz -Wl,-blibpath:$BLIB/common/lib:/usr/lib:/lib"

2.8 - Compiling the iconv library (1.12)

Source : ftp://ftp.gnu.org/gnu/libiconv

Web site : http://www.gnu.org/software/libiconv/

We need to modify one of the Makefiles because it assumes that we are building static libraries when we are building only shared ones.

cp lib/Makefile.in lib/Makefile.in.orig
sed 's!aix\*)!naix*)!' <lib/Makefile.in.orig >lib/Makefile.in

Then, we proceed the usual way :

./configure --prefix=$BLIB/iconv \
    --disable-nls \
    $CF_OPTS

gmake
gmake install

rm -rf $BLIB/iconv/bin $BLIB/iconv/share

$LINKDIR iconv/lib
$LINKDIR iconv/include

2.9 - Compiling the beecrypt library (4.1.2)

Source : http://sourceforge.net/projects/beecrypt

:note We create a symbolic link in include/beecrypt. This way, the header files can be included as '#include <xxx.h>' or '#include <beecrypt/xxx.h>'.

:note We need to redefine the DEFAULTMUTEX symbol because it is not defined on AIX.

CPPFLAGS="$CPPFLAGS -DDEFAULTMUTEX=PTHREAD_MUTEX_INITIALIZER" \
    ./configure --prefix=$BLIB/beecrypt \
    --without-java \
    --without-python $CF_OPTS

$MK
$MK install

ln -s . $BLIB/beecrypt/include/beecrypt/beecrypt

$LINKDIR beecrypt/lib
$LINKDIR beecrypt/include/beecrypt include

2.10 - Compiling the expat library (2.0.1)

Source : http://sourceforge.net/project/showfiles.php?group_id=10127

Web site : http://expat.sourceforge.net/

We need to add a flag to force the code to be compiled in 'position independent' mode.

./configure --prefix=$BLIB/expat $CF_OPTS

$MK "CPPFLAGS = $CPPFLAGS -fPIC -DHAVE_EXPAT_CONFIG_H"
$MK install

#-- Cleanup

rm -rf $BLIB/expat/bin $BLIB/expat/man

$LINKDIR expat/lib
$LINKDIR expat/include

2.11 - Compiling the ncurses library (5.7)

Source : http://www.gnu.org/software/ncurses/

:note The configure script does not support shared libraries on AIX. So, we need to teach it...

cp configure configure.orig
sed -e "s/CC_SHARED_OPTS='unknown'/CC_SHARED_OPTS='-fPIC'/" \
    -e "s/MK_SHARED_LIB='echo unknown'/MK_SHARED_LIB='gcc -shared -o \$@ \$(LDFLAGS)'/" \
    <configure.orig >configure
chmod +x configure

:note We force the Makefiles to link through gcc, instead of calling ld directly, because we want our LDFLAGS to be used and it may contain options which are not supported by ld (as the options starting with -Wl).

./configure --prefix=$BLIB/ncurses\
    --without-ada \
    --without-debug \
    --without-cxx \
    --without-cxx-binding \
    --with-shared \
    --without-normal

$SMK 'MK_SHARED_LIB = gcc -shared -o $@ $(LDFLAGS)' all install.libs

$LINKDIR ncurses/lib
$LINKDIR ncurses/include
$LINKDIR ncurses/include/ncurses include

2.12 - Compiling pkg-config (0.23)

Source : http://pkg-config.freedesktop.org/wiki/

./configure --prefix=$BLIB/pkg-config $CF_OPTS

$MK
$MK install
export PKG_CONFIG=$BLIB/pkg-config/bin/pkg-config

2.13 - Compiling gettext (0.17)

Source : ftp://ftp.gnu.org/gnu/gettext/

Web site : http://www.gnu.org/software/gettext/gettext.html

The '-with-included-gettext' configure option forces to build and use the included libintl library, even if one is present in the system libraries. This removes a dependency to the system libraries and ensures that we use the same libintl code across all platforms.

:note Note that, even if we try to disable NLS localization everywhere it is possible, some software don't allow it to be disabled. That's why we still need to embed a libintl library.

./configure --prefix=$BLIB/gettext $CF_OPTS \
    --with-libiconv-prefix=$BLIB/iconv \
    --with-libexpat-prefix=$BLIB/expat \
    --with-libncurses-prefix=$BLIB/ncurses \
    --with-included-libxml \
    --with-included-glib \
    --with-included-libcroco \
    --disable-libasprintf \
    --with-included-gettext

$MK
$MK install

#-- Cleanup

rm -rf  $BLIB/gettext/info $BLIB/gettext/share/info \
    $BLIB/gettext/share/man $BLIB/gettext/share/doc

$LINKDIR gettext/lib
$LINKDIR gettext/include
$LINKDIR gettext/bin

2.14 - Compiling the pcre library (7.8)

Source : http://www.pcre.org/

 AIX system include files don't define the 'BOOL' type, which is used by the pcre code. So, we need to define it in an header file included by every C source file. We also enclose it in a '#ifdef/#endif' block because some files include it several times (and a typedef cannot be redefined) :

cp pcre_internal.h tmp1
( echo '#ifndef __BOOL'
echo '#define __BOOL'
echo 'typedef unsigned int BOOL;'
cat tmp1
echo '#endif' ) >pcre_internal.h

Then, we configure and build :  

./configure --prefix=$BLIB/pcre --disable-cpp $CF_OPTS

$MK install

#-- Cleanup

rm -rf $BLIB/pcre/bin/pcregrep $BLIB/pcre/bin/pcretest $BLIB/pcre/share

$LINKDIR pcre/lib

2.15 - Compiling glib (2.14.5)

Source : ftp://ftp.gtk.org/pub/glib

./configure --prefix=$BLIB/glib \
    --with-libiconv=gnu \
    --with-pcre=$BLIB/pcre \
    $CF_OPTS

$MK
$MK install

$LINKDIR glib/lib
$LINKDIR glib/include/glib-2.0 include
$LINKDIR glib/lib/glib-2.0/include

/bin/rm -r $BLIB/glib/share/gtk-doc $BLIB/glib/share/man $BLIB/glib/share/locale

2.16 - Compiling the popt library (1.13)

Source : http://freshmeat.net/projects/popt/

./configure --prefix=$BLIB/popt \
    --with-libiconv-prefix=$BLIB/iconv \
    --disable-nls \
    $CF_OPTS

$MK "libpopt_la_LIBADD = -liconv"
$MK install

#-- Cleanup

rm -rf $BLIB/popt/share/man

$LINKDIR popt/lib
$LINKDIR popt/include

2.17 - Compiling libffi (3.0.4)

Source : http://sources.redhat.com/libffi/

./configure --prefix=$BLIB/ffi $CF_OPTS

$MK

$MK install INSTALL=$PWD/install-sh

$LINKDIR ffi/lib
$LINKDIR ffi/include

/bin/rm -r $BLIB/ffi/share
mv $BLIB/ffi/lib/libffi-*/include $BLIB/ffi
rmdir $BLIB/ffi/lib/libffi-*

2.18 - Compiling the sqlite3 library (3.5.7)

Source : http://www.sqlite.org (download sqlite-amalgamation-3.5.7.tar.gz)

As Sqlite does not install a 'pkg-config' information file, we add it. It is not mandatory but suppresses several warning messages in subsequent builds.

cat >sqlite3.pc.in <<EOF
prefix=@prefix@
exec_prefix=@exec_prefix@
libdir=@libdir@
includedir=@includedir@

Name: Sqlite3
Description: Sqlite3
Version: @VERSION@
Libs: -L@libdir@ -lsqlite3
Cflags: -I @includedir@
EOF

Then, we build and install (we keep the library and include files only) :

./configure --prefix=$BLIB/sqlite3 $CF_OPTS

$MK
$MK install
/bin/rm -r $BLIB/sqlite3/bin

#-- Build and install pkgconfig file

./config.status --file=sqlite3.pc
mkdir $BLIB/sqlite3/lib/pkgconfig
cp sqlite3.pc $BLIB/sqlite3/lib/pkgconfig

$LINKDIR sqlite3/lib
$LINKDIR sqlite3/include

2.19 - Compiling gnupg (1.4.8)

Source : http://gnupg.org/

./configure --prefix=$BLIB/gnupg \
    --with-zlib=$BLIB/z \
    --with-libiconv-prefix=$BLIB/iconv \
    --disable-nls \
    --without-ldap \
    --without-curl \
    $CF_OPTS

$MK
$MK install

#-- Cleanup

rm -r $BLIB/gnupg/share/man $BLIB/gnupg/share/info

$LINKDIR gnupg/bin

2.20 - Compiling libgpg-error (1.6)

Source : http://gnupg.org/download/index.en.html

./configure --prefix=$BLIB/gpg-error \
    --with-libiconv-prefix=$BLIB/iconv \
    --disable-nls \
    --disable-languages \
    $CF_OPTS

$MK
$MK install

$LINKDIR gpg-error/lib
$LINKDIR gpg-error/include

2.21 - Compiling gpgme (1.1.4)

Source : http://gnupg.org/download/index.en.html

./configure --prefix=$BLIB/gpgme \
    --with-gpg-error-prefix=$BLIB/gpg-error \
    --with-gpg=$BLIB/gnupg/bin/gpg \
    --disable-gpg-test $CF_OPTS

$MK
$MK install
$LINKDIR gpgme/lib
$LINKDIR gpgme/include

2.22 - Compiling the gdbm library (1.8.3)

Source : http://www.gnu.org/software/gdbm/

./configure --prefix=$BLIB/gdbm

$MK
$MK install

$LINKDIR gdbm/lib

/bin/rm -rf $BLIB/gdbm/info $BLIB/gdbm/man

2.23 - Compiling python (2.5.2)

Source : http://www.python.org

The configure process detects a buggy getaddrinfo() function on AIX. So, we must disable IP V6 features.

We also have to compile in several steps. This is probably not the cleanest way to do it, but it is the only one that worked for me.

CPPFLAGS="$CPPFLAGS -fPIC" \
    LINKFORSHARED=' ' \
    BLDSHARED="gcc -shared $LDFLAGS" \
    LDSHARED="gcc -shared $LDFLAGS" \
        ./configure --prefix=$BLIB/python --with-gcc \
            --with-system-ffi --disable-ipv6 $CF_OPTS

#-- First, build a shared python library

$MK libpython2.5.so SHLIBS="-ldl -lpthread"
mkdir -p $BLIB/python/lib
cp libpython2.5.so $BLIB/python/lib
$LINKDIR python/lib

#-- Then, build a dynamically linked python executable

$MK python
gcc $LDFLAGS Modules/python.o -o python -lpython2.5 -ldl

#-- Now, build and install the rest

$MK "LDFLAGS=$LDFLAGS -lm"
$MK install

#-- Fix the libraries (.a vs .so)

cd $BLIB/python/lib
rm libpython2.5.so
mv libpython2.5.a libpython2.5.so
cd $SRC_DIR/build/Python-2.5.2

#-- Cleanup

/bin/rm -rf $BLIB/python/lib/python2.5/test $BLIB/python/lib/python2.5/config/*.a

#-- We don't need these modules

/bin/rm -rf $BLIB/python/lib/python2.5/lib-tk $BLIB/python/lib/python2.5/lib-dynload/_tkinter.* \
    $BLIB/python/lib/python2.5/lib-dynload/dbm.*

export PYTHON=$BLIB/python/bin/python

$LINKDIR python/lib
$LINKDIR python/include/python2.5 include

2.24 - Compiling the xml2 library (2.7.2)

Source : http://xmlsoft.org/

./configure --prefix=$BLIB/xml2 \
    --with-iconv=$BLIB/iconv \
    --with-zlib=$BLIB/z \
    --without-python \
    $CF_OPTS

$MK
$SMK install

#-- Cleanup

rm -rf $BLIB/xml2/man $BLIB/xml2/share $BLIB/xml2/bin/xmlcatalog \
    $BLIB/xml2/bin/xmllint

$LINKDIR xml2/lib
$LINKDIR xml2/bin
$LINKDIR xml2/include/libxml2 include

#-- Build/install python bindings

cd python
$PYTHON setup.py install
cd ..

2.25 - Compiling urlgrabber (3.0.0)

Source : http://linux.duke.edu/projects/urlgrabber/

$PYTHON setup.py install

2.26 - Compiling pygpgme (0.1)

Source : http://pypi.python.org/pypi/pygpgme

$PYTHON setup.py install

2.27 - Compiling yum-metadata-parser (1.1.2)

Source : http://yum.baseurl.org/

We must add the path to the pkg-config command, as setup.py is unable to use the $PKG_CONFIG environment variable.

PATH="`dirname $PKG_CONFIG`:$PATH" $PYTHON setup.py install

2.28 - Compiling intltool (0.34.2)

Source : http://www.linuxfromscratch.org/blfs/view/stable/general/intltool.html

./configure --prefix=$BLIB/intltool

$MK
$MK install

$LINKDIR intltool/bin

2.29 - Compiling yum (3.2.13)

Source : http://yum.baseurl.org

Most of the yum Makefiles are too Linux-specific, so we have to build it 'by hand'. We also cannot use the install command because it is not compatible with the Linux install syntax.

We also modify the python code to reflect the fact that we don't install in '/'.

cp_compile()
{
for i in $1/*.py
    do
    cp_filter 644 $i $2/`basename $i`
done

$PYTHON -c "import compileall; compileall.compile_dir('$BASE/$2', 1, '$BASE/$2', 1)"
}

#--------------

cp_filter()
{
mkdir -p `dirname $BASE/$3`

#-- Set the installation dir (we also use this for createrepo later)

sed \
    -e "s;^#!/usr/bin/python;#!$PYTHON;" \
    -e "s!/usr/share/yum-cli!$BASE/share/yum-cli!g" \
    -e "s!/usr/share/createrepo!$BASE/share/createrepo!g" \
    -e "s!\(/etc/yum/\)!$BASE\1!g" \
    -e "s!\(/etc/rpm/\)!$BASE\1!g" \
    -e "s!\(/var/lib/yum\)!$BASE\1!g" \
    -e "s!\(/var/log/\)!$BASE\1!g" \
    -e "s!\(/var/cache/\)!$BASE\1!g" \
    -e "s!\(/usr/share/yum-plugins\)!$BASE/share/yum-plugins!g" \
    -e "s!\(/usr/lib/yum-plugins\)!$BASE/share/yum-plugins!g" \
    -e "s!\(/usr/sbin/yum\)!$BASE/bin/yum!g" \
    -e "s!\(/etc/yum\.conf\)!$BASE/etc/yum/yum.conf!g" \
    <$2 >$BASE/$3

chmod $1 $BASE/$3
}

#--------------

XGETTEXT="$BLIB/gettext/bin/xgettext" $MK -C po

#-- Install

cp_compile . share/yum-cli

cp_filter 755 bin/yum.py bin/yum
cp_filter 755 bin/yum-updatesd.py sbin/yum-updatesd

mkdir -p $BASE/var/cache/yum $BASE/var/lib/yum $BASE/var/log

for i in rpmUtils yum
    do
    cp_compile $i libs/python/lib/python2.5/site-packages/$i
done

mkdir -p $BASE/etc/yum/repos.d

cp_filter 644 etc/yum.conf etc/yum/yum.conf
cp_filter 644 etc/yum.logrotate etc/logrotate.d/yum
cp_filter 755 etc/yum-updatesd.init etc/rc.d/init.d/yum-updatesd
cp_filter 755 etc/yum-updatesd-dbus.conf etc/dbus-1/system.d/yum-updatesd.conf
cp_filter 755 etc/yum-updatesd.conf etc/yum/yum-updatesd.conf

cp_filter 644 docs/yum-shell.8 share/man/man8/yum-shell.8
cp_filter 644 docs/yum-updatesd.8 share/man/man8/yum-updatesd.8
cp_filter 644 docs/yum-updatesd.conf.5 share/man/man5/yum-updatesd.conf.5
cp_filter 644 docs/yum.8 share/man/man8/yum.8
cp_filter 644 docs/yum.conf.5 share/man/man5/yum.conf.5

cd po
for n in *.po
    do
    l=`basename $n .po`
    mo=$l.mo
    [ -f $mo ] || continue
    mkdir -p $BASE/share/locale/$l/LC_MESSAGES
    cp $mo $BASE/share/locale/$l/LC_MESSAGES/yum.mo
done
cd ..

2.30 - Compiling createrepo (0.9.5)

Source : http://createrepo.baseurl.org/

This tool complements yum as it allows to create a yum repository (it extracts the needed metadata from a set of rpm files and stores it, so that yum can access it when needed).

Here, we use the shell functions we defined for yum.

cp_filter 755 ./genpkgmetadata.py share/createrepo/genpkgmetadata.py
cp_filter 755 ./modifyrepo.py share/createrepo/modifyrepo.py
cp_filter 755 bin/createrepo bin/createrepo
cp_filter 755 bin/modifyrepo bin/modifyrepo

for i in createrepo
    do
    cp_compile $i libs/python/lib/python2.5/site-packages/$i
done

cp_filter 644 docs/createrepo.8 share/man/man8/createrepo.8
cp_filter 644 docs/modifyrepo.1 share/man/man1/modifyrepo.1

2.31 - Compiling RPM (5.0.2)

Source : http://rpm5.org/

On AIX, we need to modify a system include file because the gcc compiler is not as tolerant as the AIX native cc_r compiler. Edit /usr/include/sys/mman.h and replace :

extern void *mmap(void *, size_t, int, int, int, off_t);

with :

extern void *mmap(void *, size_t, int, int, int, off64_t);

(replace off_t with off64_t).

The default build is unable to install remote packages (specified as an http:// or ftp:// argument). A workaround is to force ufdio mode for every file access :

#-- Replace every occurence of %{?_rpmgio}

for i in lib/rpmgi.c rpmio/rpmio.c
    do
    sed 's!..._rpmgio.!.ufdio!g' <$i >tmp
    cp tmp $i
done

#-- Replace every occurence of .fdio with .ufdio

for i in `find . -type f  -name '*.c'` macros.in
    do
    fgrep '.fdio' $i >/dev/null || continue
    cp $i $i.orig
    sed 's!\.fdio!.ufdio!g' <$i.orig >$i
done

:note We use sqlite as the default API to access the rpm DB, because of issues with db4/mmap(), especially on HP-UX.

CPPFLAGS="$CPPFLAGS -I $BLIB/python/include/python2.5" \
    PATH="$BLIB/python/bin:$PATH" \
    ./configure --prefix=$BASE \
        --with-libiconv-prefix=$BLIB/iconv \
        --disable-nls \
        --with-zlib=$BLIB/z \
        --with-bzip2=$BLIB/bz2 \
        --with-beecrypt=$BLIB/beecrypt \
        --with-openssl=$BLIB/ssl \
        --with-popt=$BLIB/popt \
        --with-dbapi=sqlite \
        --with-sqlite=$BLIB/sqlite3 \
        --without-selinux \
        --without-perl \
        --without-pthreads \
        --with-python \
        --with-python-inc-dir=$BLIB/python/include/python2.5 \
        --with-python-lib-dir=$BLIB/python/lib/python2.5 \
        --with-path-cfg=$BASE/etc/rpm \
        $CF_OPTS \
            2>&1 | tee configure.log

#-- Fix mntctl() detection (AIX specific)

echo '#define HAVE_MNTCTL 1' >>config.h

$MK
$MK install

If we don't comment out the '%buildroot' macro, the binary builds ('-bb' flag) fail with a 'file not found' error :

sed -e 's/^\(%buildroot.*\)$/#\1/' <$BASE/lib/rpm/macros >/tmp/t1
cp /tmp/t1 $BASE/lib/rpm/macros

We also comment out the '%__check_files' macro, as it is useless without '%buildroot'.

sed -e 's/^\(%__check_files.*\)$/#\1/' <$BASE/lib/rpm/macros >/tmp/t1
cp /tmp/t1 $BASE/lib/rpm/macros

The next step is optional and sets some personal preferences :

  • As I work with several operating systems and several versions, I prefer the rpm files I build to be named with an explicit OS and version. So, I modify the _build_name_fmt definition.
  • By default, rpm is configured to display the architecture field when replying to an 'rpm -q' query. I prefer the same behavior as on Linux, where only the names, versions, and releases are displayed.
  • I don't want requires/provides entries to be auto-generated by rpmbuild. The requires and provides elements I want to be associated with my packages are the ones I explicitely mention in the spec files and that's enough.

sed \
    -e 's/^\(%_build_name_fmt\).*$/\1 %%{NAME}_%%{VERSION}-%%{RELEASE}_AIX_5.2.%{_arch}.rpm/' \
    -e 's/^\(%_query_all_fmt.*\)\....arch.$/\1/' \
    -e 's/^\(%_use_internal_dependency_generator\).*$/\1 0/' \
    -e 's/^\(%_[^ ]*_provides\).*$/\1 %{nil}/' \
    -e 's/^\(%_[^ ]*_requires\).*$/\1 %{nil}/' \
    -e 's/^\(%_disable_shell_interpreter_deps\).*$/\1 1/' \
        <$BASE/lib/rpm/macros >/tmp/t1

cp /tmp/t1 $BASE/lib/rpm/macros

2.32 - Creating the RPM database

$BASE/bin/rpm --initdb

2.33 - Initializing the database

Now, we populate the database with some software supposed to be present on every host.

The first step is to define a shell function to register a virtual package. This function receives the description of a package in spec format, and registers it in the rpm database :

virtual_rpm()
{
tmp=.t$$

#-- Create the temporary spec file

cat >/tmp/$tmp.spec

#-- Build the package

$BASE/bin/rpmbuild --define="_rpmdir /tmp" \
    --define="_build_name_fmt $tmp.rpm" \
    -bb /tmp/$tmp.spec

#-- Put the package into the DB

$BASE/bin/rpm -U --justdb --ignoresize /tmp/$tmp.rpm

#-- Cleanup

/bin/rm -rf /tmp/$tmp.spec /tmp/$tmp.rpm
}

Now, we register our virtual package :

(
echo "Summary: System dependencies"
echo "Name: sysdeps"
echo "Version: 1.0"
echo "Release: 1"
echo "Group: System Environment/Libraries"
echo "License: none"
echo "Source: none"

for i in /bin/sh /sbin/sh /bin/csh /bin/ksh /bin/bash /bin/zsh /bin/tcsh \
/usr/bin/sh /usr/bin/csh /usr/bin/ksh /usr/bin/bash /usr/bin/zsh \
/usr/bin/tcsh
do
[ -f $i ] && echo "Provides: $i"
done

echo "%description"
echo "Virtual package"
echo "%files"
) | virtual_rpm

We also register the RPM software itself in its own database :

virtual_rpm <<EOF
Summary: rpm
Name: rpm
Version: 2.1
Release: 1
Group: System Environment/Base
License: none
Source: none
%description
rpm package
%files
$BASE
EOF

You can use the virtual_rpm function again to register other files and directories specific to your environment.

3 - Checking

To check that your RPM/YUM package is functional, here are some typical commands you may run and the results to expect (<nothing> means that nothing should be displayed):

# export PATH="$BASE/bin:$PATH"
# rpm -qa

sysdeps-1.0.1
rpm-2.1-1

# rpm -qla | grep $BASE/bin

<$BASE>
/bin
<$BASE>/bin/createrepo
<$BASE>/bin/gendiff
<$BASE>/bin/modifyrepo
<$BASE>/bin/rpm
<$BASE>/bin/rpm2cpio
<$BASE>/bin/rpmbuild
<$BASE>/bin/rpmconstant
<$BASE>/bin/yum

# rpm -qa --qf '%{NAME} %{VERSION} %{SUMMARY}\n'

sysdeps 1.0 System dependencies
rpm 2.1 rpm

# rpm -qi sysdeps

Name        : sysdeps                      Relocations: (not relocatable)
Version     : 1.0                               Vendor: (none)
Release     : 1                             Build Date: Fri Apr  3 15:57:13 2009
Install Date: Fri Apr  3 15:57:14 2009      Build Host: myserver
Group       : System Environment/Libraries  Source RPM: sysdeps-1.0-1.src.rpm
Size        : 0                                License: none
Signature   : (none)
Summary     : System dependencies
Description :
Virtual package

# rpm -q --provides sysdeps

/bin/sh
/sbin/sh
/bin/csh
/bin/ksh
/usr/bin/sh
/usr/bin/csh
/usr/bin/ksh
sysdeps = 1.0-1

# rpm -q --requires sysdeps

<nothing>

# yum list

Installed Packages
rpm.ppc                                 2.1-1                  installed      
sysdeps.ppc                             1.0-1                  installed

Of course, these basic tests are not exhaustive, but they at least ensure that the basic mechanisms are OK : the RPM database is accessible, the sysdeps and rpm virtual packages are correctly registered, the provides/requires engine is functional, and yum correctly communicates with the RPM layer. It is quite basic but, speaking for my own case, everytime I had problems with an RPM build, even these basic tests did not work. So, if you get there, you should be quite confident in your RPM/YUM software.

4 - Packaging

4.1 - Tar/gzip

A tar package just contains every files under $BASE.

4.1.1 - Creating the package

The commands to build a tar/gz package are :

cd $BASE
tar cf - . | gzip --best >/tmp/rpm-2.1-1.ppc.tgz

This creates a '.tgz' package file in /tmp.

4.1.2 - Installing the package

To install the package :

  • Download the package file on the target host
  • Create the $BASE directory
  • Ensure the file system containing the $BASE directory has enough free space
  • Untar/uncompress the package file in $BASE.

5 - The end

:love Enjoy !

 

Please login or register to add a comment