RPM Installation Guide
RPM
RPM [1], RPM Packet Manager (formerly the Red Hat Package Manager), tracks and manipulates the applications installed on the system. It takes care of the laborious process of software installation by making it easier to upgrade the package with minor changes in the specification (SPEC) file. The clean uninstall or erase is possible with a single command (rpm -e). If you need to frequently install/upgrade (rpm -i/rpm -U) the recent version of the software, this is the recommended way of installing and maintaining your software in your HPC home directory. Refer to RPM documentation [1] for details.
Important Notes
Pre-reqisite: Look for HPC Software Installation Guide. This helps you to debug the package during installation going to the SOURCE directory in case you encounter issues.
As a user, you can not apply "yum" to install RPMS
The required files can be downloaded from this site for the demo. For HPC users, the path for the files is /usr/local/doc/RPM
RPM Installation
Package Directories
Create a "RPM installation root directory" in your HPC Home.
mkdir -p $HOME/.usr/local/RPMS
Create a repository for Source Packages in RPM installation root directory and create specific sub-directories (RPMS, SRPMS, SOURCES, BUILD, and SPEC) for each package. For demo purpose, replace the <package-name> with ismrmrd. We are going to build, install, and remove ISMRMRD package.
mkdir -p $HOME/.usr/local/RPMS/src/<package-name>
mkdir $HOME/.usr/local/RPMS/src/<package-name>/{BUILD,SOURCES,SPEC,SRPMS,RPMS}
The package sub-directories have sensible names:
BUILD to store unpacked source files and temporary files during config/build operation
SOURCES for you to store the source file(s), patches or icon files, usually compressed ones (.zip, .tar.gz, etc)
SPEC where you create specification file (.spec) to control the RPM build
SRPMS where a source RPM package is created after successful build so as to make it possible to install it in other plaforms (different architecture and Operating Systems)
RPMS which has platform specific binary RPMs to be installed in the same platform
Load the dependency packages.
module load <pckage-name>
You can see the available package by running "module spider"
RPM Database Relocation & Initialization
When a package is installed, upgraded, or erased, RPM keeps track of this information through RPM database; else it wouldn't have been able to perform these operations correctly. You can use this database to query (rpm -q ) about the packages.To make it accessible to users, it needs to be relocated to user's space and then initialized.
Create RPM database, .db directory in RPM Installation root directory:
mkdir $HOME/.usr/local/RPMS/.db
Initialize the database
rpm --initdb --dbpath $HOME/.usr/local/RPMS/.db
Build RPM
Let's try to build the ISMRMRD package as a demo.
Change directory to your package directory:
cd $HOME/.usr/local/RPMS/src/<package-name>
It is assumed that you created a package directory ismrmrd along with its sub-directories following the section "Package Directories".
Download the SPEC file "ismrmrd-1.1.spec" from this site (all the way down) and include it in the SPEC directory (for non-HPC users). The SPEC file is heavily documented to provide you better insight about the build process. Please go through it. You can also use the uploaded spec files "ismrmrd-1.1.spec" and "openmpi-1.8.5.spec" as a template for installing other packages.
cp /usr/local/doc/RPMS/ismrmrd-1.1.spec SPEC/ # for HPC users
For the module section of the spec file to work, you need to include the "app" file included in this site and keep it at /home/<caseID>/imp/ after creating imp directory as showed if it has not already been created.
mkdir $HOME/imp
cp /usr/local/doc/RPM/app $HOME/imp # for HPC Users
Download the .zip file (Download ZIP) from github site https://github.com/ismrmrd/ismrmrd into SOURCES directory
cd SOURCES
wget https://github.com/ismrmrd/ismrmrd/archive/master.zip
It has master as a package name and it does not show any version. So, let's fix it as indicated in the SPEC file (Name ismrmrd & Version:1.1) and let's be back to ismrmrd package directory
mv master.zip ismrmrd-1.1.zip
You can view the files inside the compressed file as:
unzip -l ismrmrd-1.1.zip # for zip file
tar -tvf openmpi-1.8.5.tar.gz # for .tar and .tar.gz file
output:
0 07-30-2015 11:05 ismrmrd-master/ # root directory name for ismrmrd
8635 07-30-2015 11:05 ismrmrd-master/CMakeLists.txt # need cmake installation
drwxrwxr-x mpiteam/projects 0 2015-05-05 11:30 openmpi-1.8.5/ # root directory name for opnmpi
-rwxrwxr-x mpiteam/projects 11085324 2015-05-05 11:28 openmpi-1.8.5/configure #Configure
This is important to have correct entry (package Name) in the spec file.
Go to previous directory
cd ..
Load the dependency modules. More on the "Package Dependency" section.
module load depends
module load cmake
module load hdf5
module load boost
Build the package
rpmbuild -ba SPEC/ismrmrd-1.1.spec
ISMRMRD uses CMAKE build process. Try another spec file "openmpi-1.8.5.spec" for OPENMPI package (http://www.open-mpi.org/software/ompi/v1.8/) that uses configure/make/make-install process. For the build section of SPEC file, refer to Software Installation Guide @HPC.
After the successful build, check the files:
ls $HOME/.usr/local/RPMS/src/ismrmrd/SRPMS $HOME/.usr/local/src/ismrmrd/RPMS/x86_64/
output:
ismrmrd-1.1-1.el6.src.rpm
ismrmrd-1.1-1.el6.x86_64.rpm
Also, check the bin/bin/include and other files in the installation directory:
rpm -qlp RPMS/x86_64/ismrmrd-1.1-1.el6.x86_64.rpm
output:
/home/<caseID>
...
/home/<caseID>/.usr/local/ismrmrd/1.1/bin
...
/home/<caseID>/.usr/local/ismrmrd/1.1/include/ismrmrd
/home/<caseID>/.usr/local/ismrmrd/1.1/lib
Install RPM
You can install the RPM built for your architecture now.
rpm -ivh --force --nodeps RPMS/x86_64/ismrmrd-1.1-1.el6.x86_64.rpm --dbpath $HOME/.usr/local/RPMS/.db
output:
Preparing... ########################################### [100%]
1:ismrmrd ########################################### [100%]
Query RPMs
rpm -qa --dbpath $HOME/.usr/local/RPMS/.db
output:
openmpi-1.8.5-1.el6.x86_64
ismrmrd-1.1-1.el6.x86_64
Check the complete details along with full path using repoquery
repoquery -l openmpi --dbpath $HOME/.usr/local/RPMS/.db
Module Section
The spec file also has a section to create a module to setup the package environment variables for you avoiding entries in .bashrc file. Check it out. You may need to log out and then log in.
module avail ismrmrd
output:
-- /home/<caseID>/.usr/local/share/modulefiles --
ismrmrd/1.1(default)
Include the dependency modules that you have included before the build in the module file as:
#%Module1.0
module-whatis "ISMRMD - MRI Image format"
module load hdf5
module load boost
...
You can set the package environment by loading the module:
module load ismrmrd
Uninstall RPM
rpm -e ismrmrd-1.1-1.el6.x86_64 --dbpath $HOME/.usr/local/RPMS/.db
The symlinks may still be there as residues but they won't take any space.
Package Dependency
In the spec file, ismrmrd-1.1.spec, the dependencies for cmake and hdf5 have been included as PATH under build section:
export PATH=/usr/local/cmake/2.8.10.2/bin:/usr/local/hdf5/1.8.11/bin:$PATH
It is not a good practice. Users need to be prompted about the dependencies as soon as they start building the package as showed.
rpmbuild -ba SPEC/1.1
errors:
sh: line 0: fg: no job control
error: Failed build dependencies:
cmake is needed by ismrmrd-1.1-1.el6.x86_64
hdf5 >= 1.8.11 is needed by ismrmrd-1.1-1.el6.x86_64
So, in the spec file you can include the following in the definition section.
# Dependency to build packages
BuildRequires: cmake, hdf5 >= 1.8.11
# Dependency for the built RPM
# Requires: hdf5 >= 1.8.11
You should install the required packages and load the modules before building the package as we are not using the default RPM installation path.
module load cmake
module load hdf5
Also, you need to specify the dbpath to the packages to build.
rpmbuild -ba --dbpath $HOME/.usr/local/RPMS/.db SPEC/1.1
Sometimes, the module files may have been missing compiler and library flags to build the package properly. In that case, we need to make use of package config (.pc) files. The format for .pc file that was manually created for hdf5 looks like:
Name: HDF5
Description: File Format
Version: 1.8.11
URL:
#
prefix=/usr/local/hdf5/1.8.11
exec_prefix=${prefix}
includedir=${prefix}/include
libdir=${exec_prefix}/lib
#
Libs: -L${libdir} -lhdf5
Cflags: -I${includedir}
Note that all RPM installation does not provide .pc files. The .pc file if available is accessed through the environment variable PKG_CONFIG_PATH. For example, in the module file, it is pre-pended as:
prepend-path PKG_CONFIG_PATH /usr/local/hdf5/1.8.11/lib/pkgconfig
Now, after loading the hdf5 module, when you issue pkg-config command, it retrieves the information from that metadata .pc file. Here, hfd5.pc does not include any flags.
pkg-config --cflags --libs hdf5
output:
-I/usr/local/hdf5/1.8.11/include -L/usr/local/hdf5/1.8.11/lib -lhdf5
The pkg-config command can also be used as build dependency as showed
BuildRequires: pkgconfig(hdf5-1.8.11)
FYI, you can also compile the code with hdf5 dependency using:
gcc -o hdf5test testhdf5.c $(pkg-config --libs --cflags hdf5)
References:
[2] Maximum RPM: http://www.rpm.org/max-rpm/index.html