Managing multiple software versions with Org-mode
Introduction
Have you ever had to deal with multiple versions of the same C++ software? Right now I am dealing with those questions: Where should I build the binaries? How can I keep track of which version? How can I keep track of which configuration options? Where should I install it? How to link to the right libraries with a specific version? A lot to think about and keep track of. Fortunately, Org mode makes things a bit easy (maybe even fun).
In this post, I will go to explain my workflow on how to manage this situation with Org-mode. The particular software that I am dealing with is Trilinos. Therefore, another title for this post could be "installing Trilinos for absolute beginners".
Keeping track of configuration settings and versions with :tangle
I like consistency and order (who doesn't?). To make this organized, I keep track of configuration files within a code block. This code block is then exported (tangled) to a specific folder.
For example, let's say we want to install Trilinos1 on this specific release:
1cd ~/.local/src/Trilinos
2git describe --tagtrilinos-release-13-4-1
Then we create a build folder inside the source directory2 build/trilinos-release-13-4-1/ where we place our specific configuration3.
Notice that, in the CMake configuration we specify the CMAKE_INSTALL_PREFIX, which is where the installation files will be copied to.
I use /.opt for prefixed installations4.
1set(CMAKE_INSTALL_PREFIX "/home/nasser/.opt/trilinos-release-13-4-1" CACHE PATH "Set in my config.cmake" FORCE)
2set(Tpetra_INST_INT_INT ON CACHE BOOL "Set in my config.cmake")
3set(Trilinos_ENABLE_TESTS ON CACHE BOOL "Set in my config.cmake")
4set(Trilinos_ENABLE_Amesos ON CACHE BOOL "Set in my config.cmake")
5set(Trilinos_ENABLE_Amesos2 ON CACHE BOOL "Set in my config.cmake")
6set(Amesos2_ENABLE_TESTS ON CACHE BOOL "Set in my config.cmake")
7set(Trilinos_ENABLE_Epetra ON CACHE BOOL "Set in my config.cmake")
8set(Trilinos_ENABLE_EpetraExt ON CACHE BOOL "Set in my config.cmake")
9set(Trilinos_ENABLE_Ifpack ON CACHE BOOL "Set in my config.cmake")
10set(Trilinos_ENABLE_AztecOO ON CACHE BOOL "Set in my config.cmake")
11set(Trilinos_ENABLE_Sacado ON CACHE BOOL "Set in my config.cmake")
12set(Trilinos_ENABLE_SEACAS OFF CACHE BOOL "Set in my config.cmake")
13set(Trilinos_ENABLE_MueLu ON CACHE BOOL "Set in my config.cmake")
14set(Trilinos_ENABLE_ML ON CACHE BOOL "Set in my config.cmake")
15set(Trilinos_ENABLE_ROL ON CACHE BOOL "Set in my config.cmake")
16set(Trilinos_ENABLE_Belos ON CACHE BOOL "Set in my config.cmake")
17set(Belos_ENABLE_EXAMPLES ON CACHE BOOL "Set in my config.cmake")
18set(Belos_ENABLE_TESTS ON CACHE BOOL "Set in my config.cmake")
19set(Trilinos_ENABLE_Ifpack2 ON CACHE BOOL "Set in my config.cmake")
20set(Kokkos_ENABLE_SERIAL ON CACHE BOOL "Set in my config.cmake")
21set(Trilinos_ENABLE_Teuchos ON CACHE BOOL "Set in my config.cmake")
22set(rilinos_ENABLE_Tpetra ON CACHE BOOL "Set in my config.cmake")
23set(Tpetra_ENALE_DEPRECATED_CODE ON CACHE BOOL "Set in my config.cmake")
24set(Tpetra_ENABLE_TESTS ON CACHE BOOL "Set in my config.cmake")
25set(Trilinos_ENABLE_Xpetra ON CACHE BOOL "Set in my config.cmake")
26set(Xpetra_ENABLE_Experimental ON CACHE BOOL "Set in my config.cmake")
27set(Xpetra_ENABLE_Kokkos_Refactor ON CACHE BOOL "Set in my config.cmake")
28set(Xpetra_ENABLE_Epetra ON CACHE BOOL "Set in my config.cmake")
29set(Trilinos_ENABLE_Galeri ON CACHE BOOL "Set in my config.cmake")
30set(Galeri_ENABLE_Xpetra ON CACHE BOOL "Set in my config.cmake")
31set(Trilinos_ENABLE_COMPLEX_DOUBLE ON CACHE BOOL "Set in my config.cmake")
32set(Trilinos_ENABLE_COMPLEX_FLOAT ON CACHE BOOL "Set in my config.cmake")
33set(Trilinos_ENABLE_Zoltan ON CACHE BOOL "Set in my config.cmake")
34set(Trilinos_ENABLE_PyTrilinos OFF CACHE BOOL "Set in my config.cmake")
35set(Trilinos_VERBOSE_CONFIGURE OFF CACHE BOOL "Set in my config.cmake")
36set(TPL_ENABLE_MPI ON CACHE BOOL "Set in my config.cmake")
37set( TPL_ENABLE_BLAS ON CACHE BOOL "Set in my config.cmake")
38set( TPL_ENABLE_LAPACK ON CACHE BOOL "Set in my config.cmake")
39set( TPL_ENABLE_UMFPACK ON CACHE BOOL "Set in my config.cmake")
40set(BUILD_SHARED_LIBS ON CACHE BOOL "Set in my config.cmake")
41set(CMAKE_VERBOSE_MAKEFILE OFF CACHE BOOL "Set in my config.cmake")
42set(CMAKE_BUILD_TYPE RELEASE CACHE INTERNAL "Set in my config.cmake")
43set(Trilinos_ENABLE_EXPLICIT_INSTANTIATION ON CACHE BOOL "Set in my config.cmake")
44set( Trilinos_ASSERT_MISSING_PACKAGES ON CACHE BOOL "Set in my config.cmake")
45set(CMAKE_MESSAGE_LOG_LEVEL "DEBUG" CACHE STRING "Set in my config.cmake")In this block, I use the source block header option with the tangle file specification.
#+begin_src makefile :tangle ~/.local/src/Trilinos/build/trilinos-release-13-4-1/config.cmake
Configuring and building
Now that the configuration setting is defined and properly stored, we can continue to configure and build.
1TRILINOS_BUILD_DIR=trilinos-release-13-4-1
2cd ~/.local/src/Trilinos/build/$TRILINOS_BUILD_DIR
3rm -f CMakeCache.txt
4cmake -D Trilinos_CONFIGURE_OPTIONS_FILE=config.cmake ../../ | tee config.out
5make -j32
6make installWe can check the installation by "grepping" the Trilinos version in the installation directory.
1grep -R def.*TRILINOS_VERSION_STRING ~/.opt/trilinos-release-13-4-1/include/home/nasser/.opt/trilinos-release-13-4-1/include/Trilinos_version.h:#define TRILINOS_VERSION_STRING "13.4.1"
In Org mode, I use the Detached package, which runs this shell command in a process independent of emacs. This works like an "async" feature, and the result is that it does not block emacs. It could be even used to run things on another computer.
Using the specific Trilinos version in a "downstream" application
After installing the specific version we can use it (linking/including) in our application (a.k.a downstream application). The downstream application combines the Trilinos packages to solve a problem. For instance, use Belos Krylov solver with MueLu preconditioner to solve a linear system iteratively.
Here I will use CMake to configure my build files to automatically find the libraries that I want to link to and include.
CMake prefix path
CMake config files allow CMake to find the packages with find_package function.
For Trilinos, there is this option that is turned on by default.
1grep "ENABLE.*CMAKE_CONFIG_FILES.*BOOL" build/CMakeCache.txtTrilinos_ENABLE_INSTALL_CMAKE_CONFIG_FILES:BOOL=ON
On the "downstream" project, if a nonstandard location is used for the installation, we need to tell CMake where to look for the config files with5,
1-D CMAKE_PREFIX_PATH=<where you installed>
Or in the CMakeLists.txt directly6
1list(APPEND CMAKE_PREFIX_PATH "<where you installed>")I like to keep packages source codes inside ~/.local/src. The ~/.local seems to be the standard here.
Convention: I like to build in a build/ folder in the source directory. If multiple builds are expected one can create subdirectories for each version/configuration (a.k.a "prefix").
This configuration file makes it "traceable", so you know which configuration was used.
One could also "install" in the same build folder. This would make things a little more contained. But, build files sometimes are often disposable and you want to keep just the libraries/header to be used elsewhere. That is why I put it in /.opt/, which is a local user (no need for root permission) version of ~/opt/7.