Rebuilding CIAO 4.15.2 source with pre-built OTS binaries on Linux

Where should CIAO be installed?

CIAO can be installed in any location and does not need to be installed with any root or super-user privileges. Users are discouraged from installing CIAO as the root user.

Off-the-shelf software

The CIAO off-the-shelf packages include adapted OTS and some dependencies on specific versions of OTS. As such, it is highly beneficial (and will save you a lot of headaches) to build CIAO with our OTS binaries. If you do need to build CIAO completely from source, contact the CXC Helpdesk.

You can view the list of the OTS and their versions here.

Operating Systems

Building CIAO with the pre-built OTS is supported on 64-bit Linux systems.

Note: we also provide instructions for building CIAO with the pre-built OTS on Mac OS X

Compilers

Building CIAO requires C and C++ compilers. We have only tested building CIAO with the GNU compilers (gcc,g++). CIAO is now compatible with the G++14 standard which is the default for newer versions of g++.

Check to see if you have GCC on your system:

% gcc --version
% g++ --version

If either gives you an error, you will have to download and install that compiler.

CIAO is built using Makefiles, so the make utility must be installed.

CIAO also requires flex|lex and yacc|bison to compile various parsers. These can be installed using your system's package manager.

Source files + OTS binaries

Building CIAO from source using the pre-built binary OTS requires the ciao-4.15-src-*.tar.gz packages as well as the platform specific ciao-4.15-bin-Linux.tar.gz binaries.

First, install CIAO using the ciao-install script. Go to the Custom Installation, choose your platform leave the other options at their default values (which is the "Standard CIAO installation"), and then change the "Source build" option to "Full Source Build". This will select all of the necessary binaries (which contain the OTS) and will download the source code. CALDB, the background event files and the science scripts package are optional; include or exclude these packages by checking the boxes next to them. Finally, click the "Download the ciao-install Script" button.

From a terminal, run the ciao-install script, which will unpack all the files for you [recommended].

% bash <path-to>/ciao-install

If you want to unpack the files yourself, use the --download-only switch:

% bash <path-to>/ciao-install --download-only

and then unpack the tar.gz packages. For example, the commands for unpacking the "Standard CIAO installation" Linux 64-bit packages are shown below.

% tar -xzf ciao-4.15-bin-core-Linux.tar.gz
% tar -xzf ciao-4.15-bin-obsvis-Linux.tar.gz
% tar -xzf ciao-4.15-bin-sherpa-Linux.tar.gz
% tar -xzf ciao-4.15-bin-tools-Linux.tar.gz
% tar -xzf ciao-4.15-contrib-3.tar.gz
% tar -xzf ciao-4.15-src-core.tar.gz
% tar -xzf ciao-4.15-src-obsvis.tar.gz
% tar -xzf ciao-4.15-src-sherpa.tar.gz
% tar -xzf ciao-4.15-src-tools.tar.gz
% tar -xzf caldb_4.10.4_main.tar.gz

Please note that the list of files to untar depends on the selections made when downloading the ciao-install script.

% mkdir CALDB
% mv docs/ data/ software/ CALDB/

Note: ciao-4.15-bin-obsvis-<plat>.tar.gz is NOT needed as no OTS are packaged in that binary distribution.

Note: If you are having trouble with the ciao-install script, refer to "Downloading the Software" in Manually Installing CIAO.

Setting the compilers

Specifying a specific path to a compiler can be achieved by using the appropriate keyword:

• CC - Name and location of the c compiler
• CXX - Name and location of c++ compiler

Usage example:

% ./configure --with-ascii --with-fits \
  CC=<path-to>/bin/gcc \
  CXX=<path-to>/bin/g++

The compiler must also be in your PATH. If <path-to-compiler> is not in $PATH, add it using

GNU++14, ABI errors

If you are rebuilding only specific directories on certain older Linux distributions (such as CentOS 7), you may need to specify the C++ standard to use:

% ./configure --with-ascii --with-fits \
  CXX="<path-to>/bin/g++ --std=gnu++14"

You will know if you need to do this if you get an error message referencing libstdc++.so.6: version `CXXABI_1.3.9' when you run the recompiled application.

Bash

% export PATH=${PATH}:<path-to-compiler>

Tcsh/Csh

% setenv PATH ${PATH}:<path-to-compiler>

Configuration flags/options

Here is the list of configure options available for CIAO:

The flags --with-ascii and --with-fits are REQUIRED for CIAO to build correctly. All other flags are optional.

Optimization and compilers

Optimization is turned on by default in the binary distribution but not when building from source. Users should use the default value, --with-opt=-O3 for GCC. If you wish to use a different optimization level or turn off optimization, use the --with-opt= switch. To turn off optimization, use --with-opt=-O0. See GNU's Optimization Options for more details.

Rebuilding CIAO

To build and install the code, type "make install". CIAO currently requires that "make install" be issued as a single command rather than as separate commands "make; make test; make install". This is due to build dependencies on installed libraries and packages.

% make install

Note that this will take a few minutes.

Rebuilding selected source code

There are a few tools that depend on local libraries that if not installed may require rebuilding first. Most src directories can be rebuilt after running configure by simply stepping into the directory and running "make install".

For example, say a user rebuilt libfftw to take advantage of their specific hardware. After placing the libfftw files in ots/lib, they'd need to re-link the tools in src/da to their system by doing

% cd src/da
% make install

to rebuild the da tools with their custom libfftw build.

Post-processing

After the build completes, run the script ciao-python-fix:

% bash bin/ciao-python-fix

This post-processes your configuration, to byte-compile included Python files for your installation, and to ensure all CIAO packages with Python interfaces can be built. Once this script is run, the CIAO installation cannot be moved or relocated to a different directory without repeating the configure and ciao-python-fix steps.

Linking to CALDB: if you downloaded CALDB and unpacked the tarfile by hand (e.g. ran ciao-install with the --download-only switch), you can link it to CIAO at this point by creating a softlink to the CALDB directory in $ASCDS_INSTALL:

% pwd
/<path-to>/ciao-4.15
% ln -s <path-to>/CALDB CALDB

Configure script

1. For other errors, check that:

   • your compilers are in your $PATH
   • ${ASCDS_INSTALL}/ots/bin is in your $PATH

Build

If the build fails at any point during "make install", check that

1. you have compatible C/C++ compilers (see Compilers for details)
2. your compilers are in your $PATH.
3. ${ASCDS_INSTALL}/ots/bin is in your $PATH.

Then, run "make -k clean", then remove the contents of lib/ and binexe/ again,

% rm -rf binexe/* lib/*

re-configure the build with

Bash

% source bin/ciao.sh -o

Tcsh/Csh

% source bin/ciao.csh -o

and then

% ./configure --with-fits --with-ascii

and run make install again. If it still fails, try opening-up a new terminal so that you have a clean environment to work with. Check that you have compatible compilers and that you have set your $PATH variable correctly defined.