Downloading CALDB 4.1.4

CalDB 4.1.4 released 17 SEP 2009

Contents:

• Check your CalDB4 Version
• Upgrade CALDB 4.1.x to 4.1.4
• Install the CALDB 4.1.4
• Check the Index Configuration
• Verify that the CALDB is Working

Determining your current CalDB4 version

The easiest way to determine which version of the Chandra CalDB you currently have installed, is to search the last line of the caldb_version.fits file under $CALDB/docs/chandra/caldb_version/. An appropriate CIAO command for this purpose is as follows:

unix% dmlist \
$CALDB/docs/chandra/caldb_version/caldb_version.fits"[cols CALDB_VER]" \
data,clean | tail -1

Naturally you will need to have loaded CIAO 4.1.2 and have $CALDB assigned in order to do this.

If your version is 4.1.x, you may proceed with the upgrade installation procedure (next instruction).

If the version you get is NOT 4.1.x, then this upgrade procedure is inappropriate! Do not attempt to upgrade a CalDB with version 3.x.x or earlier with this upgrade procedure as it will not work. You have some options:

1. If are not aware of the CIAO 4.1.2 release: Please consider upgrading your CIAO software to CIAO 4.1.2, and then proceed with the full CalDB4 installation instructions below.

2. If you are in the process of installing CIAO 4.1.2 for the first time: You are in the wrong place; please proceed to the full CalDB4 installation instructions below.

3. If you mean maintain your CIAO 3.4 and CalDB 3.x.x installation: Proceed to the CalDB 3 downloads web page for upgrade instructions for CalDB 3.x.x.

Upgrade CalDB 4.1.x to 4.1.4

If your site has already installed CalDB 4.1.x, you may upgrade to 4.1.4 by using the "main" tar file in Table 1 of the download page. Follow instructions 1-3 in this section to upgrade your CalDB4 installation.

1. Change directories to the existing CALDB4 location.

2. Download the CalDB 4.1.4 main gzipped tar file.

3. Download the updated ACIS BKGRND files tar ball, new in the CalDB 4.1.3 release, if you have not already upgraded your CalDB to version 4.1.3 previously: acis_bkgrnd_4.1.3.tar.gz. If you have previously upgraded to version 4.1.3, then skip this file.

4. UNIX: Unpack the gzipped tar ball using gtar xvUfz. The "U" option forces gtar appropriately to reset the links to the index files and your upgrade will be properly installed.

   LINUX: You may omit the U option: gtar xvfz, or else you may get some meaningless error messages about failure to unlink non-empty directories. These errors are harmless.

   If you do not have gtar installed locally, you may use tar xvfz instead.

   unix% gtar xvUfz caldb_4.1.4_main.tar.gz ./docs ./data
   unix% gtar xvfz acis_bkgrnd_4.1.3.tar.gz
   

   NOTE: We have specified the subdirectories "./docs ./data" above because you are not doing a full CalDB4 installation. You only need to update the ./docs and ./data subdirectories in this case; there is no need to update the CalDB configuration files under ./software.

5. UPGRADE FINISHED! Proceed to Check the index configuration below.

Install the CALDB

The instructions for installing the CALDB are basically the same whether you are creating a local or sitewide installation.

1. Select a location for your CALDB at your site. This directory needs to be accessible by all users who are running CIAO 4.1 at your site.

   NOTE:If you intend to upgrade from an earlier version of the Chandra CALDB, and do not intend to use or support CIAO 3.4 on your site, you may put CalDB 4.1.4 in the same location; however CalDB 4.1.4 is a different directory structure, and is NOT compatible with CalDB 3.#.# or any earlier version. You are advised to delete the $CALDB/data/chandra directory completely before proceeding with this installation.

   If you intend to maintain CIAO 3.4/4.0 and also install CIAO 4.1, then you must install Chandra CalDB 4.1.4 in a separate directory location from CalDB 3.#.#. They cannot be maintained or upgraded in the same directory location, i.e., the same location for setting $CALDB. This is because the caldb.config file for CALDB 4.1.4 is different from that for CALDB 3.#.#.

2. Download CALDB 4.1.4 Full Installation from Table 1. There are four tar ball files in all, including the "main" tar ball.

   The ACIS Background files have been packaged separately to reduce the space requirements for CalDB when users don't need the background datasets. These background files have now been upgraded with CalDB 3.4.3, as of 02 July 2009. They add about 668 MB to the total size of the CalDB. Still, if they are necessary, they can be downloaded rather quickly and unpacked AFTER the MAIN 4.1.4 tarball has been unpacked. We clearly recommend downloading and installing these files on multi-user sites.

   Note that the PSF library files are individually gzipped, then tarred together. It is not necessary to include all these files in the installation as they require a lot of disk space (approximately 6.5 GB); see the PSF Library Manual to determine which files you will require. For sitewide installations (i.e. installations maintained by a systems administrator), it may be preferable to unpack them all if space allows.

3. From within the CALDB directory, unpack the main tar ball using gtar xvUfz

   unix% gtar xvUfz caldb_4.1.4_main.tar.gz
   

   UNIX: If you use gtar, then you should use the xvUf (include the "U" option) in case there are filename conflicts which may not be appropriately clobbered. NOTE: In LINUX, the "U" option may cause meaningless errors about failure to unlink directories that are not empty. Ignore this error, or else omit the "U" option. If you do not have access to gtar then you may use tar xvf instead without worry.

   The result will be three directory trees at that level, namely docs, data, and software:

   unix% ls
   data/           docs/           software/
   

   The docs directory contains the manifests and versioning files. while software/ contains the tools. The data directory is most important, containing the full CALDB data directory tree.

4. Unzip and untar the ACIS background package, unless you are certain that you will not need any of the blank-field files for making background datasets. Typically observers of extended sources need these files. We recommend multi-user sites to download and install these files in case anyone in your group needs them. NOTE: By unzipping the tarball in the SAME LOCATION AS THE MAIN TARBALL, you get the bkgrnd files exactly where you will want them.

   unix% gtar xvUfz acis_bkgrnd_4.1.3.tar.gz
   

5. Optional: Again from the same location as the other tarballs, untar the PSF library files. The gzipped files will be placed in the ACIS or HRC branch. The files may be used and searched for even if they are left gzipped. However, if you prefer to unzip the files (which will require 6.5 GB of extra space), move to the appropriate directory - either ./data/chandra/acis/2d_psf/ (ACIS) or ./data/chandra/hrc/2d_psf/ (HRC) - and gunzip the files individually.

6. Delete the tarfiles in order to save space. You may wish to complete the installation and verify that the CALDB is working before doing so.

Check the Index Configuration

To ensure that the index files are linked correctly, run this ls command from your CALDB directory:

unix% ls -l data/chandra/*/caldb.indx
ls -l data/chandra/*/caldb.indx
lrwxrwxrwx ... data/chandra/acis/caldb.indx -> ./index/caldbN0403.indx
lrwxrwxrwx ... data/chandra/default/caldb.indx -> ./index/caldbN0401.indx 
lrwxrwxrwx ... data/chandra/ephin/caldb.indx -> ./index/caldbN0400.indx
lrwxrwxrwx ... data/chandra/hrc/caldb.indx -> ./index/caldbN0400.indx 
lrwxrwxrwx ... data/chandra/pcad/caldb.indx -> ./index/caldbN0400.indx
lrwxrwxrwx ... data/chandra/pimms/caldb.indx -> ./index/caldbN0401.indx
lrwxrwxrwx ... data/chandra/sim/caldb.indx -> ./index/caldbN0400.indx

where "..." represents the file owner and timestamp. Check that the results from your system match those shown here.

Verify that the CALDB is Working

The first part of this step requires that you have FTOOLS available; if you do not, skip to the check_ciao_caldb example. Note that you do not need FTOOLS in order to use the CALDB with CIAO, only for this verification step.

chkcif

The FTOOL chkcif is the method for verifying the installation of a CALDB after unpacking it. chkcif ("check CALDB index file") reads the index listings and

• verifies that the index for a given instrument is present and in the proper form.
• verifies that all ONLINE files in the index are present in the CALDB. If a file that is indexed as ONLINE is missing, the tool reports it to STDOUT and then exits with an error status.
• lists the OFFLINE files as offline and does not check for them.

A file that is marked OFFLINE is not included in the CalDB directory structure. By the HEASARC standard, such a file might be available in an off-line system such as another disk drive or a web location, and isn't accessible by chkcif. For the Chandra CalDB, "OFFLINE" indicates that a file has been removed from circulation, and is NO LONGER AVAILABLE, and should not be used. Hence, when chkcif returns that a file is OFFLINE, this is normal behavior for the CalDB, and does not indicate any problem with your configuration.

Note that the tool does not verify that each calibration file is whole or uncorrupted, only that it exists in the proper location in the directory structure.

Note that you need to start CIAO in the working window (to define some environment variables) before running chkcif. In this example, CIAO is installed in /soft/ciao:

unix% source /soft/ciao/bin/ciao.csh
unix% chkcif chandra acis STDOUT
 ** chkcif 1.1.4
 ... Contents of CIF /soft/ciao/CALDB/data/chandra/acis/caldb.indx verified
 ... See STDOUT
     for more information
 ** chkcif 1.1.4 completed successfully

This command needs to be repeated for all the Chandra CALDB instrument branches, namely "hrc", "ephin", "pcad", "sim", and "default".

IMPORTANT: IF the ACIS background files have not been downloaded and unpacked, AND/OR if the ACIS PSF library has not been downloaded and unpacked, then the ACIS branch will report errors, as shown in this example. Otherwise, the tool should return the "chkcif ... completed successfully" line for each instrument, verifying that the CALDB is correctly installed. The same holds for the HRC branch if the HRC PSF library is not downloaded.

check_ciao_caldb

It is also possible to test the installation using the script "check_ciao_caldb". This routine will locate the CalDB used by your CIAO 4.1.x+ installation, re your current CalDB version, and indicate whether or not a standard test query to the CalDB has worked correctly.

First, start CIAO in the working window.

unix% check_ciao_caldb

CALDB environment variable = /data/CALDB/ciao_4.1
             CALDB version = 4.1.3
             relelase date = 2009-07-06T16:00:00
CALDB query completed successfully.

If the routine does not give you a response equivalent to the above for your machine and directory setup, you may need to reinstall your CalDB, following the Full Installation instructions above.