Skip to the navigation links
Last modified: December 2009

URL: http://cxc-newtest.cfa.harvard.edu/ciao4.2/calindex.html
AHELP for CIAO 4.2

calindex

Context: tools

Synopsis

Create index file of calibration database (CALDB) products.

Syntax

calindex index keyconf calfile [clobber] [verbose]

Description

calindex generates an "index" file over specified calibration data source as specified by "calfile", and the index then will be served in CALDB product retrievals by other applications. The index is formated into multi-columns in FITS with extension "CIF" and it includes a set of mendatory columns of predefines and the optional ones of run-time specification. A header-key configuration table, "keyconf", allows user to extend the index table with defined column data types, data formats, and other entries. The name field, "calfile", sets access of the calibration data products for the indexing.

Example 1

calindex index=xfile.1.fits keyconf=keys_1.cfg calfile=datadir
clobber=yes verbose=0

Create index file named as "xfile.1.fits" to index all FITS files in directory, "datadir" and its recursively down directories. The header-key config file, "keys_1.cfg", is a text file and has following columns and rows

     #keys_01.cfg 
     #colName   dataType        format	hdrKey		query	null
     TELESCOP   string		a10	TELESCOP        yes     ""
     INSTRUME   string		a10     INSTRUME        yes	""
     CAL_VSD    string		a10	DATE-OBS	no	""
     CAL_VST    string		a8	DATE-OBS	no	""
     CAL_DESC   string		a70	CDES		no	""
     CAL_CLAS   string		a4	FOO		no	""

     

The lines starting with '#' are comments and will be ignored at run time. Only the first three columns (colName, dataType, and format) are interesting for index creation and the last 3 columns are served for CALDB product retrieval in other applications. What it says here is that, for example, "TELESCOP", a column name entry, has "string" data type and data format of 10 ASCII characters ("a10"). The rows of the "TELESCOP" values are extracted from all FITS files under "datadir" and then indexed in "xfile.1.fits". The other five rows of "keys_1.cfg" follow the same suit. calindex gets right CALDB FITS files by checking the file postfix, "fits" or "fit", which is case-insensitive.

In this example, the output of "xfile.1.fits" with "CIF" block is created with 14 columns, as shown below. Column #1,2,11-14 are user's addition as specified in "keys_1.cfg" above and the rest are defaults, common to any index file.

--------------------------------------------------------------------------------
Columns for Table Block CIF
--------------------------------------------------------------------------------
 
ColNo  Name                 Unit        Type             Range        Comment
   1   TELESCOP                          String[10]                   Telescope
   2   INSTRUME                          String[10]                   Instrument
   3   CAL_DEV                           String[20]                   CALDB Device
   4   CAL_DIR                           String[70]                   CALDB Directory
   5   CAL_FILE                          String[40]                   CALDB File
   6   CAL_CNAM                          String[20]                   CALDB Product
   7   CAL_CBD[9]                        String(9)                    Condition
   8   CAL_XNO                           Int2           -             Extension #
   9   CAL_QUAL                          Int2           -             Quality
  10   CAL_DATE                          String[10]                   date
  11   CAL_VSD                           String[10]                   Start Date
  12   CAL_VST                           String[8]                    Start Time
  13   CAL_DESC                          String[70]                   description
  14   CAL_CLAS                          String[4]                    class

Example 2

calindex index=xfile.1a.fits keyconf=keys_1.cfg calfile=mycaldb1+
or
calindex index=xfile.1a.fits keyconf=keys_1.cfg calfile=mycaldb1/+

Create index, "xfile.1a.fits", with the same optional columns as above example but having different data products indexed. The index rows of the output are filled from all FITS files in the directory "mycaldb1" and in its recursive down directories. The symbol, '+', following directory name and or '/' indicates a recursive readout of calibration files from the current location.

Example 3

setenv CALDB mycaldb7
calindex index=xfile.7.fits keyconf=keys_7.cfg calfile=CALDB

Create index, "xfile.7.fits" with "CALDB" to "calfile". The "CALDB" is an environmental variable and it, in the current example, is pointed to "mycaldb7+" by pre-commanding "setenv CALDB mycaldb7". In the current cae, mycaldb7 contains 3 FITS files for feeding calindex.

"keys_7.cfg" has two more rows, "CAL_VED" and "CAL_VET", added over the "keys_1.cfg", as shown below

#keys_7.cfg
#colName        dataType        format  hdrKey          query           null
TELESCOP        string          a10     TELESCOP        yes             ""
INSTRUME        string          a10     INSTRUME        yes             ""
CAL_VSD         string          a10     ""              yes             ""
CAL_VST         string          a8      ""              yes             ""
CAL_VED         string          a10     ""              yes             ""
CAL_VET         string          a8      ""              yes             ""
CAL_DESC        string          a70     CDES            no              ""
CAL_CLAS        string          a4      FOO             no              ""

Example 4

setenv CALDB mycaldb6
calindex index=xfile_6b.fits keyconf=keys_1.cfg calfile=CALDB,mycaldb2

Index file, "xfile_6b.fits", is created on directory list, CALDB and "mycaldb2". CALDB is aliased to "mycaldb6" in which there are 3 FITS files and one sub-dir containging one file. "mycaldb2" has only one fits file within. So, the output has, at least, 4 FITS files indexed.

Example 5

calindex index=xfile.5.fits keyconf=keys_1.cfg calfile=@cfile_5.lst

This example shows an alternative to assign calfile value with named file list, marked by '@' before the named file, "cfile_5.lst". Below is what the list looks like

     #cfile_5.lst
     cfile_5A.fits
     cfile_5B.fits

When calindex detects the first char, '@', of the calfile string and it opens up the file list, "cfile_5.lst", and feeds "cfile_5A.fits" and "cfile_5B.fits" in to the array of calibration files.

Example 6

calindex index=xfile.1g.fits keyconf=keys_1.cfg calfile=./*.foo

Create index, "xfile.1g.fits", to all files ending with ".foo", in the current directory.

Example 7

calindex index=xfile.1h.fits keyconf=keys_1.cfg calfile=''
or
calindex index=xfile.1h.fits keyconf=keys_01.cfg calfile="NONE"

Create index, "xfile.1h.fits", to all FITS files in and under the CALDB directory tree. Note that calindex, with any empty or 'NONE' input to 'calfile', will look into CALDB directory tree.

Example 8

setenv CALDB /data/regression_test/development/CALDB/data/chandra
calindex index=xfile.8a.fits keyconf=keys_1.cfg calfile=$CALDB/acis/cpf+

Create index, "xfile.8a.fits" to all FITS files in "$CALDB/acis/cpf" directory and its resursive subdirectories.

Example 9

calindex index=xfile.9.fits keyconf=keys_9.cfg
calfile=cfile.9A.fits,cfile.9B.fits

As usual, calindex creates "xfile.9.fits" to index two calibration files, "cfile.9A.fits" and "cfile.9B.fits". However "keys_9.cfg" adds a "MYDETNAM" column name, an incoventional entry, as shown below. With this in check, calindex geneartes the output containing "MYDETNAM" column.

#keys_9.cfg
#colName        dataType        format  hdrKey          query         null
TELESCOP        string          a10     TELESCOP        yes           ""
INSTRUME        string          a10     INSTRUME        yes           ""
CAL_VSD         string          a10     ""              yes           ""
CAL_VST         string          a8      ""              yes           ""
MYDETNAM        string          a20     "dnam"          yes           "NONE"
CAL_DESC        string          a70     CDES            no            ""
CAL_CLAS        string          a4      FOO             no            "DATA"

Parameters

name type ftype def min max reqd
index file output       yes
keyconf file input       yes
calfile file input       yes
clobber boolean   no      
verbose integer   0 0 5  

Detailed Parameter Descriptions

Parameter=index (file required filetype=output)

Output file name

The named file, created either from scratch or from an existing one, has an extension named after "CIF", Calibration Index File. The "CIF" table contains columns internally defined (mandatory) as defaults and optionally defined by users. The mandatory columns are

         CAL_DEV, CAL_DIR, CAL_FILE, CAL_XNO, 
         CAL_CNAM, 
         CAL_CBD, 
         CAL_QUAL,
         CAL_DATE

conforming with the current CALDB version 4. The set of optional columns are specified in an external text file, an input to 'keyconf'. The default name of 'index' will be given to ./caldb.indx' if the value of 'index' is empty or 'NONE' or 'none'.

Parameter=keyconf (file required filetype=input)

Input file name

The key config file is used for user to define optional columns of the Index. The frequent conventional columns in the CALDB category include

	 TELESCOP, 
         INSTRUME, 
         CAL_VST, CAL_VSD, 
         CAL_VET, CAL_VED, 
         DETNAM 

In addition, user can adds on any names other than the above, like FIDELITY, MYDETNAM, etc.

There are two special notes about the creation of the keyconf file. The column names of "TELESCOP" and "INSTRUME" must appear in keyconf file, so do in index file, as they will be served as the starting point of CALDB query later. Though the first three columns, or colName, dataType, and format, will be only interesting to index generation, the format of the entire set of columns have to be carried on as required by the software.

Parameter=calfile (file required filetype=input)

Input file name

"calfile" file, as the name explains, serves as a source directories of calibration data products. Following entries are allowed by the tool.

	 1> CALDB (the root of calibration data tree), 
	 2> single file, 
	 3> file stack, and 
	 4> named file list. 

The stacking files, as usual, are separated in comma, ','. The file list is indicated by '@' symbol in front of the named file string. Below is to summarize the possible syntaxes for the input string to 'calfile':

	 [directory]
	 [directory]+  or  [directory]/+
	 CALDB   
	 [file1, file2, ..., filen]
	 [directory]/*.fits
	 [directory]/*.[foo]
	 @[file-list]
	 NONE, none or empty 

Parameter=clobber (boolean default=no)

Remove the existing file if the value=true otherwise append the output to the existing file.

Parameter=verbose (integer default=0 min=0 max=5)

level of error message for standard printout. 0-no message and more message output in the order of the number ascending.

Bugs

See the bugs page for this tool on the CIAO website for an up-to-date listing of known bugs.

See Also

calibration
ardlib, caldb
modules
pixlib
tools
calquiz, calvalid, check_ciao_caldb

Last modified: December 2009