ciao_runtool

Synopsis

Run CIAO tools from Python as if they were functions (CIAO contributed package).

from ciao_contrib.runtool import *

Run the dmcopy tool:
dmcopy("evt2.fits[bin sky=::8]", "img.fits")

Run dmstat:
dmstat.punlearn()
dmstat.median = True
dmstat("src.dat[cols z,fx]", verbose=0)
print("The medians are {0}".format(dmstat.out_median))

Description

The ciao_contrib.runtool module allows CIAO tools to be run as if they were Python functions, and supports a pset-like mode where tool parameters can be set before the tool is run, named parameters when calling the tool, and easy access to the parameter values after the tool has completed. Parameter values are given using native Python types - e.g. True and False for boolean parameters.

The ciao_contrib.runtool module is provided as part of the CIAO contributed scripts package.

Loading the routines

The module can be loaded by saying:

from ciao_contrib.runtool import *

Note that it is not provided by the ciao_contrib.all module, unlike the other ciao_contrib modules.

Tools and parameter files

The runtool module provides a function for each major tool in the CIAO bin and contrib/bin directories that has a parameter file. It also provides functions to set and get settings for those parameter files without an associated tool, such as ardlib, lut and colors.

Extra routines

The following routines are also provided:

• get_pfiles() - return the user or system portion of the PFILES environment variable;
• set_pfiles() - easily change the user portion of the PFILES environment variable;
• make_tool() - create a routine that can be used to call a CIAO tool;
• list_tools() - returns a list of the tools and parameter files that are covered by this module;
• new_tmpdir() - a context manager that creates a temporary directory and ensures that the directory is removed once the block has finished;
• new_pfiles_environment() - a context manager that creates a temporary directory and uses it as the user path of the PFILES environment variable for the duration of the block; on exit the PFILES variable is restored to its original value and the temporary directory is removed;
• add_tool_history() - adds a CIAO history block to one or more files to indicate that a tool has been run with a given set of parameters (this block can be read by the dmhistory tool).

The new_pfiles_environment context manager should be used if you are running multiple copies of the same CIAO tool simultaneously (either using Python's multiprocessing module or by running several copies of the same script). See the "Running tools in parallel" section below for more information.

The new_tmpdir context manager is useful when you need access to a unique temporary directory, as shown below in the "Changing the temporary directory used by a tool" example.

For more information on these tools please use the Python help system - e.g.

>>> import ciao_contrib.runtool as rt
>>> help(rt.new_pfiles_environment)

Running a tool

A tool can be run by calling the routine with all the arguments it needs; required arguments are given either by positional order or as a named argument whereas optional arguments always have to be given as a named argument. As an example

>>> dmcopy("in.fits[sky=region(src.reg)][bin sky=::1]", "img.fits")
>>> dmstat(infile="img.fits", median=True, centroid=False)

Parameters can also be set using a parameter-style interface of <toolname>.<parname> before calling the tool as shown below:

>>> dmcopy.outfile = "img.fits"
>>> dmcopy.infile = "in.fits[sky=region(src.reg)][bin sky=::1]"
>>> dmcopy()
>>> dmstat.median = True
>>> dmstat.centroid = False
>>> dmstat(infile="img.fits")

You can mix both approaches, as shown above in the last dmstat call.

Running tools in parallel

The Python multiprocessing module makes it possible to run tools in parallel, and is used to provide the multiprocessing feature in scripts like fluximage and merge_obs. Alternatively, you can run several copies of your script at the same time. In either case, you may see errors or problems because the same parameter file is being used for each run of the tool (this can lead to invalid results, as inconsistent parameter settings are used, or the tool can fail). To avoid this, use the new_pfiles_environment context manager to create a new copy of your PFILES environment. An example of its use is shown below (this uses Sherpa's parallel_map routine, which provides a simple way to use the multiprocessing module):

import ciao_contrib.runtool as rt
import sherpa.utils

# The asphist/mkinstmap/mkexpmap call below will use their own
# PFILES directory and so will not collide with other runs of
# these tools (or changes to the default ardlib parameter file
# since this has been copied over to the new directory)
#
def getemap((ccd, gridstr)):
    """Create an ACIS exposure map for the ccd and grid that can
    be run in parallel. The routine can only accept a single
    argument as it is to be called via Sherpa's parallel_map
    routine. As we need multiple values, we therefore use a tuple.

    This routine is only for didactic purposes since it assumes
    a number of things, such as file names, that will not be
    true in general.
    """

    # We copy over the ardlib so that if the bad-pixel file
    # has been set, its value is used in the new environment.
    #
    with rt.new_pfiles_environment(ardlib=True):
        afile = "asphist{0}.fits".format(ccd)
        ah = rt.make_tool("asphist")
        ah(infile="asol1.fits", outfile=afile,
           evtfile="evt2.fits[ccd_id={0}]".format(ccd),
           clobber=True)

        ifile = "imap{0}.fits".format(ccd)
        mki = rt.make_tool("mkinstmap")
        mki(outfile=ifile, monoenergy=1.7, pixelgrid="1:1024:1,1:1024:1",
            obsfile=afile+"[asphist]", detsubsys="ACIS-{0}".format(ccd),
            maskfile="msk1.fits", clobber=True)

        efile = "emap{0}.fits".format(ccd)
        mke = rt.make_tool("mkexpmap")
        mke(asphistfile=afile, outfile=efile, instmapfile=ifile,
            xygrid=gridstr, clobber=True)

# Create the arguments for CCDs 0-3 and 6
grid = "2824.5:6552.5:#932,2868.5:5328.5:#615"
ccds = [0, 1, 2, 3, 6]
args = [(ccd,grid) for ccd in ccds]

# Run getemap on all available CPUs
sherpa.utils.parallel_map(getemap, args)

Note that new_pfiles_environment automatically creates the temporary parameter directory, changes the PFILES environment variable, runs your code, and then restores the PFILES variable and deletes the temporary directory for you.

The location of the temporary direction is determined by the ASCDS_WORK_PATH environment variable.

Clearing, setting, and viewing parameter settings

Each tool has a punlearn() method which will reset the values to the CIAO defaults. As shown above, parameter values can be set either when calling the tool or via the <toolname>.<parname> approach, and the current settings can be displayed using print. As an example

>>> dmjoin.punlearn()
>>> dmjoin.interpolate = "closest"
>>> print(dmjoin)
Parameters for dmjoin:

Required parameters:
              infile =                  Input file
            joinfile =                  Input join file
             outfile =                  Output file
                join =                  Join col

Optional parameters:
         interpolate = closest          Interpolate mode
             verbose = 0                Debug Level(0-5)
             clobber = False            Clobber

There are also write_params() and read_params() methods that can be used to create a .par file from the current settings or read values in from a given .par file. See "help <toolname>.read_params" or "help <toolname>.write_params" for more information on these methods.

Support for unique parameter prefixes

As with the command line, parameter names can be shortened to any unique prefix. This means that dmjoin.inf can be used to read or write the infile parameter of dmjoin, but dmstat.joi will fail since it matches both the join and joinfile parameters.

This support also holds when calling a tool, so

>>> dmcopy(infile, outfile, cl=True)

calls dmcopy with the clobber parameter set to yes.

An error will be raised if the prefix is not unique; for instance

>>> dmjoin.joi
AttributeError: Multiple matches for dmjoin parameter 'joi', choose from:
  joinfile join

A SyntaxError will occur if a Python reserved word - such as "in" or "for" - is used as the parameter name.

Stack support

There is limited support for stacks in this module: if a string or file parameter is sent an array then the elements are turned into a comma-separated string and this is used as the parameter value.

The following example creates the average of all the files that match imgn.fits, with n being from 1 to 9, to create avg.fits:

>>> import glob
>>> from ciao_contrib.runtool import dmimgcalc
>>> dmimgcalc.punlearn()
>>> infiles = glob.glob("img[1-9].fits")
>>> nf = len(infiles)
>>> inames = ["img{0}".format(i) for i in range(1,nf+1)]
>>> opstr = "imgout=({0})/{1}.0".format("+".join(inames), nf)
>>> dmimgcalc(infiles, out="avg.fits", op=opstr)

The build() routine from the stk module can be used to convert a parameter value to an array of values, and the make_stackfile() routine from the ciao_contrib.stacklib module can be used to create a temporary stack file from an array.

Parameter constraints

If a parameter can only be one of a set of values then the parameter value can be set to a unique substring, just as on the command line. As an example, the interpolate parameter of dmjoin can be one of "linear", "first", "last", "closest", "furthest", "minimum", or "maximum", so the following are all valid:

>>> dmjoin.interp = "min"
>>> dmjoin("a.dat","b.dat","c.dat","x",interp="lin")

An error occurs if a parameter is set to an invalid value; e.g. either using a number outside the valid range or if there are a limited set of choices, as shown below:

>>> dmjoin.verbose = 10
ValueError: dmjoin.verbose must be <= 5 but set to 10
>>> dmjoin.interp = "foo"
ValueError: The parameter interpolate was set to foo when it must be one of:
  linear first last closest furthest minimum maximum

The return value

If there is no screen output from the tool then the tool returns None, otherwise it returns a single string containing both the STDOUT and STDERR channels. This output can either be stored in a variable or - if used from an interactive environment such as IPython and Sherpa - it will be automatically displayed. In the following, the dmcopy call creates no screen output, so it returns None, and therefore is no screen output, whereas the dmstat output is displayed:

>>> dmcopy(infile, outfile)
>>> dmstat(outfile)

y
    min:	10 	      @:	1 
    max:	87.3 	      @:	5 
   mean:	35.3 
  sigma:	28.55548984 
    sum:	176.5 
   good:	5 
   null:	0
>>> out = dmstat(outfile)
>>> lines = out.split("\n")
>>> print(lines[1])
    min:	10 	      @:	1 

What happens if the tool fails?

An IOError is raised if the tool returns a non-zero exit status. The error contains the screen output of the tool, as shown below where I used an invalid file:

>>> dmstat("simple.dat[cols z]")
IOError: An error occurred while running 'dmstat':
  # DMSTAT (CIAO 4.11): could not open infile simple.dat[cols z].

The return value can be retrieved using the get_runtime_details() method of the tool, which is described below.

Unfortunately not all CIAO tools set the exit status correctly on error, so these will appear to have run successfully even on failure.

How long did the tool run for?

The get_runtime_details() method returns a dictionary which lists information on the last time the tool was run using this interface. This information is cleared when the punlearn() method is used. The keyword and values returned are listed below. Note that no keyword is guaranteed to be present and that the timings are not intended for precise bench-marking so should not be considered to be accurate to more than about one second.

See the documentation for the Python time module for more information on how to use the start and end values.

Examples

Example 1

Providing all the arguments in the call

In this example we run the dmcopy tool and provide all the necessary arguments when the routine is called. The dmcopy.punlearn routine is used to ensure the settings are at their default values.

>>> dmcopy.punlearn()
>>> dmcopy("evt2.fits[ccd_id=7]", "ccd7.fits", clobber=True)

We adapt this to loop through the ccd_id values 0, 1, 2, 3, and 6 and create the files "ccd<n>.fits". We also take advantage of the support for using parameter prefixes to shorten "clobber" to "cl".

>>> for ccd in [0,1,2,3,6]:
           infile = "evt2.fits[ccd_id={0}]".format(ccd)
           outfile = "ccd{0}.fits".format(ccd)
           dmcopy(infile, outfile, cl=True)
           print("Created: {0}".format(outfile))


Created: ccd0.fits
Created: ccd1.fits
Created: ccd2.fits
Created: ccd3.fits
Created: ccd6.fits

Example 2

Setting the arguments before the call

The "pset" approach can also be used to set the arguments before the call. The original example above can be re-written as:

>>> dmcopy.punlearn()
>>> dmcopy.infile = "evt2.fits[ccd_id=7]"
>>> dmcopy.outfile = "ccd7.fits"
>>> dmcopy.cl = True
>>> dmcopy()

Example 3

Mixing both approaches

Both approaches can be combined. For instance, if you wish to call a tool multiple times with some parameters fixed you may wish to use the pset approach for those parameters and call the tool with those arguments that change.

Here we set the clobber argument and then call dmcopy with the infile and outfile arguments.

>>> dmcopy.punlearn()
>>> dmcopy.cl = True
>>> dmcopy(infile, outfile)

Example 4

Accessing parameter values after a tool runs

Some tool set parameter values after being run. These values can be inspected and used as shown below, which includes setting the parameter value for the mkexpmap tool based on the output of get_sky_limits.

>>> get_sky_limits("img.fits", verbose=0)
>>> dmf = get_sky_limits.dmfilter
>>> print("Filter: {0}".format(get_sky_limits.dmf))
Filter: X=4047.5:4098.5:#51,Y=3920.5:3971.5:#51
>>> mkexpmap.xygrid = get_sky_limits.xygrid

The whole set of parameters can also be displayed:

>>> print(get_sky_limits)
Parameters for get_sky_limits:

Required parameters:
               image = img.fits         Image for which you want to know the binning

Optional parameters:
           precision = 1                Precision [# decimal points] for output numbers
            dmfilter = X=4047.5:4098.5:#51,Y=3920.5:3971.5:#51  DM filter syntax to match image
              xygrid = 4047.5:4098.5:#51,3920.5:3971.5:#52  xygrid parameter for mkexpmap to match image
             verbose = 0                Debug Level (0-5)

Example 5

Multiple copies of a tool

Multiple versions of a tool can be created, each with its own set of parameters. As a simple case, if you want to run dmstat with the median parameter set to True or False you could try the following:

>>> dms1 = make_tool("dmstat")
>>> dms1.median = False
>>> dms2 = make_tool("dmstat")
>>> dms2.median = True

Then using dms1 or dms2 will run dmstat with the median parameter set to False and True repsectively:

>>> dms1("lc.fits[cols count_rate]")
>>> dms2("lc.fits[cols count_rate]")

Example 6

Changing the temporary directory used by a tool

Several tools allow you to change the location of the temporary directory used to store intermediate products. This parameter should be changed if multiple copies of the tool are being run simultaneously, to avoid clashes and potential tool failure. One way to do this is to use the new_tmpdir context manager, which creates a temporary directory and guarantees it's clean up. As an example:

    with new_tmpdir() as tmpdir:
        # tmpdir has been created and can be used in this block
        bmap = make_tool("create_bkg_map")
        bmap.tmpdir = tmpdir
        ...
        bmap(...)

    # at this point the temporary directory is guaranteed to have been
    # deleted, even if there is was an error in the block of code above
    #
    ...

Controlling the location of temporary files and directories

The ASCDS_WORK_PATH environment variable (not TMPDIR) is used to determine the location to use for temporary files and directories. This location is used when running CIAO tools - those that are instances of the CIAOToolParFile class - since a copy of the parameter file is created to ensure there to ensure that tools can be run in parallel without conflict. It is also used by the new_tmpdir and new_pfiles_environment context managers.

Routines with no corresponding tool

There are a number of CIAO parameter files which have no corresponding tool - for instance ardlib, geom, colors and lut. These are provided in this module for reading and setting parameters, but you can not call them. Also, as discussed below in the "PARAMETER FILES" section, setting these parameters does not change the on-disk parameter file unless you call the write_params method.

Here is an example of accessing the lut parameter file:

>>> print(lut)
Parameters for lut:
                   a = ${ASCDS_CALIB}/a.lut  Color lookup table
                aips = ${ASCDS_CALIB}/aips.lut  Color lookup table
                   b = ${ASCDS_CALIB}/b.lut  Color lookup table
                  bb = ${ASCDS_CALIB}/bb.lut  Color lookup table
                blue = ${ASCDS_CALIB}/blue.lut  Color lookup table
               color = ${ASCDS_CALIB}/color.lut  Color lookup table
                cool = ${ASCDS_CALIB}/cool.lut  Color lookup table
               green = ${ASCDS_CALIB}/green.lut  Color lookup table
                grey = ${ASCDS_CALIB}/grey.lut  Color lookup table
              halley = ${ASCDS_CALIB}/halley.lut  Color lookup table
               heaob = ${ASCDS_CALIB}/heaob.lut  Color lookup table
                heat = ${ASCDS_CALIB}/heat.lut  Color lookup table
                 hsv = ${ASCDS_CALIB}/hsv.lut  Color lookup table
                  i8 = ${ASCDS_CALIB}/i8.lut  Color lookup table
            rainbow1 = ${ASCDS_CALIB}/rainbow1.lut  Color lookup table
            rainbow2 = ${ASCDS_CALIB}/rainbow2.lut  Color lookup table
                ramp = ${ASCDS_CALIB}/ramp.lut  Color lookup table
                 red = ${ASCDS_CALIB}/red.lut  Color lookup table
                 sls = ${ASCDS_CALIB}/sls.lut  Color lookup table
           staircase = ${ASCDS_CALIB}/staircase.lut  Color lookup table
            standard = ${ASCDS_CALIB}/standard.lut  Color lookup table

>>> hl = lut.halley
>>> print(os.path.expandvars(hl))
/soft/ciao-4.11/data/halley.lut

(assuming CIAO is installed in /soft/ciao-4.11/) but it can not be called

>>> lut(b="tst.lut")
TypeError: 'CIAOParameter' object is not callable

Parameter files

Using these routines does not change any parameter file on disk, unless you use one of the routines listed below. This means it is safe to use this module to run tools from a script whilst using CIAO tools from the command line; the parameter files used by the command-line tools will not be over-written or changed by the Python script.

Any routine that is an instance of the CIAOToolParFile class - which, as of CIAO 4.9, is all tools except for axbary, dmgti, evalpos, fullgarf, mean_energy_map, pileup_map, tgdetect, and wavdetect - will not change the on-disk parameter file.

Routines that do change the on-disk parameter file

Running instances of the CIAOToolDirect class - so axbary, dmgti, evalpos, fullgarf, mean_energy_map, pileup_map, tgdetect, and wavdetect - will change the on-disk parameter file.

How to check whether a parameter file will be changed?

The class of the routine - which can be returned by the type() call or checked using isinstance() - determines whether a parameter file will be changed ("CIAOToolDirect") or not ("CIAOToolParFile"). For instance,

>>> type(dmgti)
       <class 'ciao_contrib.runtool.CIAOToolDirect'>
>>> type(dmstat)
       <class 'ciao_contrib.runtool.CIAOToolParFile'>
>>> type(wavdetect)
       <class 'ciao_contrib.runtool.CIAOToolDirect'>

These class names are rather confusing and may be changed in a later release.

WARNING: use of ardlib and other such parameter files

Many instrument-specific tools use parameter files like ardlib and geom to find certain files. When the tool is run, they still use the on-disk version of the file and not the values stored in the Python object of the same name. So the setting

>>> ardlib.AXAF_ACIS0_BADPIX = "bpix.fits[ccd_id=0]"

will not be picked up when a tool such as mkinstmap is run. One solution is to use the write_params method, which is described further below, to write out the ardlib parameter file to disk after making changes.

>>> ardlib.AXAF_ACIS0_BADPIX = "bpix.fits[ccd_id=0]"
>>> ardlib.write_params()

Updating a parameter file

If you want to write the current set of parameters to disk, you can use the "write_params" method. As an example, where e1, e2 and e3 are numbers that have been calculated elsewhere

>>> expr = "imgout=(img1*{0}+img2*{2})+{3}".format(e1,e2,e3)
>>> dmimgcalc.operation = expr
>>> dmimgcalc.write_params()

will write the expression out to the dmimgcalc parameter file (eg try "!plist dmimgcalc" to see the result).

To write to a specific file, give a file name as the argument to write_params. A ".par" suffix will be appended if the filename does not contain one.

Updating to match a parameter file

The default values - and those used when the punlearn() method is used - are taken from the CIAO defaults. To update the values to match those from an on-disk file use the "read_params" method:

>>> dmimgcalc.read_params()

To read from a specific file, give a file name as the argument to write_params. A ".par" suffix will be appended if the filename does not contain one.

Changing the PFILES environment variable

The set_pfiles() routine changes the user portion of the PFILES environment variable. This controls where the tools look for ancillary parameter files (e.g. ardlib) and where the read_params and write_params methods default to when no explicit filename is given. See 'help set_pfiles' for more information.

The get_pfiles() routine can be used to get the current user or system portions of the PFILES environment variable. See 'help get_pfiles' for more information.

The new_pfiles_environment() context manager can be used to automatically handle the creation and clean up of a directory, as well as changing the PFILES environment variable. For instance:

def getemap(ccd, gridstr):
    """Create an ACIS exposure map for the given CCD and using gridstr
    as the xygrid parameter of mkexpmap.

    The input files are hard coded to simple names for simplicitly.
    """

    with new_pfiles_environment(ardlib=True):
        afile = "asphist{0}.fits".format(ccd)
        ah = make_tool("asphist")
        ah(infile="asol1.fits", outfile=afile,
           evtfile="evt2.fits[ccd_id={0}]".format(ccd),
           clobber=True)

        ifile = "imap{0}.fits".format(ccd)
        mki = make_tool("mkinstmap")
        mki(outfile=ifile, monoenergy=1.7, pixelgrid="1:1024:1,1:1024:1",
            obsfile=afile+"[asphist]", detsubsys="ACIS-{0}".format(ccd),
            maskfile="msk1.fits", clobber=True)

        efile = "emap{0}.fits".format(ccd)
        mke = make_tool("mkexpmap")
        mke(asphistfile=afile, outfile=efile, instmapfile=ifile,
            xygrid=gridstr, clobber=True)

defines a routine, getemp(), which will run the asphist, mkinstmap and mkexpmap tools using a temporary directory and PFILES setting. This directory will be automatically removed, and the PFILES environment variable restored at the end of the routine, even if one of tools fails.

Differences to CIAO tools

No mode parameter

All tools are run with the mode parameter set to "hl". This is to avoid the tool trying to prompt you for missing parameter values, which would lead to it appear to hang as the prompt would not be seen by the user.

Missing parameters

As well as the mode parameters, there are a few tools which have a parameter that can only be set to a single value. These parameters are excluded from the interface.

Parameter values

Parameters should be set using the appropriate Python type, namely booleans, integers, floats or strings. For floating-point values, None is used to indicate the "INDEF" setting. None can be used for filenames and strings to indicate the value "". File names are given as strings, but you do not need to add extra quotes if it contains a space - so the following is valid

>>> dmstat("evt2.fits[sky=region(src.reg)][cols energy]")

Redirected values

There is limited support for the use of parameter redirects - the use of ")<parname>" to use the value from another parameter - but it only supports redirects to the same parameter file.

Parameter names

When required, parameter names are adjusted to make sure they are valid Python. At present the only change used is to convert any "-" characters to "_"; this is only required for several parameters in ardlib. Purely numeric parameter names are ignored; this only occurs for the "helper" parameters of dmtabfilt.

Changes in the scripts 4.15.1 (January 2023) release

The module has been updated to include the new color_color script.

Changes in the scripts 4.11.1 (December 2018) release

Updated for CIAO 4.11

The module has been updated to reflect the parameters for the tools included in CIAO 4.11.

Changes in the scripts 4.10.1 (April 2018) release

Updated for CIAO 4.10

The module has been updated to reflect the parameters for the tools included in CIAO 4.10.

Improved Python 3 support for INDEF parameters

Fixed an issue which prevented the use of INDEF as a parameter value when run on a Python 3.5 version of CIAO.

Changes in the scripts 4.9.4 (July 2017) release

Running scripts

The following tools have been changed to use the CIAOToolDirect class, which means that care should be taken if multiple copies of these tools are run in parallel: axbary, evalpos, mean_energy_map, pileup_map, tgdetect, and wavdetect. It is suggested that the new_pfiles_environment context manager is used to run these tools, as it automates the creation and deletion of a temporary parameter directory for use by the tools.

Changes in the scripts 4.9.2 (April 2017) release

Changed tool

The blanksky routine supports the new 'random' seed parameter in order to create reproducible background files.

Changes in the scripts 4.8.4 (October 2016) release

New tools

The module was updated to support the new blanksky, blanksky_image, and correct_periscope_drift scripts added in this release.

Changed tool

The specextract routine supports the new 'binarfcorr' parameter to explicitly specify detector pixel size used by arfcorr PSF correction. The 'srcflux' script has the 'psfmethod=marx' option enabled to simulate a monochromatic PSF for aperture correction if MARX is installed.

Changes in the scripts 4.8.3 (April 2016) release

New tools

The module was updated to support the new simulate_psf script added in this release.

Changes in the script 4.8.2 (January 2016) release

Changed tools

Support for the add_grating_orders and add_grating spectra scripts has been removed, superseded by the combine_grating_spectra script to combine grating orders and spectra.

Changes in the script 4.8.1 (December 2015) release

Removed tools

The fluximage, merge_obs, flux_obs, and readout_bkg routines support the 'random' seed parameter in order to create reproducible results.

Changes in the scripts 4.7.4 (September 2015) release

New tools

The module was updated to support the new readout_bkg, install_marx, detilt, dewiggle, symmetrize, and download_obsid_caldb scripts added in this release.

Changed tool

The specextract routine supports the new 'binarfcorr' parameter to explicitly specify detector pixel size used by arfcorr PSF correction. The 'srcflux' script has the 'psfmethod=marx' option enabled to simulate a monochromatic PSF for aperture correction if MARX is installed.

Changes in the scripts 4.7.2 (April 2015) release

New tools

The module was updated to support the new splitobs, gti_align, and multi_chip_gti scripts added in this release.

Changed tool

The mktgresp routine supports the new 'orders' parameter to allow the creation of responses for an arbitrary set of orders. The responses for different orders are can now be created in parallel, controlled by the new 'parallel' and 'nproc' parameters.

Changes in the scripts 4.6.7 (October 2014) release

New tools

The module was updated to support the new combine_grating_spectra and tgsplit scripts added in this release.

Changes in the scripts 4.6.6 (September 2014) release

Using $ASCDS_WORK_PATH for temporary files

The new_pfiles_environment() and new_tmpdir() context managers have a new parameter tmpdir, and default to using the $ASCDS_WORK_PATH environment variable to set the location of the temporary directories they create; previously they used /tmp or the $TMPDIR environment variable. The tmpdir argument has been added to the add_tool_history() function.

Changes in the scripts 4.6.3 (March 2014) release

New tool

The module was updated to support the new ecf_calc script added in this release.

Changes in the scripts 4.6.1 (December 2013) release

The module was updated to support the new tools and scripts in the CIAO 4.6 release (e.g. srcflux,tgdetect2,tg_choose_method) the removed tools and scripts (acis_classify_hotpix, acis_find_hotpix, acis_run_hotpix, merge_all), and for changes in parameters of existing tools (such as mkinstmap, acis_process_events, and specextract).

Automatic stack handling

Long parameter values are automatically written out as a stack file, to avoid problems with long line lengths. Prior to this release the stack files were accessed using '@filename', which has now been changed to '@-filename' to avoid the path to the stack file being added to the entries. See 'ahelp stack' for more information.

Improved support for modelflux

The modelflux routine now no longer changes the on-disk parameter file (i.e. it is now an instance of the CIAOToolParFile rather than CIAOToolDirect class). See the "Parameter files" section above for more information.

Changes in the scripts 4.5.4 (August 2013) release

New tools

The module supports the new scripts in this release: search_csc and obsid_search_csc.

Changed tools

The specextract and find_chandra_obsid commands have been updated to reflect the changes in the parameter files of these tools.

Running tools in parallel

A new section to the ahelp file has been added: Running tools in parallel.

Changes in the scripts 4.5.2 (April 2013) release

New tools

The module supports the new scripts in this release: mktgresp, tgmask2reg, and reg2tgmask.

Changed tools

The chandra_repro command supports the new recreate_tg_mask parameter.

Renamed parameter files

The list of look-up tables available via the ximage and imagej parameter files are now accessed through their new names: ximage_lut and imagej_lut.

Changes in the scripts 4.5.1 (December 2012) release

The module was updated to support the new tools in the CIAO 4.5 release (tg_findzo), the removed tools (mkpsf), and for changes in parameters of existing tools (e.g. modelflux). Support for INDEF with numeric parameters has been improved.

Bugs

Use of parameter redirects to other parameter files

Parameter values of the form ")toolname.parname", which use the parameter value from another parameter file, are not supported. So, for example tgdetect, which has its thresh parameter set to ")celldetect.thresh", will not run unless you manually set these parameters to a valid value. For example:

>>> from ciao_contrib.runtool import *
>>> print(tgdetect.thresh)
)celldetect.thresh
>>> tgdetect.thresh = celldetect.thresh
>>> print(tgdetect.thresh)
3.0

See the bugs pages for an up-to-date listing of known bugs.

Refer to the CIAO bug pages for an up-to-date listing of known issues.