CIAO 4.4 Release Notes

---

Notable changes and improvements in CIAO 4.4:

• The new acis_find_afterglow identifies cosmic-ray afterglows and hot pixels for TIMED mode observations. This tool should be used in place of "acis_run_hotpix" for all analyses. This tool improves over previous algorithms by identifying shorter afterglows with possibly larger gaps between frames. The chandra_repro script has been updated to use the new tool.

• The wavdetect tool takes an observation-specific PSF map file - created by mkpsf - instead of using the Chandra PSF table data or the tool's internal parameterization of the ROSAT PSF.

• The addresp tool has been updated to combine grating response files and to create output RSP files. The new functionality is available via the combine_spectra script as well.

• An updated asphist algorithm uses the full tangent plane offset computations and to correct angle between pointing and North. Both of these affect data that have been reprojected to different tangent points and/or multi-obi observations with significantly different pointings.

• The ChIPS graphical user interface (GUI) allows users to adjust the layout of their plots and make edits to many of the plot properties. A Python terminal is built into the GUI, allowing access to ChIPS and other Python commands. The GUI is available to any process that creates a ChIPS plot, such as Sherpa and Prism.

• Bayesian Analysis with pyBLoCXS: the pyBLoCXS module is now integrated into Sherpa. pyBLoCXS is a sophisticated Markov chain Monte Carlo (MCMC) based algorithm designed to carry out Bayesian Low-Count X-ray Spectral (BLoCXS) analysis in the Sherpa environment.

• Fitting Template Models to 1D Data: Sherpa supports a new template model (an extension of table model). Sherpa can now read in a collection of templates from a directory full of template files, and can compare a data set to all the templates in that collection.

• The Monte Carlo optimization method has been significantly improved, and can now find more reasonable parameter values in cases where it formerly came up against parameter limits. The LMDIF optimization method has also been updated to avoid getting "stuck" at the parameter space boundaries for certain cases.

---

• How CALDB 4.4.10 Affects Your Analysis
• How CALDB 4.4.8 Affects Your Analysis
• How CIAO 4.4 and CALDB 4.4.7 Affect Your Analysis
• Installation
• Tools
• Parameter Files
• Data Model
• ChIPS
• Sherpa
• GUIs (Graphical User Interfaces)
• Analysis Scripts
• Libraries
• Environment
• Documentation

---

How CALDB 4.4.10 Affects Your Analysis

ACIS Imaging Data and Grating Data

• Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

  There are new time-dependent ACIS Gain (TGAIN) files for Feb - Apr 2012 (Epoch 49):

  acisD2011-11-01t_gainN0006.fits
  acisD2011-11-01t_gainN0005.fits
  acisD2012-02-01t_gain_biN0002.fits
  acisD2012-02-01t_gainN0002.fits
  

  The files are applicable to -120 C focal plane temperature only.

  Users working with ACIS data taken during 01 Nov 2011 through the present may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

  Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

• ACIS QE Contamination Model vN0007

  The ACIS QE contamination model has been upgraded to vN0007. This version of the file contains a ~15% change in the optical depth at 700 eV for observations in 2012 as compared with those in early 2009.

  These CIAO response tools automatically apply the contamination file when creating ACIS response files:

  • mkarf
  • mkgarf
  • mkwarf
  • mkinstmap

  As well as the scripts which use them:

  • specextract (calls mkwarf and mkarf)
  • fullgarf (calls mkgarf)
  • fluximage (calls mkinstmap)
  • merge_all

---

How CALDB 4.4.8 Affects Your Analysis

ACIS Imaging Data and Grating Data

• Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

  There are new time-dependent ACIS Gain (TGAIN) files for 01 Nov 2011 - 31 January 2012 (Epoch 48):

       acisD2011-08-01t_gainN0006.fits
       acisD2011-08-01t_gainN0005.fits
       acisD2011-11-01t_gain_biN0002.fits
       acisD2011-11-01t_gainN0002.fits
  

  The files are applicable to -120 C focal plane temperature only. The acisD2011-11-01t_gain_biN0002.fits file is the new default processing file in the CALDB.

  Users working with ACIS data taken during 01 Aug 2011 through the present may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

  Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

---

How CIAO 4.4 and CALDB 4.4.7 Affect Your Analysis

ACIS Imaging Data

• Creating ACIS Bad Pixel Files

  The new acis_find_afterglow identifies cosmic-ray afterglows and hot pixels for TIMED mode observations. This tool should be used in place of "acis_run_hotpix" for all analyses.

  The chandra_repro script has been updated to use the new tool.

• Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

  There are new time-dependent ACIS Gain (TGAIN) files for 01 August 2011 - 31 October 2011 (Epoch 47):

  acisD2011-05-01t_gainN0006.fits
  acisD2011-05-01t_gainN0005.fits
  acisD2011-08-01t_gain_biN0002.fits
  acisD2011-08-01t_gainN0002.fits

  The files are applicable to -120 C focal plane temperature only. The acisD2011-08-01t_gain_biN0002.fits file is the new default processing file in the CALDB.

  Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

  Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

ACIS Grating Data

• Creating ACIS Bad Pixel Files

  The new acis_find_afterglow identifies cosmic-ray afterglows and hot pixels for TIMED mode observations. This tool should be used in place of "acis_run_hotpix" for all analyses.

  The chandra_repro script has been updated to use the new tool.

• Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

  There are new time-dependent ACIS Gain (TGAIN) files for 01 August 2011 - 31 October 2011 (Epoch 47):

  acisD2011-05-01t_gainN0006.fits
  acisD2011-05-01t_gainN0005.fits
  acisD2011-08-01t_gain_biN0002.fits
  acisD2011-08-01t_gainN0002.fits

  The files are applicable to -120 C focal plane temperature only. The acisD2011-08-01t_gain_biN0002.fits file is the new default processing file in the CALDB.

  Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

  Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

• HETG Grating Efficiency (version N0007)

  This file was also released in the CALDB 4.4.6.1 patch on 02 December 2011.

  The cross-calibration of the first-order HEG and MEG portions of the HETG in the grating efficiency file (GREFF) has been adjusted via analysis of 30 Blazar observations. The new file (N0007) will be used by default when creating HETG response files (ARFs). The CIAO tools and scripts which use this file are:

  • mkgarf
  • fullgarf (uses mkgarf)

  For instructions on how to continue using the vN0006 file, read the HETG Grating Efficiency why topic.

HRC-I Imaging Data

• HRC-I Gain Maps

  The latest time-dependent gain maps (GMAP) for the HRC-I have been released:

  hrciD2010-09-25sampgainN0002.fits    # 2010-09-25T12:00:00 to 2011-09-19T12:00:00
  hrciD2011-09-19sampgainN0001.fits    # 2011-09-19T12:00:00 (open-ended range)
  

  The gain maps are updated annually to maintain 2% consistency in the PI. Technical details on the gain maps are described in the gain maps section of the HRC calibration website.

  Users working with HRC-I data should run chandra_repro to reprocess the data and improve the gain calibration in their data.

• HRC-I background PI spectra

  The HRC calibration team has released a set of background spectra from the HRC-I. These spectra describe the particle background of the detector and vary slowly with time which means that they can be used to significantly improve the signal-to-noise of HRC-I imaging data and may be useful for point source analysis (e.g. fields with many point sources).

  The HRC-I background spectra files are included in the main CALDB tarfile.

  The 2010 HRC-I background PI spectrum file was remade with the new gain map (also released in CALDB 4.4.7), and the 2011 file has been added:

  unix% ls -1 $CALDB/data/chandra/hrc/pibgspec/
  hrciD1999-10-04pibgspecN0001.fits
  hrciD2000-12-12pibgspecN0001.fits
  hrciD2002-01-26pibgspecN0001.fits
  hrciD2003-02-22pibgspecN0001.fits
  hrciD2004-11-25pibgspecN0001.fits
  hrciD2005-10-17pibgspecN0001.fits
  hrciD2006-09-20pibgspecN0001.fits
  hrciD2007-09-17pibgspecN0001.fits
  hrciD2008-09-07pibgspecN0001.fits
  hrciD2009-09-24pibgspecN0001.fits
  hrciD2010-09-25pibgspecN0001.fits
  hrciD2011-09-19pibgspecN0001.fits
  

  For instructions on using the background event files, follow the The HRC-I Background Spectra Files thread.

HRC-S Imaging and Grating Data

• QE Upgraded to vN0011

  A new mean quantum efficiency (QE) file has been released for HRC-S (version N0011).

  $CALDB/data/chandra/hrc/qe/hrcsD1999-07-22qeN0011.fits
  

  Following in-flight calibration of the low energy QE, models of flux vs energy and parameter estimates for the two main calibration standards (HZ43 and Sirius B) underwent significant revisions. The latest non-LTE pure hydrogen model atmospheres computed by the Tubingen group have been used to devise a new low energy calibration.

  These figures show the residuals between LETG+HRC-S observations and the fiducial model before and after modifications to the HRC-S quantum efficiency.

  • LETGS QE pre-correction (vN0010): The fit and residuals to co-added spectra for HZ43 taken since launch, with the previous HRC-S QE (version N0010) in CalDB.
  • LETGS QE corrected (vN0011): The fit and residuals to co-added spectra for HZ43 taken since launch, with the new HRC-S QE (version N0011).

  The new file will be used by default when creating exposure maps or grating ARFs in CIAO.

---

Installation

Supported Platforms

• CIAO 4.4 is supported on:

  • Linux 32 bit and 64 bit (tested on CentOS 5.6 and Fedora Core 11)
  • Mac OS X 10.6 Intel 32 bit and 64 bit
  • Mac OS X 10.7 Intel 32 bit and 64 bit

  The following platforms are no longer supported:

  • Mac OS X 10.5
  • Solaris 10

Updates to the ciao-install Installation Script

• The default installation directory is $HOME instead of /usr/local.

• The script will not run if called with sudo, i.e. "sudo bash ciao-install". If ciao-install encounters a permissions problem, it prints instructions on what to fix in order to install CIAO and gives the user the chance to specify a different installation directory.

• An option was added to check for the required system libraries, e.g. verify that libGL is installed if the chips segment is being installed.

Remove old parameter files

• With every new CIAO release, some parameter files have changed: new parameters may be added and occasionally old ones removed or renamed. Deleting or renaming the local parameter directory ensures that the correct parameter files will be accessed the first time a tool is run:

  unix% rm ~/cxcds_param4/*
  

Remove the fontconfig cache

• The GUI applications (e.g. prism, ChIPS, ds9) used in CIAO may not run correctly if there are incompatible .fontconfig files cached on the user's system. To remove the .fontconfig cache:

  unix% rm ~/.fontconfig/*
  

---

Tools

acis_build_badpix

• If a bad pixel or column from the CALDB input file is excluded by the usrfile, the 3x3 island surrounding it is cleared. if badpixel/column in caldib input file is excluded via

acis_find_afterglow

• acis_find_afterglow identifies cosmic-ray afterglows and hot pixels for TIMED mode observations. This tool should be used in place of "acis_run_hotpix" for all analyses.

acis_process_events

• Added a check that the bad pixel and event files are from the same ObsID and have the same OBI number. A warning is printed if they don't match.

• Issue a warning if the bad pixel file is not in time order.

acis_streak_map

• The tool has been rewritten in Python. The functionality of the tool is unchanged.

acis_run_hotpix

• Additional acis_build_badpix parameters may be set from the acis_run_hotpix parameter file: This section removed when tool was depreated.

aconvolve

• Two new kernel options have been added: hanning and racos.

• Bug fix: if the axis length was not a power of two, the padding was doubled to the next power of two. The extra padding would create much larger images and require large amounts of memory.

addresp

• Three new parameters have been added:

  • phafile: the input list of PHA files which are used to calculate the EXPOSURE and ONTIME header values for the output ARF file.

    The combined EXPOSURE written to the output files comes from the exposures of the input PHA files, not the ARF input as was done in CIAO 4.3.

  • method: if method=avg, the average of the PHA exposure times is written to the header of the combined ARF. If method=sum, the sum of the PHA exposure times is used.

    The addresp tool may be used with method=avg to combine grating ARFs, creating the same output the dmarfadd tool.

  • type: setting type=rsp means that the tool will combine the ARF and RMF into an RSP file instead of making two separate files.

• If the input ARF only contains NaNs, the tool will exit with an error.

aprates

• Bug fix: the bounds for a value were incorrect when Poisson statistics were used and either the 'n' or 'm' value was greater than approximately 300.

• New parameter: clobber allows an existing output file to be overwritten.

arfcorr

• The tool has been rewritten in Python. The functionality of the tool is unchanged.

asphist

• The tool has been updated to work correctly on reprojected data. Previously, asphist assumed that _NOM and _PNT header values were the same when calculating the offsets, which is not the case for reprojected data.

  The updated algorithm uses the full tangent plane offset computations and to correct angle between pointing and North. Both of these affect data that have been reprojected to different tangent points and/or multi-obi observations with significantly different pointings.

celldetect

• Improvements to recursive blocking mode when the input image is non-square.

• Bug fix: atan2(~0,~0) gave slightly different results on different platforms.

dither_region

• New parameter: wcsfile is used when the input region file is in celestial format.

• Bug fix: TOTLAREA keyword was not being written to the file header. The TOTLAREA keyword records the total area of the region in units of sky pixels.

dmcoords

• A tolerance check was added to the RA_NOM and DEC_NOM comparisons to improve the accuracy of the WCS comparison.

• The limits of the chip_id parameter are explicitly included in the parameter file (min=0, max=9).

dmextract

• Added a check for floating-point NaNs values in images. The output COUNTS column would be NaN if the region enclosed a NaN-value pixel.

• Fixed a memory problem when working with exposure maps ("DM Error: No kernel found for dataset <exposure map>"). The exposure map was opened once per region, causing dmextract to run out of memory.

• The output bg_exposure values were incorrect because the background exposure file (bgexp) wasn't being applied in lightcurve mode. Added a warning when exp or bkgexp are specified but missing.

• dmextract got the exposure wrong when using a DTF file if the time bins were smaller than the resolution of the DTF file (~2 sec).

• Suppress the warning when no bin column is specified.

dmfilth

• Bug fix: the random seed is only initialized once. The tool was using the same value for each source/background pair in the stack, which makes the filled values highly correlated (if not identical).

dmhedit

• An error message is printed if the "move after" rule is used, since that functionality is broken.

• The filelist parameter can read a file from stdin, i.e. filelist=-.

dmimgfilt

• Improved error checking for missing input files.

dmimglasso

• New parameter: maxdepth sets the maximum recursion depth. A warning is issued if the maxdepth is reached.

dmimghull

• dmimghull creates a convex hull around pixels in image.

dmimgpick

• Clarified the logic when using method=closest and point is equidistant from pixel centers. This bug was leading to build-specific behavior.

dmimgpm

• Bug fix: the tool would SegV if the region contained zero pixels.

dmimgthresh

• Bug fix: the tool checks for integer BLANK values when cut=INDEF.

dmkeypar

• Bug fix: dmkeypar would return bad data if the filter selected zero rows.

dmlist

• Using the "raw" mode will suppress the mapping from null value to 'NULL' string, so that the actual value is seen.

dmsort

• Bug fix: using "copyall=no" with the "opt=text/simple" format.

  The text/simple ASCII format does not support IMAGE blocks. When sorting a table with the default value of the dmsort copyall parameter (yes), the tool attempted to copy the other blocks from the input file, including the NULL PRIMARY image.

  unix% dmsort simple.fits "simple.sort.dat[opt kernel=text/simple]" keys=y
  # DMSORT (CIAO): Error adding full header. 
  

dmstat

• Bug fix: the absolute value was being taken of the coordinate transform, meaning a negative Dec could become postive.

dmtabfilt

• dmtabfilt applies non-linear filters to columns in a table.

eff2evt

• Bug fix: the FLUX_LO/HI/WEIGHTED columns should only be created in the output file when the rmfimg parameter is set.

get_src_region

• Improved behavior for images with linear WCS.

hrc_dtfstats

• The DTF is computed as a simple mean instead of a variance-weighted mean to address a bug in computing average deadtime corrections for ObsIDs with telemetry saturation, and to make the program consistent with the HRC deadtime correction algorithm used in dmextract.

imgmoment

• imgmoment computes up to the second moment of an image.

lim_sens

• The tool checks the units of the psffile and converts them to logical pixels, if necessary. An additional check has been added to confirm that the infile and psffile files are congruent.

mean_energy_map

• mean_energy_map creates image whose pixels are the exposure-weighted mean energy of the pixel.

mkacisrmf

• propagate verbose setting to caldb

• If the upper limit of the energy specification was greater than 12 keV, the tool would segmentation fault.

mkpsf

• CIAO 4.4 is the last software release that will include the mkpsf tool. It will be removed from the next version of CIAO.

  Users can run ChaRT, SAOTrace, and/or MARX to simulate the PSF for their exact source location and spectrum which will yield much better results than the current set of PSF "hypercubes."

mkpsfmap

• The tool has been updated to work correctly on reprojected data. Previously, mkpsfmap assumed that _NOM and _PNT header values were the same when calculating the offsets, which is not the case for reprojected data.

mkrmf

• The verbosity setting is propagated to the CALDB lookups.

mkwarf

• Bug fix: there would be NaN values in the ARF if there are time header keywords with a 0 value. mkwarf has been updated to handle the case where there is flux in the WMAP but zero time for the chip (i.e. it has been filtered out of the file).

pfold

• pfold folds an event file into a grid of period bins with a fixed number of phase bins.

proposal tools

• Several bug fixes were applied to the "From Galactic" option. When using the "From Galactic" option in Cycle 13:

  • incorrect galactic coordinates were echoed to the output.
  • with the NRAO dataset, an incorrect NH was calculated.
  • (with the Bell dataset, the correct NH wa returned.)

  These issues are fixed in CIAO 4.4/Cycle 14.

• The prop_dates tool allows numeric months, i.e. "Dec" and "12" are both valid inputs.

reproject_aspect

• The tool has been rewritten in Python. The functionality of the tool is unchanged.

reproject_events

• The units and comments for the SIM_X/Y/Z keywords are copied to the output file. This means that the comment and unit fields will be blank if they are blank in the input.

roi

• roi generates source and background regions from a source list.

sky2tdet

• New parameter: bin increases the blocking factor of the WMAP in chip coordinates.

  The default value of bin=1 will be sufficient for most users. When making a weighted ARF for a large region, on order of a full chip (or larger), mkwarf may run out of memory; in this case, the bin value should be increased to create a coarser WMAP.

• The tool retains the data subspace and adds a history entry to the header of the output file.

• Additional error checks have been added to the input files to confirm that the necessary columns are present and that the input and output images are not empty (zero counts).

skyfov

• If the mask file indicates that the chip was entirely inactive for the observation, the chip is excluded from the output file.

vtpdetect

• All computations were changed from floats to doubles in order to prevent a loss of precision when computing the exposure-weighted areas.

wavdetect

• The wavdetect tool takes an observation-specific PSF map file - created by mkpsfmap and set in the new psffile parameter - instead of using the Chandra PSF table data or the tool's internal parameterization of the ROSAT PSF.

  To run mkpsfmap, use the eband and eenergy values from your CIAO 4.3 wavdetect command and your input image:

  unix% mkpsfmap infile=<image> outfile=psfmap.fits energy=<eband> ecf=<eenergy>
  

• The old PSF parameters have been removed: psftable, xoffset, yoffset, eband, and eenergy.

• increased limits on number of wavelet scales to 127 (from 15)

wrecon

• The wavdetect tool takes an observation-specific PSF map file - created by mkpsfmap and set in the new psffile parameter - instead of using the Chandra PSF table data or the tool's internal parameterization of the ROSAT PSF.

  To run mkpsfmap, use the eband and eenergy values from your CIAO 4.3 wavdetect command and your input image:

  unix% mkpsfmap infile=<image> outfile=psfmap.fits energy=<eband> ecf=<eenergy>
  

• The old PSF parameters have been removed: psftable, xoffset, yoffset, eband, and eenergy.

• increased limits on number of wavelet scales to 127 (from 15)

wtransform

• increased limits on number of wavelet scales to 127 (from 15)

---

Parameter Files

acis_run_hotpix

• Additional acis_build_badpix parameters may be set from the acis_run_hotpix parameter file: This section removed when the tool was deprecated.

addresp

• Three new parameters have been added: phafile, method, and type.

aprates

• New parameter: clobber allows an existing output file to be overwritten.

combine_spectra

• New parameter: method specifies whether the script should sum or average the PHA exposure times for the header of the combined spectrum and ARF files.

dither_region

• New parameter: wcsfile is used when the input region file is in celestial format.

dmimglasso

• New parameter: maxdepth sets the maximum recursion depth. A warning is issued if the maxdepth is reached.

sky2tdet

• New parameter: bin increases the blocking factor of the WMAP in chip coordinates.

  The default value of bin=1 will be sufficient for most users. When making a weighted ARF for a large region, on order of a full chip (or larger), mkwarf may run out of memory; in this case, the bin value should be increased to create a coarser WMAP.

specextract

• New parameter: binarfwmap specifies the binning of the sky2tdet WMAP, which is used as input to mkwarf.

wavdetect

• The wavdetect tool takes an observation-specific PSF map file, created by mkpsfmap and set in the new psffile parameter.

• The old PSF parameters have been removed: psftable, xoffset, yoffset, eband, and eenergy.

---

Data Model

PyCrates

• The CIAO 4.4 release includes a pycrates module which has been reworked and rewritten entirely in Python. This module replaces previous python-wrapped C++ module and library.

  The high level user interface (HLUI) and Crates API remain essentially the same, but we have added several new features and capabilities:

  • addition of the CrateDataset which allows users to read and write files with multiple extensions

  • addition of 'update' mode allowing users to update a dataset or extension in place.

  • use lazy initialization to save time and memory resources by reading header and data from a file only when access is needed.

  • named attributes are provided for easy access to regular (non-virtual) columns. For example, access the time column as "tab.time" and the x column of a sky(x,y) vector as "tab.sky.x".

  • Definition of a specialized PHACrateDataset for handling of OGIP compliant PHA files, both Type:I and Type:II.

• Removed support for the ARF routines (read_arf, write_arf, and is_arf), as these are now treated as normal tables.

• Caveats/Issues:

  • Users may get unexpected results if the same file is opened to more than one variable. Any open done in read-only mode will result in subsequent opens of that file to be invalid, but does not raise an error. Also note that doing so in update mode will open more than one path to modifying file content.

  • PyCrates file handles are not properly closed and garbage-collected when the references to a Crate are freed. As a work-around, the user may:

    • explicitly call the CrateDataset.__del__() function to close all file handles.
    • or use the CrateDatsaet.snip() method, which breaks the underlying circular dependencies that are preventing normal garbage collection.

    Either option should be used only when the user is ready for the objects to be garbage collected.

ASCII Kernel

• Bug fix: the ASCII kernel could not access the last column name or value of a file if it includes the carriage return character (\r). The "\r" is included as part of the text/numerical value, which broke the parsing.

Additional Enhancements & Bug Fixes

• Using the "raw" mode option will suppress the mapping from null value to 'NULL' string, so that the actual value is seen.

• When using weighted binning syntax, transfer the units of the weighting column as BUNIT of the image.

• The syntax [bin x=#10] may be used as shorthand for [bin x=::#10].

• Bug fix: correct a problem when parsing filter syntax which contains two adjacent separators (commas) or the filter starts with a comma, e.g. file.fits"[ccd_id=7,energy=10000:12000][,sky=region(fov1.fits[ccd_id=7])]".

• Bug fix: if a filter that contains parentheses is applied to a file, subsequent filtering will not read the subspace correctly, resulting in incorrect GTI information.

• Bug fix: if an ASCII filter does not contain a newline, the software reports that the filter cannot be read.

---

ChIPS

ChIPS GUI

• New in CIAO 4.4, the ChIPS graphical user interface (GUI) allows users to adjust the layout of their plots and make edits to many of the plot properties. A Python terminal is built into the GUI, allowing access to ChIPS and other Python commands. The GUI is available to any process that creates a ChIPS plot, such as Sherpa and Prism.

  The GUI is launched by right clicking on an existing ChIPS window and selecting "Show GUI" or by running the show_gui command from within a ChIPS session.

  Features include:

  • the ability to edit the properties of an object, such as the symbol style of a curve or the font used for an axis label,
  • zoom, pan, and select objects in the plot,
  • update the plot ranges,
  • move plot areas and annotations,
  • resize plot areas, labels, and points,
  • and an object tree which describes existing plot objects.

  Futher information is available from the built-in help menu in the GUI and via "ahelp chipsgui".

Plot Translucency in PDFs

• The PDF output created by ChIPS supports the transparency and alpha settings.

  The Postscript (PS and EPS) output formats do not support the alpha settings of images or the opacity settings of region or histograms. The attribute values are treated as 1 when the objects are displayed in these formats.

Manipulating Plot Objects

• A new function, highlight_object, can be used identify objects in a window by causing them to blink for approximately three seconds.

• Window layer grid lines are available to help layout a user's visualization. These lines are for visual cues only and will not appear in printouts. Manipulating the grid lines is available via the GUI.

Plot Labels and Text Formatting

• Improvements have been made in the parsing of labels. Parsing would fail when escaping underscores, and the superscript option was not being applied if it was at the beginning of the label.

• In addition, circumflex support for vowels was changed to use math/formula syntax: \hat{a} to disambiguate a circumflex character vs a superscript character. The carat character (e.g. ^{O} or ^a) is used for superscripts. For consistency, \acute, \grave, \tilde, and \ddot support was also added (but the text mode formats \`, \', \~, \", and \c are still supported).

Additional Enhancements & Bug Fixes

• The get_server_cwd command returns the working directory that is used by the ChIPS server and the GUI print dialog.

• The numeric accuracy of colorbar labels has been improved to have a nicer display. Due to very small numercial differences in the tick label values, they were not always well-formatted.

  The colorbar title now updates its location when the colorbar width is changed.

• Bug fix: make_script would failed is the current plot was set to "all". The edge.style for regions was also not written to the output script.

• Bug fix: pick_limits(X_AXIS) would also change the Y axis values.

• Bug fix: the split command will create a frame if one doesn't exist, using the specified window.

• Bug fix: set_cascading_property would not work on the properties of annotation objects - including labels, lines, points, regions, and colorbars - when the object was used directly.

• Bug fix: the add routines for annotations failed when run with a dictionary of attributes instead of a list or a string.

• Bug fix: hex were values are not being accepted as valid colors in attribute lists when used with any of the add commands.

• Bug fix: the plot title position (title.ypos and title.xpos) was not saved correctly to the state file.

---

Sherpa

Bayesian Analysis with pyBLoCXS

• The pyBLoCXS module is now integrated into Sherpa. pyBLoCXS is a sophisticated Markov chain Monte Carlo based algorithm designed to carry out Bayesian Low-Count X-ray Spectral (BLoCXS) analysis in the Sherpa environment. (For more information on BLoCXS, see van Dyk, D.A., Connors, A., Kashyap, V.L., & Siemiginowska, A. 2001, Ap.J., 548, 224, "Analysis of Energy Spectra with Low Photon Counts via Bayesian Posterior Simulation")

  The code is a Python extension to Sherpa that explores parameter space at a suspected minimum using a predefined Sherpa model to high-energy X-ray spectral data.

  • list_priors()
  • set_prior()
  • get_prior()
  • list_samplers()
  • set_sampler()
  • set_sampler_opt()
  • get_sampler()
  • get_sampler_opt()
  • get_sampler_name()
  • get_draws()

Visualization of Statistical Distributions from Simulations

• There is a new PPP statistic function, plot_pvalue(), to compute the likelihood ratios of the null and the alternative models. This function can help users determine if the fitted model is statistically significant compared to the null model.

  The get_pvalue_results() command returns the likelihood ratio test results computed by plot_pvalue().

  The get_pvalue_plot() command accesses the histogram plot of the likelihood ratios comparing fits of the null model to fits of the alternative model using faked data with Poisson noise.

• There are new statistics plot functions associated with the pyBLoCXS module to plot statistical distribution functions.

  • plot_pdf()
  • plot_cdf()
  • plot_trace()
  • plot_scatter()
  • get_pdf_plot()
  • get_cdf_plot()
  • get_trace_plot()
  • get_scatter_plot()

Fitting Template Models to 1D Data

• Sherpa supports a new template model (an extension of table model). Sherpa can now read in a collection of templates from a directory full of template files, and can compare a data set to all the templates in that collection. Sherpa finds the template that best matches the data, and reports back the parameter values associated with that template.

  The files that contain template models can be read into Sherpa via the new load_template_model() function, which has a similar signature to the load_table_model() function. The function loads templates listed by a table-of-contents file (templatefile) into a Sherpa template model (modelname).

  The gridsearch method will select the best template model by trying all of the templates.

• There is new support to overplot template models with plot_source_component().

• Table and template models now support linear, nearest-neighbor, and polynomial interpolation. (Interpolation is used by both table and template models to match the data grid to the model grid, which must match before the fit statistics can be calculated for fitting.)

Optimization Methods

• The Monte Carlo optimization method has been significantly improved, and can now find more reasonable parameter values in cases where it formerly came up against parameter limits.

• The LMDIF optimization method has also been updated to avoid getting "stuck" at the parameter space boundaries for certain cases.

• There is a new grid searching optimization method, called gridsearch. The user must provide this method with a parameter space grid to search. The method will evaluate the fit statistic for each point in the grid; the best match is the grid point with the lowest value of the fit statistic. The grid search method will report back the parameter values associated with this point.

Modeling and Fitting with PSFs

• 2-D PSF kernels can now be larger than the source data (before we restricted the kernel to be equal to or smaller that the source). FFT padding 2^n occurs using the largest of the two sizes. The PSF padding is removed down to the size of the source data.

• A fix to the PSF utility 'normalize': it no longer returns NaN when the PSF kernel is all zeros.

• The WCS in 2-D PSFs kernels is changed according to subkernel extraction. In previous versions of Sherpa, the WCS of subkernel images is wrong. The image sources now line up according to the WCS, consistent with CIAO 3.4. NOTE: this fix only affects imaging. The kernel WCS is not used during fitting.

• The normalization of a kernel now occurs before kernel extraction for both file-based and model-based PSFs.

• The coordinate system of kernel plots/contours is now based on the kernel or source data coordinate system. Originally, the coordinate system was based on the number of bins in each dimension.

• The location of model-based PSFs is now updated based on PSF parameters when set_psf() is used to associate the kernel with a source model. This is simply a convenience if the user forgets to update the kernel model parameters.

• 1-D PSF plots (from file and model) are exclusively scatter plots. (Model PSFs are no longer plotted as a red line).

  2-D PSF contours (from file and model) are exclusively scatter plots. (Model PSFs are no longer plotted as a red contour lines).

Models and Parameters

• The model library has been extended to include 24 optical models that are designed for use with spectra or SEDs at optical wavelengths. These include continuum models, absorption and emission features, and extinction curves for the ISM at optical and UV wavelengths. The position, line width, and other similar model parameters have units of Angstroms where relevant.

  • AbsorptionEdge
  • AbsorptionGaussian
  • AbsorptionLorentz
  • AbsorptionVoigt
  • AccretionDisk
  • BlackBody
  • Bremsstrahlung
  • BrokenPowerlaw
  • CCM
  • EmissionGaussian
  • EmissionLorentz
  • EmissionVoigt
  • FM
  • LMC
  • LogAbsorption
  • LogEmission
  • OpticalGaussian
  • Polynomial
  • Powerlaw
  • Recombination
  • Seaton
  • SM
  • SMC
  • XGal

• Several significant bugs associated with the functions set_full_model() and set_bkg_full_model() have been fixed. These functions are intended to allow users to define more complex models (i.e. models with some components that are folded through an RMF or a PSF and other components that are not).

• Sherpa includes the "additive" and "multiplicative" models of XSpec version 12.7.0.

  New XSpec models included in Sherpa are:

  • xsbvvapec
  • xscompth
  • xscplinear
  • xseqpair
  • xseqtherm
  • xsvvapec
  • xszigm

• Bug fix: improved the behavior of powlaw1d at x=0.0. Since the model is not defined at that value, attempting to evaluate it would result in a "model evaluation failed" error.

Additional Enhancements & Bug Fixes

• The OGIP AREASCAL keyword within Type I and II PHA files is now supported in Sherpa.

• There is a new math function from the CEPHES library, "inverse erf", in the sherpa.utils module. This function is useful for converting between confidence limits and confidence intervals (sigma/percent).

• The ATOMDB v2.0.1 database is now distributed with CIAO 4.4, and can be used with XSPEC models (such as the APEC models) that depend on the database.

• PyFITS 3.0 is now supported in the alternate PyFITS I/O backend.

• The get_arf function includes a reference to the associated RMF (if available) when used in set_full_model.

• The save command records the XSpec-related settings, such as abundance and xsect.

• Models defined by set_full_model are correctly written to the save_all output file.

• Bug fix: The axis scale of plots output by the plot_model_component and plot_source_component commands could not be changed to log scale with the set_xlog and set_ylog command.

• Bug fix: the log keyword argument was not working correctly for the reg_proj and reg_unc commands.

• Bug fix: the projection and conf functions were not using the numcores setting in the .sherparc file.

---

GUIs (Graphical User Interfaces)

prism

• Bug fix: Quick Histogram would occasionally fail to plot.

---

Analysis Scripts

chandra_repro

• The tools acis_find_afterglow (new in CIAO 4.4) and acis_build_badpix are used in place of acis_run_hotpix to identify cosmic ray afterglows and bad pixels in ACIS observations. Details are in the "badpixel" parameter description of the ahelp file.

  If badpixel=yes (the default) in CIAO 4.4, a new bad pixel file is made for all ACIS observations, except those taken in continuous-clocking mode.

• All of the ACIS bad pixel and afterglow status bits are reset with the acis_clear_status_bits script before the new bad pixel file is made. Previously, only the bits set by acis_detect_afterglow were reset.

• The destreak tool is run before the bad pixel tools are applied. Removing the streaks first improves the accuracy of the afterglow correction.

• The default filename root is chosen from the first segment of the level=1 event file, e.g. changed from "acisf01838_000N002" to "acisf01838".

combine_spectra

• New parameter: method specifies whether the script should sum or average the PHA exposure times for the header of the combined spectrum and ARF files.

• Bug fix: the ANCRFILE header keyword in an output combined background spectrum was incorrectly being set to "NONE" when the input backround response files exactly matched the input source responses.

• Internal updates to support the new addresp phafiles and method parameters.

specextract

• New parameter: binarfwmap specifies the binning of the sky2tdet WMAP, which is used as input to mkwarf.

  The default value of binarfwmap=1 will be sufficient for most users. When making a weighted ARF for a large region, on order of a full chip (or larger), mkwarf may run out of memory; in this case, the binarfwmap value should be increased to create a coarser WMAP.

  The binwmap parameter is stiil used for creating the WMAP used with mkacisrmf.

• The remove_extra_time_keywords has been removed from specextract. It is no longer needed, since the sky2tdet bug is fixed in CIAO 4.4.

Python 2.7 compatibility

• Many of the scripts were updated to be compatible with python 2.7. These changes should be transparent to the user.

Removed: psextract

• The psextract script has been removed from the tarfile. specextract should be used in psextract. A comparison of the specextract and psextract syntax:

  specextract infile="evt2.fits[sky=region(src.reg)]" bkgfile="evt2.fits[sky=region(bg.reg)]" \
  outroot=point asp=point.asphist pbkfile=acisf063875928N003_pbk0.fits \
  weight=no correct=yes \
  grouptype=NUM_CTS binspec=15
  
  psextract events="evt2.fits[sky=region(src.reg)]" bgevents="evt2.fits[sky=region(bg.reg)]" bkgresp=yes \
  root=point asol=@pcad_asol1.lis pbkfile=acisf063875928N003_pbk0.fits \
  gtype=NUM_CTS gspec=15
  

  Contact CXC Helpdesk if you need assistance transitioning from psextract syntax to specextract.

Removed: remove_extra_time_keywords

• The remove_extra_time_keywords has been removed from the tarfile. It is no longer needed, since the sky2tdet bug is fixed in CIAO 4.4.

---

Libraries

cxcparam

• Bug fix: the paccess() routine was ignoring the mode argument as always returning that the file has "rw" access.

dslib

• If an HDUCLAS keyword is blank, omit it from the output file.

hdrlib

• If the new SIM average-offset keywords (DY_AVG, DZ_AVG, DTH_AVG) are present in the input file, they are copied in the same component as the SIM keywords.

pixlib

• Improved error checking on the allowed range of chip ids.

regionlib

• Added a NULL check to regShapeGetName, e.g. to prevent a tool from crashing if multiple sky(x,y) filters do not intersect.

tcdlib

• Two new kernel options have been added: hanning and racos; see "ahelp aconvolve" for details.

• Bug fix: if the axis length was not a power of two, the padding was doubled to the next power of two. The extra padding would create much larger images and require large amounts of memory.

---

Environment

Off-the-Shelf (OTS) Package Versions

• The following OTS packages are included with CIAO 4.4. For more information on how the OTS packages are built for use with CIAO, refer to the INSTALL_SOURCE file distributed with the software.

  • CCfits v2.3
  • cfitsio v3.280
  • fftw v3.3
  • fontconfig v2.8.0
  • freetype v2.4.4
  • gsl v1.15
  • ipython v0.10.1
  • NumPy v1.5.1
  • pyds9 v1.3
  • Python v2.7.1
  • readline v6.2
  • vte v0.25.1
  • VTK v5.6.1
  • wcssubs v3.8.3
  • xpa v2.1.13
  • XSpec v12_7_0 (models only)

  Two OTS packages are no longer included with CIAO:

  • ghostscript
  • slang

---

Documentation

Analysis Threads

• The CIAO, Sherpa, and ChIPS Threads are in the process of being tested and updated for the CIAO 4.4 release. Please check the "Last Update" information at the top of each thread to see if it has been reviewed.

---