CIAO 4.8 Release Notes

CIAO 4.8 is distributed for the following platforms:

• Linux 64-bit
• Mac OS X 64-bit 10.11 (El Capitan)
• Mac OS X 64-bit 10.8 (Mountain Lion) compatible with 10.9 (Mavericks) and 10.10 (Yosemite)

CIAO is no longer available for 32bit Linux operating systems or for older 64bit Linux machines (CentOS 5 era). CIAO is also no longer available for older versions of OSX (notably 10.6 and 10.7). CIAO 4.7 is still available for users who are unable to upgrade. More details on the Platform Support page.

Notable changes and improvements in CIAO 4.8:

• This is primarily a maintenance release: bug fixes, supporting new compilers and OTS upgrades.

• CIAO includes version 7.3.2 of SAOImage ds9: users may find that they are unable to select or create region files using this version due to a change in ds9; please see the watchout page for more information on this.

• The major updates for Sherpa are: the XSPEC models have been updated to version 12.9.0d; the wstat statistic (based on the XSPEC version) which extends the cstat statistic to include a Poisson background component; user statistics now have access to the background of a data set; and this is the first version to be based on the Sherpa GitHub repository.

• ChIPS has been updated to support XQuartz version 2.7.8 on OS-X systems.

• There have been several updates to acis_process_events and tg_resolve_events to allow for better calibrations of ACIS+HETG data taken in Continous Clocking (CC) mode. chandra_repro has been upgraded to take advantage of these changes.

• Updates to reproject_aspect (including wcs_match and wcs_update) and to axbary have significantly simplified their respective threads: Correcting Absolute Astrometry and Apply Barycenter Correction.

• The CXC datamodel has improved support for filtering on string columns and supporting NULL values for logical (True/False) data-types. Filtering and area calculations involving complex polygons (sides intersect) have also been improved.

• There have been several new scripts released since the CIAO 4.7 release, including: readout_bkg, install_marx, download_obsid_caldb, detilt, dewiggle, symmetrize, splitobs, gti_align, and multi_chip_gti. The add_grating_orders and add_grating_spectra scripts have been removed in this release; the combine_grating_spectra script should be used instead.

• How CALDB 4.7.2 Affects Your Analysis
• How CALDB 4.7.1 Affects Your Analysis
• How CALDB 4.7.0 Affects Your Analysis
• Installation
• Tools
• Parameter Files
• ChIPS
• Sherpa
• Graphical User Interfaces
• Analysis Scripts
• Python Modules
• Libraries
• Environment
• Documentation

How CALDB 4.7.2 Affects Your Analysis

ACIS graded CC-mode

• ACIS GRADE file version N0005 retracted

  The N0005 version of the ACIS grade file is retracted, which only affects ACIS graded CC-mode observations taken since 2009 November 1, when flight grade 66 events were first telemetered with CC-mode data due to an on-board operational change. No new calibration data are included in CalDB 4.7.2; only the default ACIS GRADE file and the modified ACIS QE for CC-mode are altered in the CalDB index.

  While this changes the name of the ACIS GRADE file selected in acis_process_events (used by the chandra_repro script) for all ACIS ObsIDs with DATE-OBS later than 2009-11-01T00:00:00 (UTC), this only affects the value of GRD_FILE in the headers of your resulting events files.

  Unless you are using GRADED CC-mode data (DATAMODE=CC33_GRADED), this filename change will have NO EFFECT on your event lists.

  For users of GRADED CC-mode data taken since 2009 November 1, the GRADE mapping is being reverted to eliminate all flight grade "66" events from level 2 events files. These events have been found to introduce only background noise to the resulting grating spectra in this mode.

  Because the flight grade "66" events are now being removed in reprocessing for GRADED CC-mode taken since 2009 November 1, these observations should be analyzed using the modified ACIS QE file for CC-mode, which was introduced in CalDB 4.7.0 (December 2009). With CalDB 4.7.2, this modified QE_CC file will automatically be selected in CIAO for ALL GRADED CC-mode data.

How CALDB 4.7.1 Affects Your Analysis

ACIS Imaging and Grating Data

• Time-dependent ACIS Gain (T_GAIN) Files for -120 C Data

  The new, time-dependent ACIS Gain (T_GAIN) files for November 2015-January 2016 (Epoch 64) are introduced and updated for Epoch 63 in this release, and therefore, affects observations taken since 2015 August 01. Other observations are unaffected by these new calibration files.

  Users working with ACIS data taken since 2015 August 01 may wish to run chandra_repro and reprocess the data to improve the T_GAIN calibration. The DATE-OBS header keyword records the observation start date.

  See the time-dependent ACIS T_GAIN why page for more information.

  The gain corrections are at the nominal and expected levels of less than 2% of the energy value and users interested in CCD spectroscopy may benefit in applying the new gain adjustment. Grating spectroscopy benefits as well, in the form of improved order sorting. It is unnecessary to apply this adjustment for only doing timing or imaging analysis, although doing so will not have a negative effect.

  Note that only spectra with several hundreds of counts and/or prominent features (in emission or absorption) will show changes from the T-gain refinement that exceed the uncertainties from the gain calibration.

How CALDB 4.7.0 Affects Your Analysis

ACIS Imaging and Grating Data

• Time-dependent ACIS Gain (T_GAIN) Files for -120 C Data

  The new, time-dependent ACIS Gain (T_GAIN) files for August-October 2015 (Epoch 63) are introduced and updated for Epoch 62 in this release, and therefore, affects observations taken since 2015 May 01. Other observations are unaffected by these new calibration files.

  Users working with ACIS data taken since 2015 May 01 may wish to run chandra_repro and reprocess the data to improve the T_GAIN calibration. The DATE-OBS header keyword records the observation start date.

  See the time-dependent ACIS T_GAIN why page for more information.

  The gain corrections are at the nominal and expected levels of less than 2% of the energy value and users interested in CCD spectroscopy may benefit in applying the new gain adjustment. Grating spectroscopy benefits as well, in the form of improved order sorting. It is unnecessary to apply this adjustment for only doing timing or imaging analysis, although doing so will not have a negative effect.

  Note that only spectra with several hundreds of counts and/or prominent features (in emission or absorption) will show changes from the T-gain refinement that exceed the uncertainties from the gain calibration.

• ACIS CC-mode

  CalDB 4.7.0 and CIAO 4.8 bring improved CTI-correction and QE for all ACIS CC-mode observations.

  The improved CTI-correction for CC-mode is included in the standard data processing (SDP version 10.4.2)—via the acis_process_events tool—since 2015 September 30. Users working with ACIS CC-mode data observed prior to 2015 may wish to run chandra_repro and reprocess the data to improve the calibration. Applying the new CTI-correction changes the corrections to photon CCD energies (i.e. PI pulse heights) versus position on the detector by up to ~100 eV. It changes the photon arrival time versus position by up to ~1 secound.

  Since the CTI-correction is now working correctly for CC-mode, the EDSER sub-pixel repositioning algorithm is also applied to all CC-mode observations in reprocessing. If sub-arcsecond positioning is crucial, chandra_repro needs to be run for data prior to 2015 September 30.

  The SDP version and observation date are recorded in the header keywords ASCDSVER and DATE-OBS, respectively.

  An adjustment to the ACIS QE is required for CC-mode observations prior to 2009 November 01: the adjustment is provided in CalDB 4.7.0. The effect is only present above 3 keV, and these modified calibration data will be selected automatically for the appropriate CC-mode observations in the software— by mkarf, mkwarf, mkgarf, and mkinstmap; and scripts that invoke these tools such as specextract, fullgarf, and fluximage.

HRC Imaging Data

• HRC-I Background Event Files

  New HRC-I background event files are included in this release, being generated using the latest level 1 calibration data, including SAMP-based gains (GMAPs) and the latest DEGAP corrections. The newly introduced background files cover the years 2014-2015. All users performing HRC-I imaging or spatial analyses, e.g. when creating a fluxed image, are encouraged to download and use the background files.

  The HRC-I background event files are not included in the main CALDB tarfile. There is a separate HRC background event tarfile available via ciao-install or from the the the CALDB Download page.

  For instructions on using the background event files, follow the The HRC-I Background Event Files thread.

• HRC-I Gain Maps

  The latest time-dependent HRC-I Gain Map (GMAP) and the corresponding PI background spectrum—to be applied to observations after 2015 September 27, have been released.

  Note that HRC-I gain maps are only useful to observers doing hardness ratio or PI spectral analysis studies with HRC-I.

  As the HRC gain continues to slowly decline, the gain maps are updated annually. The change in gain correction is less than 1% on-axis and roughly 5-10% off-axis. Towards the edge of the microchannel plate, the change is up to 15%. Technical details on the gain maps are described in the gain maps section of the HRC calibration website and the in the HRC-I Gain Map Why Document.

  The corresponding background spectrum applicable to the same HRC-I GMAP time period have been generated.

  Users working with HRC-I data may wish run chandra_repro to reprocess the data and improve the gain calibration.

Installation

Supported Platforms

• CIAO 4.8 is supported on:

  • Linux 64 bit (tested on CentOS 6.6 and Red Hat Enterprise 6 )
  • Apple OS X 10.8 & 10.9; & 10.10 64 bit
  • Apple OS X 10.11 (El Capitan)

Updates to the ciao-install Installation Script

• Added switches --copy-caldb and --link-caldb to allow users to bring in an existing copy of the CALDB into CIAO. This may allow users to get the CALDB PATCH instead of downloading the full CALDB. For example

  unix% bash ciao-install --copy-caldb /soft/ciao-4.7/CALDB
  

  Will copy the CALDB files from the /soft/ciao-4.7/CALDB directory to the ciao-4.8/CALDB directory and will then check to see if it need to be patched to the latest version.

• A new --remove flag has been added to ciao-install that allows user to remove the graphics package, and thereby letting CIAO use the system version of the GTK libraries.

  unix% bash ciao-install --remove graphics
  

  This can help some Linux users who experience problems with the Oxygen GTK theme, especially on Ubuntu systems.

• Fixed a bug that prevented users from using relative paths when downloading and installing in separate directories.

• Some OSX users reported problems downloading the CIAO tar files. This was frequently tracked to the ciao-install script using an incompatible version of ftp. The script now uses /usr/bin/ftp to avoid this problem.

Remove old parameter files

• With every new CIAO release, some parameter files are changed: new parameters may be added and occasionally old ones removed or renamed. Deleting or renaming the local parameter directory ensures that the correct parameter files will be accessed the first time a tool is run:

  unix% rm ~/cxcds_param4/*
  

IPython Settings

• The CIAO Ipython directory has changed from ~/.ipython-ciao4.5 to ~/.ipython-ciao.

  Users who had previously installed CIAO 4.0 through CIAO 4.4 and already have this directory will be notified that the obsolete directory has been moved to a different name to make way for the updated IPython configuration files. This happens the first time users start either ChIPS or Sherpa.

  unix% chips
  
   ATTENTION: The old IPython files found in /home/user/.ipython-ciao are not compatible with 
   this version of chips. They have been moved to /home/user/.ipython-ciao/old_ipython.
  
  -----------------------------------------
  Welcome to ChIPS: CXC's Plotting Package
  -----------------------------------------
  CIAO 4.8 ChIPS version 1 Thursday, December 3, 2015
  
  chips-1>
  

Tools

axbary

• The tool no longer updates the RA_NOM and DEC_NOM keywords in the output files. RA/DEC corrections are identified in the RA_BARY and DEC_BARY keywords
• Several new keywords have been added to be more consistent with the latest FITS WCS standards. These include: TREFDIR, TREFPOS, TIMRDER, TIMSYER, OBSORBIT, XPOSUR#, TCNAM1, XPOSURE. The keywords are added to appropriate FITS block. Also the tool now modifies the existing MJD_OBS keyword value based on the time corrections.
• Updated for 2015 leap second.

acis_process_events

• There have been significant updates to support Continuous Clocking (CC) mode, especially when used together with the HETG.
  • The EDSER subpixel algorithm is now available for CC imaging mode datasets.
  • acis_process_events can now be used in conjunction with tg_resolve_events to provide a better guess at the true CHIPY location of HETG dispersed events. The CHIPY estimated by using the known grating clocking angles, chipy_tg can be fed back into acis_process_events, and it will use that value to calibrate the event file (gain, CTI, times). The Reprocessing Data to Create a New Level=2 Event File shows how to iteratively use the tools to obtain better calibrated results.
  • The calc_cc_times parameter has been removed; this step will always be performed for CC mode datasets.
  • Two new event definitions have been added for gratings+CC mode: cclev1a and ccgrdlev1a.
• Coordinates for events detected at times outside the range of aspect solution are now set to NULL (for integers) or NaN (for reals) values.
• The alignmentfile parameter has been deprecated.
• Additional error checks on input parameters.

arfcorr

• Added a new tmpdir parameter to allow users to control the location of temporary files.

dmcopy

• Clean up of error messages to remove excess exit_upon_error chatter.

dmellipse

• The output file contains a new column with the requested fraction (in addition to the actual fraction achieved). The output file also now contains a FRACTOL keyword which records the tolerance on the fraction.
• Corrected typo in parameter description.

dmextract

• The tool will now exit with an error if the weight column is requested (it is optional) and it can not be found.

dmhistory

• Now sets exit status to a non-zero value if the requested tool is not found in the history of the file.

dmimgcalc

• Additional error checking when working with arbitrary expressions, and aets the exit status to a non-zero value when an error occurs.

dmimgfilt

• Corrects a bug in the kuwahara function logic.

dmimglasso

• Corrects a bug if the output filename is REGION (case insensitive).

dmimgpm

• Improved error checking

dmsort

• Corrects logic when copying all blocks from input to output (copyall=yes).

mkwarf

• The input feffile and output weightfile file are now optional. They either must both be specified or neither should be specified. The weight file is only needed as input to mkrmf.
• The tool now supports creating weighted ARFs for HRC-I and HRC-S datasets. (Including the zero order gratings case.) Users should use a weight map created with either dmextract (setting the wmap parameter), or by using dmcopy to bin an image in either DET or TDET coordinates. The sky2tdet tool cannot be used with HRC data.
• The verbose output has been upgraded to include more details.

modelflux

• Added additional error checking for missing parameter values.

colden

• Correct for NaN values when a column density (nH value) is unavailable.

dates

• Added the 2015 leap second.

reproject_aspect

• A new method parameter has been added to the script. It can either be rst (rotate, scale, translate), or trans (translation only). It is passed to the wcs_match tool.

reproject_events

• Fixes a bug in the interpolation of the RA and DEC coordinates in the aspect solution when the dither crosses the RA=356:0 boundary. Events detected during that short aspect time no longer have bogus sky coordinates.

skyfov

• Updates to error message formats.

tg_resolve_events

• There have been significant updates to tg_resolve_events to provided better calibrated data for ACIS data taken in Continuous Clocking (CC) mode. The changes include the following:
  • Processing the event data for the 2nd-pass-only changed such that for the 0th order events the 2nd pass copies the input to the output.
  • Added a new keyword ORD_ADJ with the value of NONE for the 1st pass and HETG for the 2nd pass.
  • Two new event definitions have beed added for gratings+CC mode: cclev1a and ccgrdlev1a.
• Coordinates for events detected at times outside the range of aspect solution are now set to NULL (for integers) or NaN (for reals) values.
• The range limits on the chipy_zo and chipy_tg columns have been removed (previous limits were incorrect).

wavdetect

• The bug that prevented wavdetect from using the falsesrc parameter with Chandra exposure maps has been fixed.
• Additional error checking has been added for missing (blank) output file names.

wcs_match

• Fix for the verbose output showing incorrect values for the translation matrix when using method=trans.
• Fix for the verbose output not being sent to the logfile.

wcs_update

• The tool now applies the translation matrix to the RA_NOM/DEC_NOM and the RA_PNT/DEC_PNT sets of keywords in the input files (regardless of file type). This keeps the information about the tangent plane and the optical axis consistent with WCS.

Parameter Files

arfcorr

• New tmpdir parameter to control the location of temporary files created by the script.

acis_process_events

• The calc_cc_times parameter has been removed. Times based on the estimated CHIPY coordinates will always be computed for CC mode datasets.
• Two new standard event definitions have been added for CC mode: cclev1a, and ccgrdlev1a.
• The order of the columns specified in the stdlev1 (FAINT and VFAINT mode) and the grdlev1 (GRADED mode) have been changed to group similar columns together. Except when merging files, the order of the columns is irrelevant.

dmellipse

• Corrected typo in the angle parameter description.

reproject_aspect

• A new method parameter has been added to the script. It can either be rst (rotate, scale, translate), or trans (translation only). It is passed to the wcs_match tool.

tg_resolve_events

• Two new standard event definitions have been added for CC mode: cclev1a, and ccgrdlev1a.

ChIPS

TrueType Font Support

• On OSX, support for True Type fonts, added with load_font, has changed. True Type fonts are not supported in vector format outputs (eps, pdf, ps). They are still supported in bitmap formats (png, jpg).

  The standard, built-in fonts are supported in all output formats.

Internal Cleanup

• Cleaned up memory leaks that could result in memory growth when many thousands of plots were created.
• Updated ChIPS to use the new ciao versioning infrastructure.

Sherpa

XSpec

• The XSPEC models have been updated to version 12.9.0d and the extension has been rewritten to be more robust.

Statistics

• A new statistic has been added to Sherpa: wstat (based on the XSPEC version) which extends the cstat statistic to include a Poisson background component.
• User statistics - created with the load_user_stat command - now have access to the background component of a data set.

Documentation

• The high-level User Interface functions - namely those in the sherpa.astro.ui and sherpa.ui modules - have had their Python docstrings updated to match the ahelp files. The contents of these docstrings are now available from the Sherpa application (in earlier releases the docstrings were replaced by a message telling users to use the ahelp function).

IPython

• The location of the IPython configuration files has changed from $HOME/.ipython-ciao4.5/ to $HOME/.ipython-ciao/.

PEP8

• As part of the move to developing Sherpa on GitHub, large parts of the code base were updated to be more compliant with Python's PEP 8 style guide.

Pull Requests

• #44: save_quality now correctly outputs QUALITY as the column name instead of GROUPS.
• #48: Fix up several issues seen in plot labels - titles and Y-axis labels for commands such as plot_data, plot_fit_resid, and plot_chisqr.
• #59: Fix bug #38 (grouping twice gives an IndexError exception). An unhandled corner case in one of the Sherpa internal methods (sherpa.utils.create_expr) was triggering an IndexError when two group_counts operations were performed back to back. The fix handles the case so that applying group_counts twice does not result in an Exception.
• #77: Replace == and != comparisons to None with checks using is and is not.
• #78: OutOfBoundErr exceptions in some sherpa.utils functions are properly caught. Replaced several uses of print and sys.stderr.write with the correct Sherpa logging code.
• #81: Ensure that XSPEC models which fail - for instance, due to missing a needed data file - return 0's for all bins, rather than random values. This should make it more obvious that something has gone wrong.
• #82: The XSPEC "spectrum number" value is now set to 1 rather than 0, when calling the XSPEC model function, as done in XSPEC.
• #83: Removed S-Lang scripts, files, and references in the code
• #84: Clarified error messages in the XSPEC extension and changed some errors from RuntimeError to more appropriate exception types; in particular LookupError, ValueError, and KeyError. This is a backwards incompatible change, since code which previously caught the RuntimeError will not catch the new error.
• #87: Some methods in sherpa.fit assigned mutable objects to default arguments. This has now been fixed. More instances of this issue have been identified (Bug #95) and will removed in the future.
• #90: Added background data to the UserStat class
• #94: Implement the wstat statistic as described in the XSPEC statistics manual.
• #96: Remove the unused myoptfct module.
• #99: Correct the documentation for the set_exposure function.
• #100: Fix bug #97, whereby the save_all function would not create the necessary set_source() line. This does not fix all cases, but it does for many PHA cases, such as:

  sherpa> load_pha('src.pi')
  sherpa> set_source(xsphabs.gal * powlaw1d.pl)
  sherpa> save_all('test.out).
  

  It also ensures that files created by save_all can be run using IPython's %run directive, by explicitly importing the sherpa.astro.ui module.
• #101: Fix handling of non-contiguous bins in XSPEC - i.e. when a model is called with both xlo and xhi arguments but the bins do not fully cover the energy, or wavelength, range. This fixes #62 using XSPEC 12.8.2 (switching to XSPEC 12.9.0 also fixes this issue) and #56. It also fixes a problem with handling of non-contiguous grids when using a table model, where a crash was likely.
• #110: Update the sherpa.astro.datastack module documentation to include information from the CIAO ahelp documentation and to match the style used by the sherpa.astro.ui module.
• #111: Update the documentation to include more information about pyBLoCXS.

Graphical User Interfaces

Analysis Scripts

chandra_repro

• The processing logic for ACIS+HETG data taken in continuous clocking mode has been updated for the new functionality in acis_process_events and tg_resolve_events. The new HETG/ACIS-S/CC-Mode Grating Spectra thread shows the new iterative approach in detail.

install_marx

• Updated to download and install MARX 5.2.0

add_grating_orders

• This script has been removed from CIAO 4.8. Users can use the combine_grating_spectra script to combine grating orders and spectra.

add_grating_spectra

• This script has been removed from CIAO 4.8. Users can use the combine_grating_spectra script to combine grating orders and spectra.

readout_bkg

• This script can be used to model the Out Of Time (OOT) contribution of bright and extended sources to the ACIS background. The OOT events are detected during the ~0.04s it takes to transfer each exposure into the readout array during which source events are detected at a random CHIPY location.

install_marx

• This script automates the standard steps necessary to download, build, and install MARX version 5.2. MARX can be used to simulate Chandra observations and can used with ChaRT to simulate the Chandra PSF.

download_obsid_caldb

• *experimental*

  This experimental script downloads only the subset of Calibration Database (CALDB) files required to analyze one observation at a time. As additional observations are processed, the CALDB will grow incrementally as new files are accrued.

  Some users with limited bandwidth and disk space have reported problems installing the full Chandra CALDB. This script should allow those users to download only the files required for their specific dataset.

  Users who plan to analyze a large number of observations and site adminstrators supporting multiple, varied users will want to continue to download the full CALDB.

detilt

• This script corrects the time-dependent tilt of LETG/HRC-S spectra by adjusting the TG_D column values; generally in combination with the dewiggle and symmetrize scripts to prepare LETG/HRC-S spectra so that narrower spectral extraction regions can be used to improve the S/N of low-S/N data.

dewiggle

• This script corrects the LETG/HRC-S spectra by adjusting the TG_D values of events to remove small "wiggles" to straighten the dispersed spectra; generally in combination with the detilt and symmetrize scripts to prepare LETG/HRC-S spectra so that narrower spectral extraction regions can be used to improve the S/N of low-S/N data.

symmetrize

• This script adjusts the LETG/HRC-S spectra by adjusting the TG_D values of events to make cross-dispersion profiles more symmetric about TG_D=0; generally in combination with the detilt and dewiggle scripts to prepare LETG/HRC-S spectra so that narrower spectral extraction regions can be used to improve the S/N of low-S/N data.

splitobs

• This script automates the steps necessary to separate multi-obi and interleaved-mode observations into individual directories. For multi-obi grating observations, the REGION extension is also correctly attached to the level 2 event file.

gti_align

• Synchronizes the input GTIs to the ACIS exposure boundaries. For most uses this is tiny fraction of the total exposure time; however, if the observation has many GTIs then the accumulated error can be significant.

multi_chip_gti

• Create a single multi-chip GTI from individual GTIs.

Python Modules

crates

• Now creates a default subspace for vector columns as well as scalar columns.
• Improved error checking (added filenames to error messages)
• Fixed problem trying to insert a new column as the first column in a crate.

pixlib

• Fixes problem when trying to do multiple Init calls.

Libraries

datamodel

• Improved support for variable length array columns.

• Filtering on string columns has been greatly improved. Several bugs have been fixed (including some crashes) and the logic has been adjusted to make it consistent with other columns

  [{c}=null]

  Returns null string records (including TDNULL matches if set).

  [{c}=into,STAR]

  Returns all records matching either value

  [{c}=:]

  Returns all non-null records; consistent with other data types.

  [{c}=:A]

  Returns records lexically <= 'A'; excludes null records

  [{c}<con]

  Returns records lexically < 'con'; excludes null records

  [{c}>con]

  Returns records lexically > 'con';

  [{c}<=con]

  Returns records lexically <= 'con'; excludes null records

  [{c}!=!Annulus]

  Returns all records except null and those matching '!Annulus'

  [{c}!=GALAXY:STAR]

  This is now reported as an error. The range syntax is not supported when using the '!=' filter.

  [exclude {c}=Annulus]

  Returns all records except nulls and those matching 'Annulus'

  [exclude {c}=!Annulus]

  Returns all records except nulls and those matching '!Annulus'

  [exclude {c}=between:inside]

  Excludes records within range, exclude ends, exclude nulls (since ends are part of positive filter, should be excluded on invert)

  Datatypes supporting TDNULL (real, string) when using open-ended filter dyntax (such as '<', '>', and 'a:), may get results including the TDNULL records. If the TDNULL value used is on the open end of the filter, they will be included.

  Applying string filters on 2 different string columns, produces incorrect results.

• Add support for tri-state logical data: True, False, and Null. The CXCDM will now recognize and accommodate the FITS convention allowing a Null logical state in table data. Previously, any Null values in data files were internally converted to False (or 0).

  Syntax	Explanation
  [qcol=T]	True records only, (same for t|Y|y|1).
  [qcol=F]	False records only, (same for f|N|n|0).
  [qcol=U]	Null records only, (same for u|2,null).
  [qcol=0,U]	False and Null records.
  [qcol=1,2]	True and Null records.
  [qcol=null,y]	True and Null records.
  [qcol=0:4]	These are all ERROR conditions. Despite allowing a numerical input for the logical filter, range syntax is NOT permitted for the logical type. Attempted range filtering produces an error message.
  [qcol<2]
  [qcol=1:]
  [qcol=:2]
  [qcol>n]
  [qcol<=U]
  [qcol>=Y]
  [exclude qcol=T]	False records only.
  [exclude qcol=F]	True records only.
  [exclude qcol=null]	empty result.

  The Datamodel's ASCII kernel supports the nullstr option, which allows the user to define the string which represents NULL values in a file. It is valid on both input and output file specifications.

  filename[opt nullstr=somevalue]
  

  This option/setting will now apply to logical data as well as the numerical and string types.

psf

• A bug in the PSF fraction routine, psfFrac has been corrected. The interpolation produced incorrect results for limited sets of input parameters, in some cases yielding fraction > 1.

region

• Large parts of the CXC region library have been rewritten. The public programming interface (API) is largely unchanged. The filtering and area calculations are also the same with the following exception.
• The behavior of complex polygons, those with edges that overlap (think bow-tie) or duplicate points, has changed.
  • The area of complex polygons is now forced to be computed by pixelating the data-space.
  • The algorithm used to test whether a point is inside a polygon has also changed. This may lead to a difference in the filtering for complex polygons that cross back over themselves.

Environment

Off-the-Shelf (OTS) Package Versions

• The following OTS packages are included with CIAO 4.8. For more information on how the OTS packages are built for use with CIAO, refer to the INSTALL_SOURCE file distributed with the software.

  Executables

  • ds9 7.3.2

  Python

  • Python 2.7.10
    Frameworks on OSX have been disabled to avoid a problem with users picking up the wrong version of NumPy.
  • NumPy 1.9.2
  • ipython 3.0.0

  Libraries

  • cfitsio v3.370
  • readline 6.3
  • wcssubs 3.9.0
  • XSPEC v12.9.0d (models only)
  • VTK 5.10.1
  • CCfits 2.4
  • fontconfig 2.8.0
  • freetype 2.4.4
  • gsl 1.15
  • vte 0.25.1
  • fftw 3.3.4
  • xpa 2.1.15
  • jpeg 9a
• The ciaover command will now report the version of the Calibration Database as well as the version of the Analysis Scripts package. In addition the full path to the Python executable is shown (even when using the precompiled CIAO version).

Compiler versions

• The following versions of the gcc compiler were used to compile the binary CIAO releases:

  System	Build System	Version
  Linux64	CentOS 6	gcc 4.4.7
  osxml	Mountain Lion	gcc 4.2.1
  osxElCap	Mountain Lion	gcc 4.2.1

Documentation

Ahelp

• The XML ahelp files have moved from $ASCDS_INSTALL/doc/xml/ and $ASCDS_INSTALL/contrib/doc/xml/ to $ASCDS_INSTALL/share/doc/xml/ and $ASCDS_INSTALL/contrib/share/doc/xml/.

• Many pages have been updated to reflect the changes in this release. Several analysis threads are still under review.