CIAO 4.14 Release Notes
CIAO 4.14 is distributed for the following platforms:
- Linux 64 bit
- Apple macOS 10.14 (Mojave) through macOS 12 (Monterey)
Visit our platform support page for our statement about support for Apple's M1 (ARM) chip support used with macOS 11 (Big Sur) and 12 (Monterey). .
CIAO is available using the conda package manager: conda builds are available for Linux and Mac for Python 3.9, 3.8, and 3.7. More information can be found in the conda section of the Installation notes below.
There is no support for 32 bit operating systems, older Linux (CentOS 6 era) or older macOS platforms (macOSX High Sierra and earlier). More details can be found on the Platform Support page.
- Notable changes and improvements in CIAO 4.14
- How CALDB 4.10.2 Affects Your Analysis
- How CALDB 4.9.8 Affects Your Analysis
- How CALDB 4.9.7 Affects Your Analysis
- How CALDB 4.9.6 Affects Your Analysis
- Installation
- Tools
- Parameter Files
- Sherpa
- Graphical User Interfaces
- Analysis Scripts
- Python Modules
- Libraries
- Environment
- Documentation
Notable changes and improvements in CIAO 4.14
-
CIAO 4.14 includes several important bug fixes and includes updates to support newer compilers. There have been enhancements to sherpa and CIAO 4.14 also provides updates to Off The Shelf (OTS) packages like SAOImage DS9.
-
CIAO can be installed using the conda package manager. Users can install the conda edition with Python 3.9, 3.8, or 3.7 on Linux and Mac.
CIAO 4.14 conda install requires conda-forge channelUpdating to the latest version of XSpec requires CIAO to use a newer version of libgfortran, which is not available in the default channel.
Therefore users will need to include the conda-forge channel when they install CIAO 4.14. Because of this, users are encouraged to create a new conda environment for CIAO 4.14 rather than trying to upgrade an existing environment.
$ conda create -n ciao-4.14 -c https://cxc.cfa.harvard.edu/conda -c conda-forge \ ciao sherpa ds9 marx ciao-contrib caldb_main
The default version of Python is variable and will depend on which other packages are available at the time of installation. Users who want a specific version of python must include python=3.7, python=3.8, or python=3.9 on the conda create command.
Alternatively, the ciao-install script can be used to install CIAO 4.14 without conda, now with Python 3.8.
-
Several tools have been updated including important bug fixes:
- The ardlib default bad pixel mask, BPMASK, now includes the recently added Frame Store Shadow bits. This fixes the problem where the Effective Area is erroneously higher at the bottom edges of the CCDs. For details see the Caveats about ACIS Frame Store Shadow Bad Pixels page. This affects the following tools: mkarf, mkgarf, mkwarf, mkinstmap, and dither_region.
- The azimuth angle, phi, used to determine the vignetting factor to the effective area has been corrected. Depending on energy and off-axis angle, this could lead an error of up to several percent when using mkarf, mkgarf, and mkinstmap. It does not affect mkwarf. For additional details see the Caveats about Vignetting sign error bug page.
- The dmextract bug has been fixed which caused it to use the last GTI rather than the first when computing light curves. This is only a significant issue if the per-chip GTIs are significantly different.
- dmradar is an adaptive binning tool using a quad-tree algorithm similar to dmnautilus but in polar coordinates.
- dmdiff has been through a major overhaul to provide more robust comparisons -- especially with arrays and WCS parameters.
-
Notable Sherpa changes include:
- Update to XSPEC 12.12.0 which includes new models: xsgrbjet, xswdem, xsvwdem, xsvvwdem and xszxipab , and updates to parameter limits. New abundance tables, lpgp and lpgs, from Lodders et al.(2009) are now included. ETABLE model files can now be ready with load_xstable_model by setting the etable parameter. The default XSPEC chatter level has been increased from 0 to 10.
- Filtering and grouping area for binned (1D) spectral data has been improved with changes to the default behavior and many bug fixes resulting in changes to the statistics, degrees-of-freedom and energy flux in comparison to the previous version for the same data with the same filter.
- Update to sample_flux flux calculations including using the dataset id rather than using the default dataset, returning the statistic value for each iteration (even if rows are not used in flux distribution), corrections to the "clip" array, and it no longer excludes samples at the parameter soft bounds when calculating the flux distribution.
- Continued plotting improvements and optimization updates.
As a reminder, Sherpa is also available as a stand alone system, accessible from the Sherpa GitHub repository or from the standalone Sherpa page.
-
The CXC Datamodel has several updates mostly focuses on providing better support for scaled integer images, and in particular, the treatment of NULL valued pixels (ie the FITS BLANK keyword).
-
There have been several script updates since the CIAO 4.14 release including:
- specextract has been upgraded to support parallel processing, generally providing a factor of 2 or more improvement in runtime performance.
- The find_mono_energy script can be used to compute an appropriate monochromatic energy for an energy band given an ARF and RMF and a spectral model.
- merge_obs, reproject_obs, and flux_obs now create individual field-of-view (FOV) files along with a merged FOV file.
- dax now installs several common tasks into the new ds9 analysis button bar.
-
CIAO includes version 8.3 of SAOImage ds9. Users are reminded that they now need to change the Edit mode setting in order to select or create regions in recent versions of ds9. Please see the watchout page for more information on this.
SAOImageDS9 v8.3 highlights include:
- Option to display individual color bars for each frame.
- New note pad which can be saved separately and is included in backup
- Support for User defined Analysis buttons
Also, the CIAO ds9 wrapper script now preserves the users' LD_LIBRARY_PATH when it runs the ds9 executable.
-
The prism application has been removed from CIAO. Most of the prism functionality has been implemented in ds9. To launch prism from the command line :
$ ds9 -prism filename
or use the menu File → Prism option.
How CALDB 4.10.2 Affects Your Analysis
CALDB 4.10.2 Release Notes (release 15 November 2022)
ACIS Imaging and Grating Data
-
ACIS QE Contamination Model vN0015
The ACIS QE contamination model has been upgraded to N0015:
acisD1999-08-13contamN0015.fits
Recent calibration work with observations of previously-observed cosmic sources, such as clusters of galaxies, supernova, and blazars, has shown that the accumulation of contaminant on the optical blocking filter has defied systematic predictions.
This model updates the prior N0014 contamination model for both ACIS-S and ACIS-I, which showed an undercorrection fo the contamination effect in the most recent calibration observations. This update most readily affects observations for 2020 and beyond and should use the new CalDB to generate contamination-dependent data products.
More information about the N0015 contamination models, and prior versions, can be found in the ACIS QE contamination model Why document. A detailed presentation of the derivation of the new model—and the resulting changes in the estimated time-dependent effective area of ACIS-I and ACIS-S—is in the technical details section of the CALDB 4.10.2 Release Notes.
These CIAO response tools automatically apply the contamination file when creating ACIS response files:
As well as the scripts which use them:
How CALDB 4.9.8 Affects Your Analysis
CALDB 4.9.8 Release Notes (release 15 June 2022)
HRC Data
-
HRC-S Time-Dependent QEU Files
The HRC-S time-varying quantum efficiency uniformity files have been replaced with a new set, calculated based on unfiltered calibration observations. This latest set (N0016) shows very minor change over the prior set of QEU files, but incorporates results from the latest HZ43 calibration observations. The newest QEU for observations taken since 2021 June 10 includes updates to the outer plates' wavelength-dependent correction. The set of QEU files are unevenly distributed in effective dates and affects every HRC-S observation since the start of the mission.
These CIAO tools and scripts automatically apply the HRC-S QE and QEU files when creating response files:
-
HRC-I Gratings PIXLIB GEOM files
The geometry of HRC-I with gratings inserted has been improved in the PIXLIB, removing apparent tilts in the grating dispersion on the detector-plane.
How CALDB 4.9.7 Affects Your Analysis
CALDB 4.9.7 Release Notes (release 24 March 2022)
ACIS Imaging and Grating Data
-
Time-dependent ACIS Gain (T_GAIN) Files for -120 C Data
The new, combined time-dependent ACIS Gain (T_GAIN) file for August 2021-January 2022 (Epochs 87+88) are introduced. The combined Epochs 85+86 file has also been updated since CALDB 4.9.6. Therefore, the new T_GAIN affects observations taken since 2021 May 02. Other observations are unaffected by these new calibration files.
Users working with ACIS data taken since 2021 May 02 may wish to run chandra_repro and reprocess the data to improve the T_GAIN calibration. The DATE-OBS header keyword records the observation start date.
See the time-dependent ACIS T_GAIN why page for more information.
The gain corrections are at the nominal and expected levels of less than 1% of the energy value and users interested in CCD spectroscopy may benefit in applying the new gain adjustment. Grating spectroscopy benefits as well, in the form of improved order sorting. It is unnecessary to apply this adjustment for only doing timing or imaging analysis, although doing so will not have a negative effect.
Note that only spectra with several hundreds of counts and/or prominent features (in emission or absorption) will show changes from the T-gain refinement that exceed the uncertainties from the gain calibration.
How CALDB 4.9.6 Affects Your Analysis
CALDB 4.9.6 Release Notes (release 21 September 2021)
CALDB 4.9.6 is the latest CALDB release. There are no CALDB updates being released concurrently with CIAO 4.14.
ACIS Imaging and Grating Data
-
Time-dependent ACIS Gain (T_GAIN) Files for -120 C Data
The new, combined time-dependent ACIS Gain (T_GAIN) file for May-July 2021 (Epoch 86) and February-April 2021 (Epoch 85) are introduced. The combined Epochs 83+84 file has also been updated since CALDB 4.9.4. Therefore, the new T_GAIN affects observations taken since 2020 November 01. Other observations are unaffected by these new calibration files.
Users working with ACIS data taken since 2020 November 01 may wish to run chandra_repro and reprocess the data to improve the T_GAIN calibration. The DATE-OBS header keyword records the observation start date.
See the time-dependent ACIS T_GAIN why page for more information.
The gain corrections are at the nominal and expected levels of less than 1% of the energy value and users interested in CCD spectroscopy may benefit in applying the new gain adjustment. Grating spectroscopy benefits as well, in the form of improved order sorting. It is unnecessary to apply this adjustment for only doing timing or imaging analysis, although doing so will not have a negative effect.
Note that only spectra with several hundreds of counts and/or prominent features (in emission or absorption) will show changes from the T-gain refinement that exceed the uncertainties from the gain calibration.
HRC Data
-
HRC-S Time-Dependent Gain Maps and PI Region Filter Cut-off
The HRC-S gain correction has been unsatisfactory since early 2018 where certain assumptions in prior calibration procedures have proven faulty. Improved time-dependent gain calibration for HRC-S has since been developed.
Shortly after the release of the HRC-S gain maps introduced in CalDB 4.9.2, it was found that the HRC-S/LETG background filter was sometimes removing an excessive number of valid X-ray events. The filter is intended to be limited to remove 1% of eventts, but the excess losses were seen primarily at negative-orders for observations beginning in 2017 an increasing over time, with loses up to 6% at some wavelengths by late 2020. It was found that the event pulse height distributions were subtly distorted at very low gains and having large effects at higher PI channels— where background filtering primarily occurs—causing more real X-ray events be mis-identified as background events. Corrections have now been applied to the HRC-S TGAIN coefficients so that the 99th percentile PI values for X-ray events, rather than the medians, are constant.
Optimal background event filtering comes from an accurate time-dependent gain map applied using a corresponding PI region filter (TGPIMASK2).
The PI region background filter can be safely applied for all observations prior to the HRC-S high voltage change on 2021 May 14; however, it must not be applied to ObsIDs taken after that date. A modified chandra_repro from the updated contributed scripts package 4.13.3 will work automatically with CalDB 4.9.6 to handle the PI region filtering appropriately for all cases.
CIAO 4.12 users will need to reprocess manually to apply the new background filtering using the "LETG/HRC-S Grating Spectra" thread, replacing the letgD1999-07-22pireg_tgmap_N0001.fits region file to letgD1999-07-22pireg_tgmap_N0002.fits in the example. Again, the PI region background filtering step must be skipped for LETG/HRC-S ObsIDs taken after 2021-05-14T00:00:00.
-
HRC-S Time-Dependent QE and QEU Files
Regular calibration observations have shown that the HRC-S QE is declining at longer wavelengths (energies <200 eV, >60 Å), and decaying with the 2012-03-29 and 2021-05-14 high-voltage adjustments onboard. This spectral variation in the QE loss is accounted for using a series of time-displaced QE files, with the latest high-voltage adjustment introduced in the N0016 set in this CALDB.
The HRC-S time-varying quantum efficiency uniformity files have been replaced with a new set, calculated based on unfiltered calibration observations. This latest set (N0015) shows very minor change over the prior set of QEU files, but incorporates results from the latest HZ43 calibration observations. The newest QEU for observations taken since 2021 May 14 includes updates to the outer plates' wavelength-dependent correction. The set of QEU files are unevenly distributed in effective dates and affects every HRC-S observation since the start of the mission.
These CIAO tools and scripts automatically apply the HRC-S QE and QEU files when creating response files:
-
HRC-I Time-Dependent QE Files
The HRC-I quantum efficiency QE has been decaying for some time, particularly since the year 2011, when a 5% to 10% per year reduction had been measured. To address this issue, the HRC-I Calibration Team had introduced a time-dependence into the QE files in 2018 which are now extended in time to the latest epoch of observations. Included in this update is a new file coinciding to the recent HRC-I high-voltage adjustment affecting observations after 2021 February 16. The QE update affects the results from mkinstmap and fluximage and is applicable to all previous epochs of HRC-I observations.
Installation
Users should be aware of these installation items before installing CIAO 4.14. Additional problems which are seen less frequently are listed on the Installation & Smoke Tests bug page.
Supported Platforms
-
CIAO 4.14 is supported on:
- Linux 64 bit
- Apple macOS 10.14 (Mojave) through macOS 12 (Monterey)
Installing with the conda package manager
-
CIAO can be installed using the conda package and environment management system.
CIAO 4.14 conda install requires conda-forge channelUpdating to the latest version of XSpec requires CIAO to use a newer version of libgfortran, which is not available in the default channel.
Therefore users will need to include the conda-forge channel when they install CIAO 4.14. Because of this, users are encouraged to create a new conda environment for CIAO 4.14 rather than trying to upgrade an existing environment.
The conda release is available from the custom CXC channel.
$ conda create -n ciao-4.14 -c https://cxc.cfa.harvard.edu/conda -c conda-forge \ ciao sherpa ds9 marx ciao-contrib caldb_main
See the conda installation instructions for more information, including known problems and incompatibilities.
With the conda edition users can choose to install CIAO with Python 3.9, 3.8, or 3.7. Other OTS such as numpy and matplotlib will also be be different based on the python version.
-
CIAO will now use the TMP_DIR environment variable, if it is set, as the value for ASCDS_WORK_PATH, which is the directory used for temporary files.
-
The ciaover command now includes the caldb_main package (if it was installed).
Updates to the ciao-install Installation Script
- Changes to prevent CIAO from being installed on unsupported platforms (CentOS6 and earlier as well as macOSX High Sierra and earlier).
- The ciao-install script now includes a line at the end of the script providing the final installation location. This makes it easier to create your ciao alias.
Source builds
- With the removal of prism, the last GTK application, it is now much easier to build CIAO from source. In addition, the configure script has been updated to no longer require a FORTRAN compiler (a legacy requirement). Users can visit the Rebuilding CIAO 4.15.2 source with pre-built OTS binaries on Linux or Rebuilding CIAO 4.15.2 source with pre-built OTS binaries on mac for instructions on how to build CIAO from source.
- CIAO is now compatible with the GNU++14 C++ standard which is the default for newer compilers.
Smoke Tests
-
Updates have been made to the ipython2 smoke test to properly skip if if no DISPLAY is set and to omit the information about which matplotlib backend is in use.
IPython Settings
-
Users will be prompted to update their IPython profiles the first time they run sherpa
Remove old parameter files
-
With every new CIAO release, some parameter files are changed: new parameters may be added and occasionally old ones removed or renamed. Deleting or renaming the local parameter directory ensures that the correct parameter files will be accessed the first time a tool is run:
unix% rm ~/cxcds_param4/*
Tools
acis_process_events
- Improved handling of the new Frame Store Shadow bad pixel bits in continuous clocking mode.
addresp
- Corrects the values of the NUMGRP and NUMELT keywords in the RMF header.
asphist
- Fix issue when the ASPTYPE keyword was missing from the input aspect solution file.
calquiz
- The default for the calfile is now calfile=CALDB.
dmcontour
- Copies the generalized world coordinate system information into the output file.
dmdiff
- dmdiff has undergone a major overhaul. The output format between the different sections has been made more consistent. There have been improvements when comparing WCS and arrays along with additional improvements handling NaN, NULL, and Infs.
dmellipse
- Copies the generalized world coordinate system information into the output file.
dmextract
- Fixes bug where the last GTI was being used instead of the 1st GTI.
dmgti
- Portability update to ensure that gamma is the "true" gamma rather than log-gamma. Change made in the mtllibs.
dmimgcalc
- Portability update to ensure that gamma is the "true" gamma rather than log-gamma. Change made in the mtllibs.
dmimgfilt
- Fix region parsing if region file contains the string: "mask".
dmmakereg
- Fix region parsing if region file contains the string: "mask".
dmnautilus
- Updated dmnautilus to have a mode where the subdivision is only done if all the quadrants have SNR above the threshold. Added a new method parameter. When method=0 the original algorithm is used and the current level is split into 2x2 if its SNR is above threshold. The new algorithm is triggered when method=1|2|3|4. The value indicates the number of the sub 2x2 arrays that must have SNR above threshold to trigger the split. With method=2, the two must be side-by-side, not diagonal. With method=4, all the sub-images are required to be above the SNR threshold. If any of the sub-image is fully outside the subspace or only contains NULL/NaN values, then that sub-image is treated as if the SNR is > threshold. There is no adjustment for sub-images partially outside the subspace|NULL|NaNs.
dmpaste
- Internal cleanup; unpredictable behavior when allocating a 0 length array.
dmradar
- This new adaptive binning tool is the polar equivalent to the dmnautilus tool. Rather than divide the image in X and Y, it divides the image in Radius and Angle. Users can select circular, elliptical, or box shaped regions.
dmsort
- Corrects issue with secondary sort, when primary sort column is string type.
dmtcalc
- Portability update to ensure that gamma is the "true" gamma rather than log-gamma. Change made in the mtllibs.
hrc_process_events
- Corrects the output SAMP values. Internally the values were correct; however, the values in the output file were incorrect.
mkarf
- The new Frame Store Shadow bad pixels located at the bottom of each CCD are now automatically included in the responses. (Change made to ardlib.)
- Corrects a bug when computing the azimuthal angle used to look up the vignetting correction. For mkarf, which should be used for nearly on-axis source sources, the error is generally very small (generally less than 1%). (Change is made in libemap.)
mkgarf
- The new Frame Store Shadow bad pixels located at the bottom of each CCD are now automatically included in the responses. (Change made to ardlib.)
- Corrects a bug when computing the azimuthal angle used to look up the vignetting correction. Since most grating sources are located near the aim point, the vignetting correction is small (generally less than 1%). (Change is made in libemap.)
mkinstmap
- The new Frame Store Shadow bad pixels located at the bottom of each CCD are now automatically included in the responses. (Change made to ardlib.)
- Corrects a bug when computing the azimuthal angle used to look up the vignetting correction. The vignetting correction varies with off-axis angle and energy. The error is generally between 0 and 2%. (Change is made in libemap.)
mkwarf
- The new Frame Store Shadow bad pixels located at the bottom of each CCD are now automatically included in the responses. (Change made to ardlib.)
modelflux
- Update modelflux to link sherpa's xschatter level to the modelflux verbosity parameter. A verbose value of 1 equates to xschatter 10 and a verbose value of 5 equates to xschatter 25.
pfold
- Fixes bug where the last GTI was being used instead of the 1st GTI.
skyfov
- Improves handling when there are multiple CCD_IDs associated with the same GTI.
tg_findzo
- Updates to support numpy 1.20.
wcs_match
- Corrected an index error in wcs_match which can sometimes cause the residual ratio calculation to be in error. This is only used when the residfac param is greater than 0, and could result in some better suited source pairs being dropped from the transform calculation earlier than other, worse suited pairs.
- Added a new parameter, multimatch, that when set to "yes", allows a source that matches multiple sources in the reference list to contribute to the solution. The previous/default behavior, "no", excludes sources which have multiple matches.
Parameter Files
This section provides a summary of the changes to parameter files in this release. Refer to the Tools and Analysis Scripts sections of these release notes for complete details.
We recommend deleting all the old parameter files or renaming the parameter-file directory before running any new version of CIAO to avoid conflicts. More information is available in the FAQ.
calquiz
- The default is now calfile=CALDB
dmnautilus
- There is a new method parameter which controls algorithm used to decide when to subdivide a region. method=0 (default) is the original algorithm where the subdivision happens until the SNR in the regions all fall below the threshold. method=1|2|3|4 requires at least one|two|three|four of the subdivsions to have SNR above the threshold.
wcs_match
- There is a new multimatch parameter which when set multimatch=yes, will allow a source that has multiple matches in either the input source list or in the reference source list to contribute to the final coordinate solution. The default, previous behavior, multimatch=no requires that only unique source matches between the input and reference source list be considered.
Sherpa
Sherpa is developed as a community project, and is available on GitHub. Comments, fixes, and additions from the community are welcomed. There are additional releases for this version throughout the year, and can be installed either directly from GitHub or using the conda package manager, as described on the standalone Sherpa documentation page.
changes
-
- XSPEC
-
Update to version 12.12.0 of the XSPEC model library, from version 12.10.1s in CIAO 4.13. Support for the new models: additive xsgrbjet, xswdem, xsvwdem, xsvvwdem and multiplicative xszxipab
There are changes to the hard limits of some models, the default parameter values and units, including adding and removing units. Also, XSPEC model parameters now use the same limits for soft and hard (the hard limit from the model.dat file) and the hard limits can be changed for some models which support a value outside the normal parameter range. It is strongly suggested that any parameter set to a value outside of the original limits is also frozen.
New abundance tables (lpgp, lpgs) from Lodders et al, (2009) are added and can be selected with set_xsabund
XSPEC ETABLE table model files can be read by load_xstable_model by setting the etable parameter.
Change the default XSPEC chatter level from 0 to 10. This may result in screen output the first time an XSPEC model is evaluated. set_xschatter can be used to get back to the previous behavior.
- Filtering and grouping
-
Filtering and grouping area for binned (1D) spectral data has been improved with changes to the default behavior and many bug fixes resulting in changes to the statistics, degrees-of-freedom and energy flux in comparison to the previous version for the same data with the same filter. Highlights include:
Energy and wavelength filters (with notice and ignore) for PHA data are now treated as lo <= x < hi in all cases, and channel filters are lo <= x <= hi (where the channel values must be integers). For Data1DInt cases the notice and ignore filters are treated as lo <= x < hi. This can result in slightly different fit results because the data used in the fit can be changed and hence changing the number of degrees of freedom (by one or two depending on whether the first and last bins have been removed).
Filter expressions for Data1DInt and DataPHA cases now report the start and end value of each set of grouped data rather than use the end points. For PHA data sets users now see the same ranges displayed whether the data is grouped or not. The actual filter remains the same even if the filter expressions has changed slightly.
Update the image filters: "|" is used now instead of & to combined the regions. This affects the output of get_filter as input to calc_data_sum2d and definitions of very-complex spatial filters. Restoring sherpa sessions could be impacted.
There are also many bug fixes in this area.
- Plotting improvements
-
CIAO 4.14 continues the improvements to the plotting functionality, and includes the following:
- plot_model_component and get_model_component add the response for PHA data sets, if there is one. Model expressions which contain a response are not changed.
- Corrected the order of the points, x, and y values when displaying a CDFPlot object, improve support for a list argument.
- Bug fixes to plot_source, reg_proj, reg_unc and multi-plots.
- Internal changes to the plotting code and documentation.
- Flux calculations
-
There are changes to sample_flux which now:
- uses the id argument rather than always using the data from the default dataset.
- returns a statistic value for each iteration, even if those rows are not used in the reported flux distribution.
- returns correct information for the clip column (that is, it matches the clipping done by this routine). There may be changes to the reported error ranges because of this change.
- no-longer excludes samples at the parameter bounds (soft) when calculating the flux distribution. This could lead to an over-estimation of the flux for upper limits.
The How do we calculate a flux in Sherpa Jupyter notebook has been updated.
- Optimization methods
-
Update on how the optimization minim will behave if the free parameter is beyond the limit. The default parameter reflect is True (will reflect by an equal amount about the limit). If reflect is set to False then model function will return DBL_MAX (~1e308) and therefore it will not be included in the simplex.
Updates to the internal documentation of the methods.
- Documentation
-
The Sherpa ahelp files have been updated to match the Python docstrings. Each command now has its own ahelp file, rather than combining multiple commands into a single file.
Sherpa in CIAO 4.14 incorporates the improvements and fixes made in stand-alone sherpa releases 4.13.1 and 4.14.0. Users can review the details of those releases on the Sherpa github pages.
Graphical User Interfaces
The prism application has been removed from CIAO. Most of the prism functionality has been implemented in ds9. To launch prism from the command line:
$ ds9 -prism filename
or using the menu File → Prism option.
Analysis Scripts
There have been several updates to the contributed scripts package since CIAO 4.13 was released. These changes are specific to the CIAO 4.14.0 contributed scripts release.
Colorized Errors
- When a script errors out the message will now be displayed in red unless the NO_COLOR environment variable is set - see the site at https://no-color.org/ or the output is being written to a file rather than a TTY.
dax
- support for the new button bar feature in DS9 8.3
- the default absorption model in spectral fits is now phabs rather than wabs and a second absorbing component can be added
- a new menu - Adaptive Bin (Radial) - to call the new dmradar tool
- add the method argument to the 'Adaptive Bin' menu to support new functionality in dmnautilus
-
For dax tasks that output tables or regions, users can now
automatically open those output files using ds9's prism interface.
To enable this options, users need to run
before starting ds9.
$ pset dax prism=yes
find_mono_energy
- Calculates an "effective" mono-chromatic energy for a given spectral model, and set of parameters, given a Chandra ARF and RMF. This can be used to see how the ACIS contamination layer changes the effective response over time, or to calculate the best mono-chromatic energy when calculating exposure maps with scripts like fluximage and merge_obs.
blanksky_sample
- The dead-time correction is now used to better sample the time values created for the events in the blank-sky file.
chandra_repro
- The reprocessed event file now has a CONTENT keyword set to "EVT2" and HDUCLAS2 keyword set to "ACCEPTED".
check_ciao_version
- The script now reports the 'conda install' command that is needed when an update is needed.
convert_xspec_script
- The script now handles XCM scripts containing XSPEC table models (atable, mtable, and etable).
convert_xspec_user_model
- The script has been updated to support changes in XSPEC 12.12.0 and Sherpa. However it is not guaranteed to work for the same models as were supported in CIAO 4.13: please contact the CXC helpdesk if you are having problems with this script.
download_obsid_caldb
- The script now sets the CALDBALIAS which is needed due to a change in the caldb4 Python module in CIAO 4.14.
fluximage
- The script now creates a FOV file for the observation.
flux_obs
- The pixel size calculation used when the maxsize parameter is set has been changed to avoid problems when also setting the psfecf parameter. There have been improvements when combining a large number of observations. The FOV file is now created for each reprojected event file and a combined version.
merge_obs
- The pixel size calculation used when the maxsize parameter is set has been changed to avoid problems when also setting the psfecf parameter. There have been improvements when combining a large number of observations. The FOV file is now created for each reprojected event file and a combined version.
reproject_obs
- The script now generates FOV files for each reprojected event file and a combined file (that includes all the individual shapes).
srcflux
- Fixed a rare error when merging results. If a source was at the edge of the field of view of a CCD it could appear as if it belonged to an inactive CCD, thanks to dither, which caused the script to crash.
specextract
- The tool now runs tasks in parallel to reduce run-time for users with multi-core machines. This can be controlled with the new parallel and nproc parameters. As part of this the script should provide better diagnostic messages when there is a problem.
ciao_contrib.runtool
-
Parameter values are best handled using the natural Python type, so for boolean parameters this means using True or False, but support has been enhanced for the following values: for True, 1, "1", "true", "yes", "on" and for False, 0, "0", "false", "no", "off".
This may change the behavior of existing code, since it changes how boolean parameters are handled (e.g. setting a boolean value to "false" will now set it to False when it previously would have set it True.
- The module has been updated to reflect new tools (dmradar) and parameter settings in CIAO 4.14.
sherpa_contrib.utils
- The sherpa_contrib.utils.renorm routine has seen several minor fixes.
Python Modules
pycaldb
- Improvements to the error reporting to raise Python exceptions rather than printing error messages and setting an error status code.
pycrates
- Catch the case when trying to create a column name containing "(". A warning is raised with the actual column name created.
- Preserve the WCS transform for empty images.
- Corrects the handling of BLANK keyword; propagates it to the output for integer images.
pypixlib
- Added additional routines needed to compute grating coordinates: fpc2fc, fc2fpc, fc2gzo, gzo2fc, gzo2gac, & gac2gzo.
- Fixed additional grating related coord transform routines: mnc2fpc & gdp2gac
Libraries
caldb4
- Internal cleanup to support newer compiler standards.
datamodel
-
There have been several updates dealing with NULL pixels (FITS BLANK keyword) in images, especially scaled integer images (eg those with BSCALE and/or BZERO keywords).
- ensure that the BLANK keyword is copied to the output when image is binned.
- translate NULL pixels to IEEE NaNs when casting to a real, float-point value.
- corrects issues with binning NULL pixel values: output is now NULL instead of using the raw, ie BLANK, value.
- ensure that BLANK values are treated equally in DM's ASCII "DTF" format.
- Improvements in the DM to recognize spectral WCS coordinates, eg VELO
- The DataModel was updated to correctly parse a parenthesized list of comma-separated column names when used in '[bin (col1,col2)=min:max:step]' syntax and '[cols vectorcol(col1,col2),col3]' token.
grplib
- Internal work to resolve rpath issues when building on macOS.
regionlib
- Remove the obsolete "Zero length polygon line segment" warning.
- Corrects the PIE shape area and extent calculations when the angles are outside the 0:360 degree range.
Environment
Off-the-Shelf (OTS) Package Versions
-
The following OTS packages are included with CIAO 4.14 installed with ciao-install. For more information on how the OTS packages are built for use with CIAO, refer to the INSTALL_SOURCE file distributed with the software.
With the removal of prism, CIAO 4.14 does not depend on GTK. This greatly reduces the number of OTS packages CIAO requires.
Differences in OTS for conda installationThe conda-installed version of CIAO 4.14 has different OTS versions, and it also depends on the Python version you have chosen to use.
Executables
- ds9 8.3
Python
- Python 3.8.8
Frameworks on OSX have been disabled to avoid a problem with users picking up the wrong version of NumPy. - NumPy 1.20.3
- ipython 7.19/0
- six 1.15.0
- pip 20.3.3
- cython 0.29.21
- Matplotlib 3.3.3
- jupyter notebook 6.3.0
- pytest 6.2.1
Libraries
- cfitsio 4.0.0
- CCfits 2.6
- wcslibs 7.3.1
- wcssubs 3.9.5
- readline 7.0
- XSPEC v12.12.0 (models only)
- fftw 3.3.9
- gsl 2.5
- xerces 3.2.3
- jpeg 9.3.0
- tcl 8.6.10
- tk 8.6.11
- xpa 2.1.20
Compiler versions
-
The following versions of the gcc compiler were used to compile the binary CIAO releases installed with ciao-install. The conda-installed compiler versions are different.
System Build System Version Linux CentOS 7 gcc 7.4 macOSX Mojave llvm 10.0.0 -
Several CIAO applications have been updated to support the GNU++14 standard.