CIAO 2.2 Release Notes
- User Interface (UI)
acis_detect_afterglow . Bug fix for not checking ccdid, could have tagged a few extra events as 'glowing' that were not. acis_fef_lookup ** NEW ** . Back in the baseline CIAO distribution. . Determines the FEF file and filter needed for the source position. acis_process_events . New algorithm to compute sky positions for CC mode events. This works to remove the dither in the CHIP Y direction. . should stop warning about finding two calibration files for the grade file. . New algorithm to check for pixel values in outer 5x5 island (very faint mode only) greater than split threshold. Will set status bit 23. AKA "very faint" checking and/or "part55". . Includes 2 new parameters for very faint checking: check_vf_pha to determine whether the check is done or not and "trail" which is an ad-hoc adjustment factor for CTI effects. . Fix to deal with bad input obs.par file . Added new parameter to control PHA randomization . Fixed bug in TE Graded mode where badpixel could have been missed. (not an issue for SDP, only user supplied badpixels isolted to a single pixel rather than 3x3 island) . worked history so that duplicate history records aren't written out. . Bug fix to not override TIMPIXR if already exists when using an obs.par file acisreadcorr ** NEW ** . Tool to identify the 'streak' of events in the chip read-out direction that coincides with a bright source. These are events that are detected during the frame shift/read-out. . Sets times for those events so some rudimentary time analysis can be done. aconvolve . fixed incomplete error message acrosscorr . no significant changes add_grating_orders ** NEW ** . Previously part of the optional download, this script is now packaged as part of the CIAO distribution. . Adds together spectra from different grating orders. add_grating_spectra ** NEW ** . Previously part of the optional download, this script is now packaged as part of the CIAO distribution. . Adds grating spectra (eg MEG and HEG) apowerspectrum . no significant changes asp_apply_sim ** deprecated ** . No longer necessary with the new 'asphist' . May not be shipped in later CIAO releases. asp_calc_offsets ** deprecated ** . No longer necessary with the new 'asphist' . May not be shipped in later CIAO releases. asphist . New asphist that works directly from the aspect solution files (asol). . Will continue to work with the output of asp_apply_sim (backwards compatible) . fixed clobbering problems for corrupted outfile axbary . This was a fundamental but minor bug that led to errors of the order of less than 1 us in the barycenter correction. . There was a error condition related with keeping track of the position in the orbit ephemeris file that was not properly taken care of. . Apply TIMEZERO: a bug that affected XTE corrections . Output writable celldetect . Added source cell locations and double detection flags in the celldetect output. New output columns CELL_X, CELL_Y, DOUBLE, and DOUBLE_ID. . Sources which are on blocking boundaries and are detected in two successive blocking passes are flagged in DOUBLE and DOUBLE_ID . checks for successful creation of the logfile . Fixed src centers for even size cells . Fixed problem with clobbering files . Enhanced duplicate suppression based on the .5 pixel centers for even sized cells. Now it can also reject duplicate sources that are slightly further apart. User's may see their source lists shrink slightly. . Upgraded default parameter file for the latest psfsize table . Fixed problem with src ellipse axis lengths when data are blocked . The region output files have the revised ascii region format acceptable to both ds9 and the DM. csmooth . Fixes clobber problem. Also increases length of filename arrays. . fixed bug for input scale map when only 1 pixel used a certain scale. . fixed incomplete history . Fixed waiting until end to test whether to clobber outputs destreak . Updated so clobber messages doesn't appear when verbose=0 dmappend . Fixed dmappend not returning appropriate exit value dmarfadd . no significant changes dmcontour . Fix some long standing issues when the contours included regions that should really be excluded. Regions will now properly be included/excluded. There where problems in the past when regions where inside each other, the logic wasn't correct. . Fixed problem when no contours are found dmcoords . fixed bad behavior when bad inputs given for celfmt dmcopy . Updated so clobber messages doesn't appear when verbose=0 dmextract . The POISSERR keyword is properly set in the counts case to reflect when the errors are Poisson or not. . proper error case when the RA/Dec cannot be determined. . Updates to deal with doubles and floats in images . Updates to catch DBL_EPSILON problems on linux . Now allow the use of Poisson or Gaussian for the pha/histogram case. . Added a warning if the user tries to use a variance file . Reset the defaults for the error values . The exposure maps were not correctly accounted for. Dmextract was weighting the images by the binning factor. . There was a problem in the way values were being indexed in image source extraction. . Fixes problem w/ normalization of exposure maps . Added reading units from exposure maps to normalize counts correctly . Fixed PHA_MIN/PHA_MAX bug . Support for BKG subtraction in PHA/histogram space . We are going to be multiplying the (cm^2.s) maps by the livetime in the file. If the user inputs a exp file, but not a bkg_exp file or vice-versa, We will ignore the exp values, and calculate without them and print out an error message. dmfilth ** NEW ** . "FIL in The Hole"; used to create background maps by excluding source regions from the image. . Various method s are presented; some are good for 'counts' images (eg Poisson), some are good for smoothed images, eg bilinear interpolation. dmgroup . Fixes case where writes out grouping keyword & column dmgti . fixed bug when using string keywords (truncating the string by 1 character). dmhedit . Added checks against bad input list files . Made operation, key, etc automatic so user will be queried (if not filelist is present, eg filelist="") . Fixed problem when given a file list with nothing in it dmimg2jpg . Cleaned up the color lookup tables. There were extra entries at the end of the files that were not part of the original color schemes . changed region support to use region lib parsing for more general use. . fixed minor annoyance in clobber output dmimgcalc . Added support for infile2="none" to allow simple scalar multiplication of images . Get rid of aim point tests for WCS matching. dmimghist . Added column for normalized counts . pixels above and below boundary are not counted in end bins. This is controlled thru new "strict" parameter. dmimgthresh ** NEW ** . Sets all values in an image above/below a certain limit to a new value. dmjoin ** NEW ** . Interpolates columns from one table onto another by joining the two files on a common column. dmkeypar . Fixed problem on Alpha where string-length was truncated. dmlist . Will report "Null" values . Increase precision of output for doubles/floats. . Fixed problem dmlisting 1x1 array dmmakepar . no significant changes dmmakereg . fixed clobbering problem . better error handling when bad input is supplied dmmerge . fixed clobbering problems for corrupted outfile dmpaste . Fix problems where error messages were causing segv's. . Added warnings when columns being pasted from 'pastefile' exist in 'infile'; 'pastefile' columns are skipped. . Fixed broken error message when no. rows in files not the same. dmreadpar . no significant changes dmrega2fits ** REMOVED ** . Removed from CIAO. Use dmmakereg to create ASC-FITS-Region style region files. Use tg_create_mask to create grating region files. dmregrid . no significant changes dmsort . changes, but some are untested. Should continue to only use in defined CIAO threads. dmstat . fixed the incorrect sum problem when centroid=yes. . Added checking for NaN's and TNULL's in columns . Added support for NaNs in floating point images. and NULL values in integer images. . Changed output format to use "%g" rather than "%f" to get a fixed precision. . Removed code that did coord transforms automatically. Users can now use the datamodel to specify columns with WCS, eg dmstat infile.fits[cols ra,dec] . Will now report centroid positions in physical coords in addition to logical coords. . Will now report the location (in physical coords) of the min . only the first instance of the min/max are reported. . Reports location of min/max regardless of centroid being computed. . WARNING: the output format has changed slightly, which may break scripts that use dmstat dmtcalc . Fixed divide by zero SIGFPE. . Fixed scientific notation so that you can enter "3E12" rather than needing to enter "3.E12" . Fixed problem with column smoothing and keeping original value unless explicitly over-written, so foo=goo:3 will have the original values of "goo" in the outfile, but goo=goo:3 will have the smoothed values. . Now propagates the WCS attached to various columns to the output. . Should also now be setting TLMIN/TLMAX's and also propagating the DSS (GTIs) properly. . fixed bug when using string keywords (truncating the string by 1 character). dmtype2split . no significant changes dmwritefef ** REMOVED ** . No longer packaged with CIAO. get_src_region . no significant changes hrc_process_events . fixed bug that was allowing PI values above 255 . Fix to deal with bad input obs.par file . worked history so that duplicate history records aren't written out. lightcurve . no significant changes mkarf . Ardlib options permitted on the mirror parameter (e.g., AREA=1). . The name of the current working directory is written to the history keyword to give meaning to relative path names. mkexpmap . The name of the current working directory is written to the history keyword to give meaning to relative path names. mkgarf . CALDB support for the OSIP file. . IRMXFIL header renamed to OSIPFILE . The engrid parameter now permits "compute" to be used to indicate that a default grid is to be used that is consistent with the resolution of the instrument. . The name of the current working directory is written to the history keyword to give meaning to relative path names. mkgrmf ** NEW ** . This program computes a grating RMF. mkinstmap . BUNIT keyword added to the output file. . The name of the current working directory is written to the history keyword to give meaning to relative path names. mkpsf . If user requests a position outside the current psf library file, the tool will return the closest match (and issue a warning). Previously, this was a fatal error. . Correct bug which sometimes identified two interpolation data columns when no interpolation was needed. . Allow alternate filepaths with autonaming for psflib data output files, check for pre-existing files and add clobber param for psflib files. . ability to extract individual psf images in addition to interpolated one. . Added additional output header keywords so that detect tools can run on psf's that are output . Added naming options for output psflib data files, . fixed bin/rotate bug for interpolated image . add ROLL_NOM keyword to output files . remove user option to specify roll angle. . fix for not write out RA/DEC in header when in DET coords . Added CALDB automatic lookup to pick the "best" PSF file ONLY when an input event file is provided. . Fixed bug which caused an error in the output interpolated PSF image when the PSF SIZE parameter was chosen to exactly match the PSF image size after rebinning. . Added bug fix for incorrect energy hdr keyword in output psflib data files when selected energy was out of range. mkrmf . Includes new "weights" parameter. When set, a weighted RMF based on the weights of the FEF regions will be created. . When "weightfile" is set, any filter appearing in 'infile' expression will be ignored . Allows user to input abbreviated "axis1=energy" when weights file is used since it uses energy grid from weights file . Still backwards compatible with the existing single FEF inputs. . Fix for SEGV when logfile name was missing or "" . Clobber checked at the beginning of run . Added CALDB lookup when using a weight file (ONLY when using a weight file). . Speed enhancements . add message to stdout to inform multi-region precessing status . add CHECKSUM/DATASUM/CREATOR keywords . remove spatial filter from FEFFILE keyword . quite unnecessary warning message . fix for incomplete error msg when bad FEF region is requested mkwarf ** NEW ** . new tool to create a weighted ARF file based on an input WMAP. . second (necessary) output is a set of weights needed by mkrmf to create the corresponding weighted RMF. . Users must supply "[wmap]" on input file name if using a wmap from dmextract. mtl_build_gti . fixed bug when using string keywords (truncating the string by 1 character). paccess . no significant changes pdump . no significant changes pget . no significant changes pline . no significant changes plist . no significant changes pquery . no significant changes psextract ** NEW ** . Was originally on the optional download list, this is now part of the basic CIAO installation. . Extracts the spectra and computes the arf and rmf for the given source region. pset . Now internally sets tool name. The upshot being that you can get help on the tool you are psetting parameter for. psf_project_ray ** NEW ** . Post CIAO2.2, the CXC will allow users to generate custom ray files thru the CXC's mirror model, "SAO-SAC". This will be a web interface. . psf_project_ray will take these rays and project them onto the detector configuration (including non-standard SIM locations) to produce a PSF at the exact desired location. punlearn . no significant changes reproject_events . New functionality to set a random time w/in the GTI of the match file. This allows the "background" datasets to be used w/in CIAO . New parameter to control time/position randomization. . various error checking on bad inputs rmfimg ** NEW ** . Creates a 2D image from an input RMF file. Can be displayed in imager and includes WCS. . Will optionally multiply by the ARF. sso_freeze . added check to verify at least 2 sso ephem records exist prior to first event. stk_build . no significant changes stk_count . no significant changes stk_read_num . no significant changes tg_create_mask . Fixes problem with HETG mask in CC mode. . added hetg and letg width factor parameters. width_factor defaults to redirect of hetg. . Removed HESF parameters . Added WPSF keyword to properly catch what CALDB resolved to at run-time . made output more consistent with CXC region format so it can be used as a dm filter. . Added history information about infile, outfile, and calibration files used. tg_resolve_events . Fix bug where 0th order photons were not getting assigned dispersion coordinates. . Fixed problem with CC mode data not being de-dithered if the aspect solution files are used rather than the aspect offsets. . Fixed ACIS CC+LETG case . Changed ACISDE keyword to OSIPFILE to properly catch which CALDB file was used. . Now gives out warnings only if GRATING keyword is HETG but the user enters LETG to the grating_obs parameter or vice versa . Removed HSEF code to clean things up. . made changes so it can now handle the cases where the center of the zero-order source falls in a chip gap . changed the chip_minrange & chip_maxrange variable from short to double. Also changed the logic to check if an event's coordinates lie within the valid range. The changes should make the tool resolve events that are very close to the edge of the detector. . fixed minor un-noticed bug that could have effected users iff they had tried to force the LETG grating type. . The tool can now sort order (for acis) by using 2 new parameter values (osort_lo, osort_hi) from the par file. This order-sorting method will be used only if the osipfile (used to be the rmfile parameter) parameter is set to "NONE". . Changes in parameter interface: 1)rename the rmfile parameter to osipfile 2)add osort_lo and osort_hi parameter tgdetect . Fixed problem when no. counts per pixel greater than 32K tgextract . made tg_lam,tg_d into a vector column to support region filtering. . Fixed problem with entering "+" before number . added MTYPE/MFORM keywords to make output regions be valid FITS region files. . Added INREGION keyword to properly catch what CALDB resolved to at run-time . deleted the tg_d_units parameter from tgextract.par it can only handle tg_d in degrees, so it's meaningless to have tg_d_units in the par file tgidselectsrc . no significant changes tgmatchsrc . no significant changes vtpdetect . The region output files have the revised ascii region format acceptable to both ds9 and the DM. wavdetect . The region output files have the revised ascii region format acceptable to both ds9 and the DM. . fixed output file autonaming extensions. . Fixed problem with src ellipse axis lengths when data are blocked wrecon . Fixed autonaming problem . Fixed problem with src ellipse parameters when data are blocked . Fixed problem with src ellipse axis lengths when data are blocked . The region output files have the revised ascii region format acceptable to both ds9 and the DM. wtransform . Fixed segv when exposure map size does not match input image size
New (as opposed to enhanced) functionality: . EQWIDTH: [command] computes the line equivalent width. . FTEST: [command] computes the F-test model comparison test significance. . MLR: [command] computes the Maximum Likelihood Ratio model comparison test significance. . UNLINK: [command] undoes previous linkage of a specified parameter. . GRIDMODEL: [model] allows one to use a numerical model in Sherpa fits; exposure maps may be specified via GRIDMODEL (see also EXPMAP). . sherpa_eval() has been added to allow execution of Sherpa commands in S-lang scripts. . One may read in systematic errors in addition to statistic errors. Changes to I/O: . For PHA files, filtering and fitting can be done in either energy *or* wavelength space, if an instrument is specified. . DATASPACE now includes HISTOGRAM (binning) option. . One may specify additional filters after issuing the command READ FILTER. . One may specify additional filters after issuing the command IGNORE BAD. . New functions (e.g., sherpa_update_mdl()) have been written to create MDLs and store them in S-Lang variables. Changes to General Fitting: . Generalized background: source and background datasets no longer need to be of the same size. . Background data may be fit alone, with no source data specified. . Background datasets may have their own instrument models. . In image analysis, one may specify that the unfiltered background dataset is the same as the source dataset using the command BACK => DATA. . PSFFROMFILE: [model] is now faster for 2D convolution and has an improved interface for 1D convolution. . PILEUP model stacks may now be specified. . XSPEC models may be used even when analysis is performed in wavelength space. Changes to Sherpa Models: . New model: JDPILEUP, a pileup model. . New model: NBETA1D, a normalized beta model for line fitting. . Sherpa user models can now be written as S-Lang scripts. . More scripts for doing high resolution spectroscopy (GUIDE). . One can apply XSPEC v11.1 models. Changes to Visualization: . Visualization of RESIDUALS now shows (data - model); DELCHI shows (data - model)/error. . More background visualization options: BFIT, BRESIDUALS, etc. . IMAGE FIT now tiles images in ds9 by default. . Contour and surface plots updated to handle arbitrarily filtered data; also, multiple surface, contour plots can be displayed at once. . New OPLOT command to overplot several curves in one drawing area. . WRITE has been updated to provided energies or wavelengths of bins; and filtered or unfiltered data. . Filtered statistics images can now be visualized. . Use of new store/restore with chips; the plotted data is stored with the Chips FITS file.
User Interface (UI)
ChiPS ----- - introduced chips state object, which signficantly improves ease and flexibility of chips customization - will read .chipsrc file at startup, if found - more streamlined/stable use of S-Lang - can now be used outside of CIAO, via import("chips") in any application that embeds S-Lang (like slsh, or ISIS, etc) - chips_eval() function introduced for executing chips commands within S-Lang scripts - store/restore: has been extended to correctly save/restore plots generated from S-Lang variables - undo/redo: has been extended to correctly save/restore plots generated from S-Lang variables - several enhancements to limits functionality - new --batch and --slscript startup options - deprecated 'rename' command GUIs ---- - cell-based table and image editing has been added to PRISM, including capability to add rows and columns - PRISM now maintains a recently-visited files list - replaced non-editable quicklook plots in PRISM with full chips plotting functionality - improved Analysis menu organization in all GUIs - FILTWIN layout enhanced - support for new RA/DEC entry formats in OBSVIS - OBSVIS roll/visibility plot calculations now use auxiliary datafile rather than hardcoded numbers Varmm/S-Lang ------------ - writefits(): extended ISIS fitsio functionality and encapsulated it an within an easy-to-use function, thus providing powerful FITS table and image writing capability from S-Lang in CIAO - will read .varmmrc file at startup, if found - introduced varmm state object, to improve customization of varmm/s-lang behavior in CIAO - introduced new VarmmFunc class to encapsulate S-Lang function as a C++ object - S-Lang structures returned by varmm I/O functions now use underscore prefix to denote fields which contain metadata - structure fields which reflect table column names are no longer lowercased, but rather reflect the case used in the file on-disk - min/max/step arrays are now present in the S-Lang structures generated by the image I/O routines Miscellaneous ------------- - ISIS spectral analysis package now bundled with CIAO, and accessible within ChIPS and SHERPA commandlines via import("isis") - several enhancements to ahelp (command line help), including better performance and access to either local or web-based content - proposal/mission planning tools (OBSVIS and proposal planning tooklit) merged into CIAO distribution - significantly improved user documentation
datamodel . Fix for area calculations not being truncated by field size . Support for row number filtering: dmcopy "evt.fits[#row=100:200,...]" However [cols #row] will fail. . Support to access WCS/Coordinate columns as virtual columns dmlist "foo_evt.fits[cols ra,dec]" . Fixed problems with string length when complex regions were used . Fix for bug in rebinning non-square images: dmcopy "image.fits[bin x=4,y=4]" rebin_image.fits . DM enhancements to support the definition of a null value when making images, use "[opt null=value]" . Problems with many-point-polygon region filtering fixed. . Can suppress BTYPE/BSCAL FITS rescaling using "[opt scale=no]" . Can rename columns on-the-fly with "[cols new=old,...]" syntax. parameter lib . Fixed some error messages to indicate which parameter files was being accessed. regions . Fixed bug that didn't parse "'" correct for arcmin. . new "bounds()" functionality to get a bounding rectangle around whatever region is supplied caldb . Added ability to override any/all parameters from the command line. stacklib . Now allows one to escape "," in file names using "\," . Blank lines (strlen()==0) in input list files (@foo.lis) are now ignored.
threads (documents_threads.html) . a number of new threads developed . existing threads reviewed and updated to include CIAO 2.2 capability . threads now available as pdf documents scripts (download_scripts.html) . new and major revisions for ciao scripts . most widely used scripts taken from the 'user contributed' area, and incorporated as a part of the standard CIAO build. these scripts are "psextract", "acis_fef_lookup", "add_grating_orders", and "add_grating_spectra". manuals (documents_manuals.html) . significant revisions to the data model manual(s) and detect guide help system (ahelp) (documents_ahelp.html) . added the "-i" option to access the "i"nternet version of the ahelp file, at "http://cxc.harvard.edu/ciao/ahelp/[toolname].html". this complements the existing "-w" flag, which accesses the *local* html version of the ahelp document. . cleaned up the appearance of online ahelp files and created a subject-ordered list . ahelp files now available as pdf documents www . redesigned the ATODMB (../atomdb) web pages