Using psextract to Extract ACIS Spectra and Response Files for Pointlike Sources
CIAO 4.2 Science Threads
Last Update: 12 Jan 2010 - updated for CIAO 4.2: ObsID 459 file versions and corresponding minor changes to screen output; calibration update - the ACIS QE contamination model has been upgraded to vN0005.
The psextract script allows you to compute different RMFs and ARFs for the source and background spectra, if needed. The output files are the source and (ungrouped) background spectra and the source RMF and ARF. If the source and background extraction regions have different average detector coordinates or the source and background event files are different, then the background RMF and ARF and a "linearly grouped" (by a factor of 20) background spectrum are also created.
To generate source and background PI (PHA) spectra of a pointlike ACIS source and build the proper RMFs and ARFs.
Read this thread if:
you are working with any ACIS observation, whether imaging or grating data.
If the source is at the very edge of a CCD, subarray or window, do not use this thread. Instead, run the Step-by-Step Guide to Creating ACIS Spectra thread which includes the use of the mask file when making an ARF.
CALDB v4.2.0 (15 Dec 2009):
The ACIS QE contamination model has been upgraded to vN0005. This version of the file contains separate components for the correction on ACIS-S and ACIS-I. The new calibration file is used by default when creating ARFs and instrument maps in CIAO 4.2.
The changes in the vN0005 file will primarily affect low-energy absorption components between C-K edge (0.283 keV) and 1.2 keV. In general, hard spectra will not be affected much by the calibration change.
Users should read the ACIS QE Contamination Model vN0005 section of the CIAO 4.2 release notes for information on how this change will affect the response file.
- Step-by-Step Guide to Creating ACIS Spectra: the manual version of this thread.
- Get Started
- Downloading the Scripts
- Build Source and Background Regions
- Run psextract
- Update Background File Header
- Analysis Caveats
- Parameter files:
Sample ObsID used: 459 (HETG/ACIS-S, 3C 273)
File types needed: evt2; asol1; pbk0
Please ensure that you have set up ardlib to use the bad pixel file for your observation before following this thread.
The tool mkacisrmf is used to create RMFs for:
- all -120 ACIS data taken in (V)FAINT mode that has the time-dependent gain adjustment and CTI correction applied
- -120 ACIS GRADED mode data on the back-illuminated chips (ACIS-S1 and S3) only
- -110 ACIS data taken on the back-illuminated chips (ACIS-S1 and S3) only
All new analyses with these types of data should be done with mkacisrmf instead of mkrmf. Although this script runs mkrmf, it can still be used to create the spectrum and ARF files for the data. Then follow the Creating ACIS RMFs with mkacisrmf thread to generate new RMFs.
It is important that the calibration applied to the event file is consistent with the RMF tool chosen, as explained in the "Using Consistent Calibration" section of the why topic. If necessary, reprocess your data with the correct gain file before beginning this thread.
unix% cat $ASCDS_CONTRIB/VERSION.CIAO_scripts 17 Apr 2009
Please check that you have at least this version of the scripts package installed before continuing. If you do not have the scripts installed or need to update to a newer version, refer to the Scripts page.
We need to define two regions, one for the source and another for the background. To do this, first display the image:
unix% ds9 acisf00459N004_evt2.fits &
In this example, we define the jet as the source with a rectangle (see this FAQ on how to rotate shapes in ds9) and four 10-pixel radius circles for the background (from source-free parts of the image around the source). All the regions are shown in Figure 1. The background region can also be selected from a different chip or different event file, if desired.
The source and background region(s) must each be confined to a single chip. psextract cannot correctly handle the case where a region contains events from two different chips.
To save the regions, follow these steps:
- Region → Save Regions... → Save As "3c273.reg" (source) and "3c273_bg.reg" (background). To select multiple regions for saving, hold down the <SHIFT> key and click on each one.
- After choosing "OK" in the region filename dialog, a format dialog is opened. Set the format to "CIAO" and the coordinate system to "Physical".
The resulting region files will look like:
unix% more 3c273.reg # Region file format: CIAO version 1.0 rotbox(4148.125,4043.625,7.58978,22.338761,44.516094) unix% more 3c273_bg.reg # Region file format: CIAO version 1.0 circle(4119,4014.75,10) circle(4077,4025.75,10) circle(4186.5,4023.25,10) circle(4196.25,4064.5,10)
psextract runs the following tools in order:
- dmextract: to extract source and background spectra
- dmkeypar: to find out the start-time of the observation (for source and background)
- dmstat: to get statistics on chipx, chipy, ccd_id, x and y (for source and background)
- acis_fef_lookup: to find the proper FEF file (for source and background)
- mkrmf: to build the proper RMF(s)
- asphist: to create the aspect histogram(s)
- mkarf: to create the proper ARF(s) In the case of grating data, psextract reads the GRATING keyword from the header of the event file and passes it to mkarf
- dmgroup: to group the source spectrum and, if needed, the background spectrum
- dmhedit: to update the BACKFILE, RESPFILE and ANCRFILE keys in the source PHA file, and the RESPFILE and ANCRFILE keys in the "linearly grouped" background PHA file (if needed).
In this case, since we are using an ACIS/HETG observation, mkarf will extract the 0th order spectra and ARF.
To extract the spectra in PI space and group the source spectrum to contain the new grouping-scheme; that is, there will be a minimum number of 15 counts per new channel:
unix% punlearn psextract unix% pset psextract events="acisf00459N004_evt2.fits[sky=region(3c273.reg)]" unix% pset psextract bgevents="acisf00459N004_evt2.fits[sky=region(3c273_bg.reg)]" unix% pset psextract pbkfile=acisf063875928N003_pbk0.fits unix% pset psextract dafile=CALDB unix% pset psextract root=3c273 unix% pset psextract asol=@pcad_asol1.lis unix% pset psextract gtype=NUM_CTS unix% pset psextract gspec=15
unix% cat pcad_asol1.lis pcadf063874624N003_asol1.fits
Running the tool with verbose=2 shows what it is doing:
unix% psextract verbose=2 Source events specification (acisf00459N004_evt2.fits[sky=region(3c273.reg)]): Background events specification (acisf00459N004_evt2.fits[sky=region(3c273_bg.reg)]): Root name for output files (3c273): Source aspect solution file (@pcad_asol1.lis): Background aspect solution file (): NONE, or name of the parameter block file (acisf063875928N003_pbk0.fits): events=acisf00459N004_evt2.fits[sky=region(3c273.reg)] bgevents=acisf00459N004_evt2.fits[sky=region(3c273_bg.reg)] root=3c273 aoff=@pcad_asol1.lis bgaoff=@pcad_asol1.lis pbkfile=acisf063875928N003_pbk0.fits dafile=CALDB ptype=pi gtype=NUM_CTS grouping=15 verbose=2 Extract source spectrum 3c273.pi from acisf00459N004_evt2.fits[sky=region(3c273.reg)] dmextract infile="acisf00459N004_evt2.fits[sky=region(3c273.reg)][bin pi]" outfile=3c273.pi opt=pha1 clobber=no verbose=0 Extract background spectrum 3c273_bg.pi from acisf00459N004_evt2.fits[sky=region(3c273_bg.reg)] dmextract infile="acisf00459N004_evt2.fits[sky=region(3c273_bg.reg)][bin pi]" outfile=3c273_bg.pi opt=pha1 clobber=no verbose=0 Inspecting events file acisf00459N004_evt2.fits[sky=region(3c273.reg)] Inspecting background events file acisf00459N004_evt2.fits[sky=region(3c273_bg.reg)] Running dmstat on source and background events files: Source Event statistics: CHIP 7 282.43352601 372.19845857 SKY 4144.2868365 4046.6510732 Background Event statistics: CHIP 7 293.6076555 382.18660287 SKY 4140.4038366 4031.5951926 Look up FEF spectral calibration file(s) Source FEF file is /soft/ciao/CALDB/data/chandra/acis/fef_pha/acisD1999-09-16fef_phaN0002.fits [FUNCTION][ccd_id=7,chipx=257:288,chipy=353:384] Background FEF file is /soft/ciao/CALDB/data/chandra/acis/fef_pha/acisD1999-09-16fef_phaN0002. fits[FUNCTION][ccd_id=7,chipx=289:320,chipy=353:384] Verified Source FEF file is present Verified background FEF file is present Source and background FEFs are different Building different RMFs and ARFs for source and background Make source RMF file 3c273.rmf with mkrmf mkrmf infile=/soft/ciao/CALDB/data/chandra/acis/fef_pha/acisD1999-09-16fef_phaN0002.fits[FUNCTION] [ccd_id=7,chipx=257:288,chipy=353:384] outfile=3c273.rmf logfile=./.psp.mlog axis1=energy=0.1:11.0:0. 01 axis2=pi=1:1024:1 clobber=no verbose=2 Created source RMF file 3c273.rmf Make background RMF file 3c273_bg.rmf with mkrmf mkrmf infile=/soft/ciao/CALDB/data/chandra/acis/fef_pha/acisD1999-09-16fef_phaN0002.fits[FUNCTION] [ccd_id=7,chipx=289:320,chipy=353:384] outfile=3c273_bg.rmf logfile=./.psp.bgmlog axis1=energy=0.1:11. 0:0.01 axis2=pi=1:1024:1 clobber=no verbose=2 Created background RMF file 3c273_bg.rmf Running asphist to create source aspect histogram 3c273.asphist asphist infile="@pcad_asol1.lis" outfile=3c273.asphist evtfile="acisf00459N004_evt2.fits[ccd_id=7]" clobber=no dtffile="" verbose=0 Running mkarf to create source 3c273.arf at (4144.2868365,4046.6510732) mkarf detsubsys=ACIS-S3 outfile=3c273.arf asphistfile="3c273.asphist" sourcepixelx=4144.2868365 sourc epixely=4046.6510732 grating=HETG obsfile="3c273.asphist" pbkfile=acisf063875928N003_pbk0.fits dafile=CALDB maskfile=NONE verbose=0 engrid="grid(3c273.rmf[MATRIX][cols ENERG_LO,ENERG_HI])" clobber=no Running mkarf to create background 3c273_bg.arf at (4140.4038366,4031.5951926) mkarf detsubsys=ACIS-S3 outfile=3c273_bg.arf asphistfile="3c273.asphist" sourcepixelx=4140.4038366 so urcepixely=4031.5951926 grating=HETG obsfile="3c273.asphist" pbkfile=acisf063875928N003_pbk0.fits dafile=CALDB maskfile=NONE verbose=0 engrid="grid(3c273_bg.rmf[MATRIX][cols ENERG_LO,ENERG_HI])" clobber=no Group source spectrum using NUM_CTS 15 dmgroup infile=3c273.pi outfile=./3c273.tmp binspec= tabspec= grouptype=NUM_CTS grouptypeval=15 ycolu mn=counts xcolumn=channel tabcolumn= clobber=no verbose=0 Source spectrum 3c273.pi has been grouped Group Background spectrum using BIN=1:1024:20 dmgroup infile=3c273_bg.pi outfile=3c273_bg_grp.pi binspec="1:1024:20" tabspec= grouptype=BIN groupty peval= ycolumn=counts xcolumn=channel tabcolumn= clobber=no verbose=0 Background spectrum 3c273_bg_grp.pi has been grouped Updating PHA file 3c273.pi with BACKFILE RESPFILE ANCRFILE keys Background spectrum (BACKFILE) = 3c273_bg.pi SOURCE RMF (RESPFILE) = 3c273.rmf SOURCE ARF (ANCRFILE) = 3c273.arf Updating PHA file 3c273_bg_grp.pi with RESPFILE ANCRFILE key BACKGROUND RMF (RESPFILE) = 3c273_bg.rmf BACKGROUND ARF (ANCRFILE) = 3c273_bg.arf
Important: the output source spectrum (in this case, 3c273.pi) is grouped if a grouping was specified; in this case, there is not an ungrouped spectrum created.
Note that the ungrouped background file name is used as the BACKFILE header keyword value. The source grouping is applied to the background grouping when fitting in Sherpa. For fitting only background data, or simultaneous fitting of source and background data, Sherpa can group background dynamically with the group functions; see the Fitting section of this thread for more information.
The contents of the parameter file may be checked with plist psextract.
Currently psextract only updates the RESPFILE and ANCRFILE header keywords in the source and grouped background file. These should be added to the ungrouped background file as well.
unix% dmhedit infile=3c273_bg.pi filelist="" operation=add key=RESPFILE value=3c273_bg.rmf unix% dmhedit infile=3c273_bg.pi filelist="" operation=add key=ANCRFILE value=3c273_bg.arf
There are several things to note about how the tool works:
Due to the satellite dithering, a point-like source can in principle move across all 4 nodes of a single chip (or different chips) during the observation. This tool builds ARFs and RMFs for the position occupied by the source during most of its exposure (> 50 %).
The output source and background spectra are by default created in the PI channel space, and so are the respective response matrices. To create PHA output files, set the ptype parameter to pha:
unix% pset psextract ptype=pha
psextract uses a hardcoded binning for the RMF of axis2 = (energy=0.1:11.0:0.01), which defines the binning to go from Emin=0.1 keV to Emax=11 keV in steps of 0.01 keV.
Users should be cautious about analyzing the data for sources near the edges of the ACIS CCDs.
For X-rays passing through the mirrors, the very bottom of each CCD is obscured by the frame store. As a result, some of the events in rows with CHIPY <= 8 are not detected. (The set of rows affected varies from CCD to CCD.) Since the CIAO tools do not compensate for this effect, the ARFs and exposure maps for sources in these regions may be inaccurate.
For sources within about thirty-two pixels of any edge of a CCD, the source may be dithered off the CCD during part of an observation. The aspect histogram, which is used to create ARFs and exposure maps, is designed to compensate for this effect.
An ARF calculated at the edge of a chip will not be accurate. The response tools for spectral extraction (specifically the ARF) assume that 100% of the PSF is enclosed - i.e. on the chip - all the time, which may not be the case. The amount of error introduced depends on how close the source is to the edge, the morphology of the source, and the characteristics of the PSF, which depends on the source spectrum.
A contaminant has accumulated on the optical-blocking filters of the ACIS detectors, as described in the ACIS QE Contamination why topic. Since there is a gradient in the temperature across the filters (the edges are colder), there is a gradient in the amount of material on the filters. (The contaminant is thicker at the edges.) Within about 100 pixels of the outer edges of the ACIS-I and ACIS-S arrays, the gradient is relatively steep. Therefore, the effective low-energy (' 1 keV) detection efficiency may vary within the dither pattern in this region. The ARF and instrument map tools are designed to read a calibration file which describes this spatial dependence.
If you would like to fit the background-subtracted source spectrum using a common RMF and ARF for source and background, simply read the source spectrum FITS file into Sherpa, subtract the background, and fit it. See the Introduction to Fitting PHA Spectra thread for details.
To fit source and background spectra simultaneously with distinct RMFs and ARFs, follow the Independent Background Responses thread.
The thread is now complete. The source and the ungrouped background spectra are named 3c273.pi and 3c273_bg.pi; the grouped background spectrum is named 3c273_bg_grp.pi. The source and background (common) RMF is named 3c273.rmf, while source and background ARFs are named 3c273.arf and 3c273_bg.arf.
Parameters for /home/username/cxcds_param/psextract.par events = acisf00459N004_evt2.fits[sky=region(3c273.reg)] Source events specification bgevents = acisf00459N004_evt2.fits[sky=region(3c273_bg.reg)] Background events specification root = 3c273 Root name for output files asol = @pcad_asol1.lis Source aspect solution file bgasol = Background aspect solution file pbkfile = acisf063875928N003_pbk0.fits NONE, or name of the parameter block file (dafile = CALDB) CALDB, NONE, or name of ACIS dead-area calibration file (ptype = pi) pi or pha (gtype = NUM_CTS) Spectrum grouping type (same as grouptype in dmgroup) (gspec = 15) Spectrum grouping specs (NONE,1:1024:10,etc) (clobber = no) OK to overwrite existing output file? (verbose = 2) Debug Level(0-5) (mode = ql)
|14 Dec 2004||updated for CIAO 3.2: created Using Consistent Calibration and Downloading the Scripts sections|
|23 Jun 2005||CIAO 3.2.2 patch: new calibration for mkacisrmf is available; see the Using Consistent Calibration: mkrmf vs mkacisrmf section|
|15 Dec 2005||updated for CIAO 3.3: the psextract script has been updated to version 3.3 for the new asphist tool syntax; note about specextract in the Overview; default value of dmextract error and bkgerror parameters is "gaussian" (this tool is called by psextract)|
|01 Feb 2006||added link to specextract thread|
|31 Mar 2006||specextract use update added to Overview; title changed from "Extract ACIS Spectra for Pointlike Sources and Make RMFs and ARFs" to "Using psextract to Extract ACIS Spectra and Response Files for Pointlike Sources"|
|14 Jun 2006||corrected link in "Calibration Updates"; clarified information on GRADED mode data|
|25 Jul 2006||added Creating an RMF with mkacisrmf section, moved "Using Consistent Calibration: mkrmf vs mkacisrmf" section|
|01 Dec 2006||updated for CIAO 3.4: psextract version 3.4 (mkarf obsfile parameter is set to event file instead of asphist file); acis_fef_lookup version 1.19 (changes for CTI lookup)|
|02 Feb 2007||updated for CALDB 18.104.22.168 patch|
|26 Feb 2007||psextract v4.0 and acis_fef_lookup v1.20: Dead area correction parameters added to the psextract.par file: pbkfile and dafile (added ACIS dead area correction section). Script prints a warning if RMF should be remade with mkacisrmf. Several other minor code changes that don't affect the output files (see the Script History page for details).|
|22 Jan 2008||updated for CIAO 4.0: psextract v4.0.1 (parameter updates for applying ACIS dead area correction, which is turned on by default); acis_fef_lookup v1.21 (updated to to run under S-Lang v2; functionality is unchanged); available links updated to point to Sherpa Beta website; removed old calibration updates|
|31 Mar 2008||updated for CALDB 3.4.3: use mkacisrmf for -110 BI chips if TGAIN calibration has been applied|
|30 May 2008||psextract v4.0.3 (Unix head and tail commands replaced by pget and dmkeypar); thread images are inline|
|04 Aug 2008||the source and background region(s) must be confined to a single chip|
|02 Feb 2009||updated for CIAO 4.1: image converted to inline; acis_fef_lookup vCIAO 4.1 - 1.0 (rewritten to be compatible with CALDB 4.1); "ARDLIB warning ... Assuming the first "interesting" extension." no longer printed; updated Sherpa thread link to 4.1 version; input data must have a CTI_APP keyword|
|05 Mar 2009||added information about matching the ARF and RMF grids to the Creating an RMF with mkacisrmf section|
|06 May 2009||check the version of the CIAO scripts package instead of the individual script|
|21 May 2009||added explanation of why ungrouped background filename is used in the BACKFILE header keyword of the grouped source spectrum|
|12 Jan 2010||updated for CIAO 4.2: ObsID 459 file versions and corresponding minor changes to screen output; calibration update - the ACIS QE contamination model has been upgraded to vN0005.|