Extract Spectrum and Response Files for an Extended Source
CIAO 4.15 Science Threads
Overview
Synopsis:
Using a combination of CIAO tools, we extract source and background spectra for an extended source. The background spectrum is grouped, if desired. The appropriate Response Matrix Files (RMFs) and Ancillary Response Files (ARFs) are created for both source and background.
The specextract script automates these steps for extended and pointlike sources observed with the ACIS detector.
Due to on going thermal challenges, some ACIS observations will be taken with the focal plane temperature, FP_TEMP, warmer than the currently calibrated high temperature limit of -109 C (164.15 K). Use the following command to get the median FP_TEMP in degrees Kelvin for the observation:
Users fitting ACIS imaging spectra that fulfill all the following three conditions may see line-centroids shifted by 1-2% from systematic offsets due to calibration uncertainty:
- Spectra with more than 1000 counts on a front-illuminated (FI) chip or more than 2000 counts on a back-illuminated (BI) chip (CCD_ID 5,7).
- More than half of source counts were acquired above the current temperature limit of -109 C.
- More than half of source counts are in emission or absorption lines.
Under these conditions, users may benefit from trying to identify time periods when the focal plane temperature was within the currently calibrated range, below -109C, using the following thread: Removing Warm ACIS Data.
Purpose:
To generate source and background spectra of an extended ACIS source and build the proper RMFs and ARFs.
Related Links:
- The Extract Spectrum and Response Files for a Pointlike Source thread handles the pointlike source case.
Last Update: 18 Jan 2022 - Review for CIAO 4.14. Updated for Repro5 (coordinates) and CALDB 4.9.6.
Contents
- Get Started
- Build Source and Background Regions
- Using specextract
- Step-by-Step Guide
- Extracting Multiple Spectra
- Fitting
- Analysis Caveats
- Parameter files:
- History
- Images
Get Started
Download the sample data: 869 (ACIS-S, ARP 220)
unix% download_chandra_obsid 869
Re-process it with chandra_repro:
unix% cd 869 unix% punlearn chandra_repro unix% chandra_repro mode=h Output from acis_process_events: Output from acis_process_events: # acis_process_events (CIAO): The following error occurred 81 times: dsAPEPULSEHEIGHTERR -- WARNING: pulse height is less than split threshold when performing serial CTI adjustment.
As only a small number of events are reported, the warning can be ignored.
The files we are going to use are in the repro/ directory, shown in bold (note that only the event file needs to be explicitly specified since the remaining files can be found automatically by specextract):
unix% cd repro/ unix% ls acisf00869_000N005_bpix1.fits acisf00869_asol1.lis acisf00869_000N005_fov1.fits acisf00869_repro_bpix1.fits acisf00869_000N005_msk1.fits acisf00869_repro_evt2.fits acisf00869_000N005_mtl1.fits acisf078247287N005_pbk0.fits acisf00869_000N005_stat1.fits pcadf00869_000N001_asol1.fits
How specextract Works
specextract runs the following tools for the extended source case:
- dmextract: to extract source and (optionally) background spectra. This tool also creates the WMAP used as input to mkacisrmf.
- sky2tdet: to create the WMAP input for mkwarf.
- mkwarf: to create weighted ARF(s).
- mkrmf or mkacisrmf: to build the RMF(s), depending on which is appropriate for the data and the calibration; see the Creating ACIS RMFs why topic for details.
- dmgroup: to group the source spectrum and/or background spectrum.
- dmhedit: to update the BACKFILE, RESPFILE and ANCRFILE keys in the source and background spectrum files.
Responses
Weighted ARFs are generated by default, since weight=yes. For point-source analysis, or when you are not interested in the spatial variation of the effective area, unweighted ARFs can be calculated by setting weight=no.
RMFs are - by default - generated without any weighting, since this can considerably speed up the running of the script and - for most cases - does not significantly change the result, since the RMF variation with position is small. If you do want to create weighted RMF files, then set weight_rmf=yes and weight=yes.
Users can also specify a location at which to calculate unweighted responses - using the refcoord parameter - rather than using the location of the source and background regions.
Build Source and Background Regions
We need to define two regions, one for the source and another for the background. To do this, first display the file:
unix% ds9 acisf00869_repro_evt2.fits &
Refer to the Using CIAO Regions thread for information on creating region files. The CIAO-format files for this example are:
unix% cat simple.reg # Region file format: CIAO version 1.0 ellipse(4026.2341,4125.3693,50,40,0) unix% cat simple_bkg.reg # Region file format: CIAO version 1.0 circle(3955.5,4252.5,50)
The regions are shown displayed on the event file in Figure 1.
[Version: full-size]
Figure 1: Extraction regions on the event file
Choosing a background file
The examples in this thread use a background region that was chosen from a source-free area of the same chip as the source region. The source or field of sources may cover the chip, however, making it impossible to select a local background. There are several options in this case:
-
Choose the background from another same-type chip that was on in the observation. If your source is on ACIS-S3 and ACIS-S1 (the other back-illuminated chip) was also turned on, define the background on ACIS-S1.
-
If using another chip from the observation is not an option, use one of the ACIS "blank-Sky" background files which are distributed in the CALDB.
If you plan on using one of the ACIS "blank-sky" background files from the CALDB and intend to subtract the background spectrum (i.e. not fit it), then you do not need to create a background RMF and ARF. Set the bkgresp parameter to "no" and the script will only extract a background spectrum.
- Omit the background completely (i.e. leave the bkgfile blank).
Using specextract
1. Set the input parameters
specextract can determine the ancillary files (aspect solution, bad pixel, mask, DTF if HRC data) from the header of the event file, which means that you only need give the source file, output directory, and - if desired - the background file:
unix% punlearn specextract unix% pset specextract infile="acisf00869_repro_evt2.fits[sky=region(simple.reg)]" unix% pset specextract outroot=simple unix% pset specextract bkgfile="acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)]"
2. Run the script
Running the tool is then just (the lines in bold indicate the ancillary files that have been picked up automatically):
unix% specextract mode=h Running specextract Version: 18 November 2021 Aspect solution file repro/pcadf00869_000N001_asol1.fits found. Bad pixel file repro/acisf00869_repro_bpix1.fits found. Mask file repro/acisf00869_000N005_msk1.fits found. Setting bad pixel file Setting bad pixel file Extracting src spectra Extracting bkg spectra Using mkacisrmf... Using mkacisrmf... Creating bkg ARF Using mkacisrmf... Creating bkg RMF Creating src ARF Using mkacisrmf... Creating src RMF Grouping src spectrum Updating header of simple.pi with RESPFILE and ANCRFILE keywords. Updating header of simple_grp.pi with RESPFILE and ANCRFILE keywords. Updating header of simple.pi with BACKFILE keyword. Updating header of simple_grp.pi with BACKFILE keyword. Updating header of simple_bkg.pi with RESPFILE and ANCRFILE keywords.
The contents of the parameter file may be checked with plist specextract.
If you want more details on what the script is doing, set verbose=2 before running specextract.
Ancillary files
If you want to explicitly give the location of the ancillary files then, in this case, you would add the following before the call to specextract (note that the asp parameter accepts a stack for those observations with multiple aspect solution files):
unix% pset specextract asp=pcadf00869_000N001_asol1.fits
or
unix% pset specextract asp=@acisf00869_asol1.lis
and for the other files:
unix% pset specextract mskfile=acisf00869_000N005_msk1.fits unix% pset specextract badpixfile=acisf00869_repro_bpix1.fits
Background responses
The above call created an ARF and RMF for the background region; if you only intend to subtract the background, rather than fitting it, you do not need these responses so can save time (particularly if the background area is large) by setting bkgresp=no when running specextract.
Grouping the output spectra
We choose to use the default grouping values: the source spectrum will be grouped to a minimum number of 15 counts per new channel and the background spectrum will be ungrouped. The grouptype and binspec parameters are used to specify the source spectrum grouping, and the bkg_grouptype and bkg_binspec parameters specify the background spectrum grouping.
Setting the energy range for the WMAP
The energy_wmap parameter determines the energy band used to create the weight map when weight=yes.
If you use the soft band (e.g. the default value of 300:2000 eV), then you are weighting by the emission from the source (unless you have a very hard spectrum). To weight by the effective area of the telescope (again, unless you have a hard source), set the range to, for example, 2000:7000 eV.
Unless the spectrum of your source is unusual, adjusting the WMAP energy range will not affect the ARF - or, therefore, the spectral fit - noticeably.
3. Examining the Output Files
The number of files created depends on if a background event file was provided and if source and/or background grouping was specified. In this case, the output files are:
Source:
File | Description |
---|---|
simple.pi | ungrouped spectrum |
simple.arf | ARF (weighted) |
simple.rmf | RMF (weighted) |
simple_grp.pi | grouped spectrum |
Background:
File | Description |
---|---|
simple_bkg.pi | ungrouped spectrum |
simple_bkg.arf | weighted ARF |
simple_bkg.rmf | weighted RMF |
Step-by-Step Guide
This section explains how to manually run the analysis steps that are used by specextract.
Make sure that you have set up ardlib to use the bad pixel file for your observation before following this section of the thread.
1. Extract Source and Background Spectra
First, we extract spectra from the events enclosed by the source and background regions.
To create a WMAP block in a PHA file, dmextract must be given a binning specification for its wmap option. This WMAP will be used as input to mkacisrmf. It must be in detector coordinates, and the suggested binning factor is 8; an optional energy filter is included for the WMAP.
unix% punlearn dmextract unix% pset dmextract infile="acisf00869_repro_evt2.fits[sky=region(simple.reg)][bin pi]" unix% pset dmextract outfile=simple_steps.pi unix% pset dmextract wmap="[energy=300:2000][bin tdet=8]" unix% dmextract mode=h
And for the background spectrum:
unix% pset dmextract infile="acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)][bin pi]" unix% pset dmextract outfile=simple_steps_bkg.pi unix% dmextract mode=h
You can check the parameter file that was used with plist dmextract.
2. Calculate the ARFs
Compute the Aspect Histogram (asphist)
We now need to create the aspect histogram, which is a binned representation of aspect motion during the observation.
In few cases, there will be more than one aspect solution file (pcad_asol1.fits) for an observation. All the files must be input to the infile parameter, either as a list or as a stack. Here we use the stack file generated by chandra_repro:
unix% cat acisf00869_asol1.lis /data/ciao/869/primary/pcadf00869_000N001_asol1.fits
We also need to know what chip the source and background regions are on:
unix% dmstat "acisf00869_repro_evt2.fits[sky=region(simple.reg)][cols ccd_id]" ccd_id min: 7 @: 1 max: 7 @: 1 mean: 7 sigma: 0 sum: 17906 good: 2558 null: 0 unix% dmstat "acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)][cols ccd_id]" ccd_id min: 7 @: 1 max: 7 @: 1 mean: 7 sigma: 0 sum: 10276 good: 1468 null: 0
Each region is on ccd_id=7. This means we only need to make one aspect histogram; if the regions were on different chips, we would make one for each chip. Now run the tool:
unix% punlearn asphist unix% pset asphist infile=@acisf00869_asol1.lis unix% pset asphist outfile=simple_steps.asphist unix% pset asphist evtfile="acisf00869_repro_evt2.fits[ccd_id=7]" unix% asphist mode=h
The contents of the parameter file may be checked with plist asphist.
Create a WMAP for the ARF (sky2tdet)
The WMAP created by sky2tdet properly weights the ARF based on how much of the source flux fell onto the bad pixels, columns, or a node boundary, and which bad pixels are actually exposed. Without accounting for these effects, the ARF is over-estimated.
unix% punlearn sky2tdet unix% pset sky2tdet infile="acisf00869_repro_evt2.fits[sky=region(simple.reg)][energy=300:2000][bin sky=1]" unix% pset sky2tdet asphistfile="simple_steps.asphist" unix% pset sky2tdet outfile="simple_steps_tdet.fits[wmap]" unix% sky2tdet mode=h
If you made a separate aspect histogram for the background, set it in the asphistfile parameter before running the tool for the background spectrum.
unix% pset sky2tdet infile="acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)][energy=300:2000][bin sky=1]" unix% pset sky2tdet outfile="simple_steps_bkg_tdet.fits[wmap]" unix% sky2tdet mode=h
You can check the parameter file that was used with plist sky2tdet.
Compute the ARFs (mkwarf)
mkwarf creates a weighted ARF file based on the WMAP that was created with sky2tdet. It also produces a set of weights that are used by mkrmf to create the associated weighted RMF.
Run mkwarf for the source:
unix% punlearn mkwarf unix% pset mkwarf infile="simple_steps_tdet.fits[wmap]" unix% pset mkwarf outfile=simple_steps.arf unix% pset mkwarf weightfile=simple_steps.wfef unix% pset mkwarf egridspec=0.3:11.0:0.01 unix% pset mkwarf mskfile=acisf00869_000N005_msk1.fits unix% mkwarf mode=h
and for the background:
unix% pset mkwarf infile="simple_steps_bkg_tdet.fits[wmap]" unix% pset mkwarf outfile=simple_steps_bkg.arf unix% pset mkwarf weightfile=simple_steps_bkg.wfef unix% mkwarf mode=h
You can check the parameter file that was used with plist mkwarf.
3. Calculate the RMFs
The observation used in this thread (ObsID 869) was taken at the -120 C focal plane temperature. Therefore, mkacisrmf should be used to create the RMF file.
The syntax for both mkacisrmf and mkrmf are given in this section. Users should choose the appropriate tool for the data and calibration.
Using mkacisrmf
The mkacisrmf why topic has more details on using the mkacisrmf tool.
The RMF doesn't change quickly enough to require the detail of the sky2tdet WMAP, and its precision will cause mkacisrmf to run very slowly. Nearly identical results are obtained with the dmextract WMAP with a more reasonable runtime.
For the source:
unix% punlearn mkacisrmf unix% pset mkacisrmf infile=CALDB unix% pset mkacisrmf outfile=simple_steps_mkacisrmf.rmf unix% pset mkacisrmf energy=0.3:11.0:0.01 unix% pset mkacisrmf channel=1:1024:1 unix% pset mkacisrmf wmap="simple_steps.pi[WMAP]" unix% mkacisrmf mode=h Total 18 regions to be processed: 1> reg# 1263 processed 2> reg# 1264 processed 3> reg# 1265 processed 4> reg# 1266 processed 5> reg# 1294 processed 6> reg# 1295 processed 7> reg# 1296 processed 8> reg# 1297 processed 9> reg# 1298 processed 10> reg# 1326 processed 11> reg# 1327 processed 12> reg# 1328 processed 13> reg# 1329 processed 14> reg# 1330 processed 15> reg# 1359 processed 16> reg# 1360 processed 17> reg# 1361 processed 18> reg# 1391 processed
For the background, repeat the command with the background spectrum:
unix% pset mkacisrmf outfile=simple_steps_bkg_mkacisrmf.rmf unix% pset mkacisrmf wmap="simple_steps_bkg.pi[WMAP]" unix% mkacisrmf mode=h Total 17 regions to be processed: 1> reg# 1393 processed 2> reg# 1394 processed 3> reg# 1395 processed 4> reg# 1424 processed 5> reg# 1425 processed 6> reg# 1426 processed 7> reg# 1427 processed 8> reg# 1456 processed 9> reg# 1457 processed 10> reg# 1458 processed 11> reg# 1459 processed 12> reg# 1488 processed 13> reg# 1489 processed 14> reg# 1490 processed 15> reg# 1491 processed 16> reg# 1521 processed 17> reg# 1522 processed
You can check the parameter file that was used with plist mkacisrmf.
You can now skip to the Update the Spectrum Files section.
Using mkrmf
The mkrmf tool uses the weightfile from mkwarf to create the weighted RMF. The tool queries the CALDB for the correct FEF file and the energy axis grid is ignored since the grid is read from the weights file. Note that although the axis1 parameter is ignored, a syntactically-correct value must be entered; this example uses axis1="energy=0:1".
For the source:
unix% punlearn mkrmf unix% pset mkrmf infile=CALDB unix% pset mkrmf outfile=simple_steps_mkrmf.rmf unix% pset mkrmf axis1="energy=0:1" unix% pset mkrmf axis2="pi=1:1024:1" unix% pset mkrmf weights=simple_steps.wfef unix% mkrmf mode=h
and for the background:
unix% pset mkrmf outfile=simple_steps_bkg_mkrmf.rmf unix% pset mkrmf weights=simple_steps_bkg.wfef unix% mkrmf mode=h
You can check the parameter file that was used with plist mkrmf.
4. Update the Spectrum Files
Group the Source Spectrum (dmgroup)
Sherpa has dynamic grouping capabilities so you can easily set or change the grouping of a dataset at the Sherpa prompt. For other data analysis packages it can be useful to create a grouped version of the spectrum as a seprate file.
The source spectrum is grouped to have a minimum number of 15 counts per new channel:
unix% punlearn dmgroup unix% pset dmgroup infile=simple_steps.pi unix% pset dmgroup outfile=simple_steps_grp.pi unix% pset dmgroup grouptype=NUM_CTS unix% pset dmgroup grouptypeval=15 unix% pset dmgroup xcolumn=channel unix% pset dmgroup ycolumn=counts unix% dmgroup mode=h
You can check the parameter file that was used with plist dmgroup.
Update File Headers (dmhedit)
Finally, update the appropriate header keywords in the both the ungrouped and grouped source files.
Note that the ungrouped background file name is used as the BACKFILE header keyword value. The source grouping is applied to the background grouping when fitting in Sherpa. For fitting only background data, or simultaneous fitting of source and background data, Sherpa can group background dynamically with the group functions; see the Fitting section of this thread for more information.
unix% punlearn dmhedit unix% pset dmhedit operation=add filelist="" mode=h unix% dmhedit simple_steps.pi key=BACKFILE value=simple_steps_bkg.pi unix% dmhedit simple_steps.pi key=RESPFILE value=simple_steps_mkacisrmf.rmf unix% dmhedit simple_steps.pi key=ANCRFILE value=simple_steps_corr.arf unix% dmhedit simple_steps_grp.pi key=BACKFILE value=simple_steps_bkg.pi unix% dmhedit simple_steps_grp.pi key=RESPFILE value=simple_steps_mkacisrmf.rmf unix% dmhedit simple_steps_grp.pi key=ANCRFILE value=simple_steps.arf
And for the ungrouped background file:
unix% dmhedit simple_steps_bkg.pi operation=add key=RESPFILE value=simple_steps_bkg_mkacisrmf.rmf unix% dmhedit simple_steps_bkg.pi operation=add key=ANCRFILE value=simple_steps_bkg.arf
Extracting Multiple Spectra
In this example, we show how specextract can create multiple output spectra from a single run of the script.
Build Source Regions and Background Regions
This example uses two observations of G21.5-09. Both observations will be processed by specextract at the same time, producing two sets of output files; this is explained further in the Run specextract section.
We download and reprocess the data:
unix% download_chandra_obsid 1842,1843 ... unix% punlearn chandra_repro unix% chandra_repro indir=1842,1843 mode=h
and then use reproject_obs to reproject the data sets to a common tangent-plane position (i.e. so that the SKY coordinate system is the same in the two event files).
unix% punlearn reproject_obs unix% reproject_obs 1842,1843 reproj/ Running reproject_obs Version: 15 November 2021 Found 1842/repro/acisf01842_repro_evt2.fits Found 1843/repro/acisf01843_repro_evt2.fits Verifying 2 observations. Calculating new tangent point. New tangent point: RA=18h 32m 21.347s Dec=-10d 34' 38.44" Observations to be reprojected: Obsid Obs Date Exp DETNAM SIM_Z FP Sepn PA (ks) (mm) (K) (') (deg) --------------------------------------------------------------- 1 1843 2000-09-02 7.9 ACIS-012367 -245.857 153.4 2.6 -79 2 1842 2000-09-02 7.4 ACIS-012367 -231.239 153.4 2.6 +101 Running tasks in parallel with 4 processors. Reprojecting 2 event files to a common tangent point. Merging reprojected events files to multi_reproj/merged_evt.fits The following files were created: The reprojected FOV files: multi_reproj/1843.fov multi_reproj/1842.fov The combined FOV file: multi_reproj/merged.fov The reprojected event files: multi_reproj/1843_reproj_evt.fits multi_reproj/1842_reproj_evt.fits The merged event file: multi_reproj/merged_evt.fits Warning: the merged event file multi_reproj/merged_evt.fits should not be used to create ARF/RMF/exposure maps because the RA_NOM keyword varies by 0.0851 (limit is 0.0003) the DEC_NOM keyword varies by 0.0159 (limit is 0.0003) the ROLL_NOM keyword varies by 2.5 (limit is 1.0) the SIM_Z keyword varies by 14.6 (limit is 0.1)
As the screen output describes, the merged event file - in this case reproj/merged_eve.fits - should not be used for spectral analysis.
We have defined a source and background region for each observation (so that different radii and, for the backgrounds, centers can be used):
unix% cat 1842_src.reg # Region file format: CIAO version 1.0 circle(1943.3,4164.2,102) unix% cat 1843_src.reg # Region file format: CIAO version 1.0 circle(1943.3,4164.2,135) unix% cat 1842_bg.reg # Region file format: CIAO version 1.0 circle(2259.3,4072.3,100) unix% cat 1843_bg.reg # Region file format: CIAO version 1.0 circle(2361.5,4104.8,140)
The regions are shown displayed on the event files in Figure 2.
[Version: full-size]
Figure 2: Extraction regions on the event files
Run specextract
The event files are input to the script as a stack; this syntax means that a separate spectrum will be created for each of the file/region pairs (since we set combine=no):
unix% cat multi_src.lis reproj/1842_reproj_evt.fits[sky=region(1842_src.reg)] reproj/1843_reproj_evt.fits[sky=region(1843_src.reg)]
When working with stack inputs to specextract, the source and background stacks must contain the same number of items, unless you are not extracting background spectra. Make sure that the background file definitions are in the same order as the source files:
unix% cat multi_bg.lis reproj/1842_reproj_evt.fits[sky=region(1842_bg.reg)] reproj/1843_reproj_evt.fits[sky=region(1843_bg.reg)]
Using an external file for a stack - such as multi_src.lis and multi_bg.lis - is useful when either the contents are going to be used multiple times or the contents are long, but you can also use a comma-separated list of values, which is what we do for the stack of output names (the outroot parameter). If you prefer, you may just give a single value for outroot and specextract will create output files designated as "src1", "src2", "bkg1", "bkg2", etc. The default values for combine, weight, and correctpsf (no, yes, and no respectively) are used. In the output below the bold lines show the ancillary files that were selected by the script.
unix% mkdir spec unix% punlearn specextract unix% pset specextract infile=@multi_src.lis unix% pset specextract outroot=spec/1842,spec/1843 unix% pset specextract bkgfile=@multi_bg.lis unix% specextract mode=h Running specextract Version: 18 November 2021 Aspect solution file multi_reproj/1842.asol found. Aspect solution file multi_reproj/1843.asol found. Bad pixel file multi_reproj/1842.bpix found. Bad pixel file multi_reproj/1843.bpix found. Mask file multi_reproj/1842.mask found. Mask file multi_reproj/1843.mask found. Setting bad pixel file [1 of 2] Setting bad pixel file [2 of 2] Setting bad pixel file [1 of 2] Setting bad pixel file [2 of 2] Extracting src spectra [1 of 2] Extracting bkg spectra [2 of 2] Extracting bkg spectra [1 of 2] Extracting src spectra [2 of 2] Using mkacisrmf... Using mkacisrmf... Using mkacisrmf... Using mkacisrmf... Creating bkg ARF [1 of 2] Using mkacisrmf... Creating bkg RMF [1 of 2] Creating bkg ARF [2 of 2] Using mkacisrmf... Creating bkg RMF [2 of 2] Creating src ARF [1 of 2] Creating src ARF [2 of 2] Using mkacisrmf... Creating src RMF [1 of 2] Using mkacisrmf... Creating src RMF [2 of 2] Grouping src spectrum [1 of 2] Grouping src spectrum [2 of 2] Updating header of spec/1842.pi with RESPFILE and ANCRFILE keywords. Updating header of spec/1842_grp.pi with RESPFILE and ANCRFILE keywords. Updating header of spec/1842.pi with BACKFILE keyword. Updating header of spec/1842_grp.pi with BACKFILE keyword. Updating header of spec/1843.pi with RESPFILE and ANCRFILE keywords. Updating header of spec/1843_grp.pi with RESPFILE and ANCRFILE keywords. Updating header of spec/1843.pi with BACKFILE keyword. Updating header of spec/1843_grp.pi with BACKFILE keyword. Updating header of spec/1842_bkg.pi with RESPFILE and ANCRFILE keywords. Updating header of spec/1843_bkg.pi with RESPFILE and ANCRFILE keywords.
The contents of the parameter file may be checked with plist specextract.
Ancillary files
If you want to explicitly give the location of the ancillary files then, in this case, you would add the following before the call to specextract. The files should be listed in the same order as the infile paramter. So, for the bad-pixel, mask, and aspect solution files:
unix% cat multi_bpix.lis repro/1842.bpix repro/1843.bpix unix% cat multi_mask.lis repro/1842.mask repro/1843.mask unix% cat multi_asp.lis repro/1842.asol repro/1843.asol
Examining the Output Files
The number of files created depends on if a background event file was provided and how the source and background groupings were specified. In this case we have:
Grouped source spectra
unix% ls -1 spec/*grp.pi spec/1842_grp.pi spec/1843_grp.pi
Ungrouped source spectra
unix% ls -1 spec/184?.pi spec/1842.pi spec/1843.pi
Background spectra
These are ungrouped since the default setting bkg_grouptype=NONE was used:
unix% ls -1 spec/*bkg.pi spec/1842_bkg.pi spec/1843_bkg.pi
Responses
These responses are weighted and there are responses for the background spectra, since weight=yes and bkgresp=yes.
unix% ls -1 spec/*arf spec/1842.arf spec/1842_bkg.arf spec/1843.arf spec/1843_bkg.arf unix% ls -1 spec/*rmf spec/1842.rmf spec/1842_bkg.rmf spec/1843.rmf spec/1843_bkg.rmf
Setting outroot to a single value
If the run had used outroot=spec/multi instead then the files would have been labelled as "src1" and "src2"; so the grouped spectra would have been called
unix% ls -1 spec/*grp.pi spec/multi_src1_grp.pi spec/multi_src2_grp.pi
Fitting
Below we show the data from the single source example being read into Sherpa and displayed (Figure 3):
unix% sherpa sherpa In[]: %matplotlib Using matplotlib backend: TkAgg sherpa In[]: load_data('simple.pi') read ARF file simple.warf read RMF file simple.wrmf read ARF (background) file simple_bkg.warf read RMF (background) file simple_bkg.wrmf read background file simple_bkg.pi sherpa In[]: notice(0.3, 7) sherpa In[]: group_counts(20) sherpa In[]: plot_data() sherpa In[]: subtract() sherpa In[]: plot_data(overplot=True) sherpa In[]: import matplotlib.pylab as plt sherpa In[]: plt.xscale("log") sherpa In[]: ply.yscale("log")
[Version: full-size]
Figure 3: Spectrum from ObsId 869
If you would like to fit the background-subtracted source spectrum using a common RMF and ARF for source and background, simply read the source spectrum FITS file into Sherpa, subtract the background, and fit. See the Introduction to Fitting PHA Spectra thread for details.
To fit source and background spectra simultaneously with distinct RMFs and ARFs, follow the Independent Background Responses thread.
The RESPFILE and ANCRFILE header keywords in the source and background spectra have been updated, as well as the BACKFILE in the source spectra. This means that when the spectra are read into Sherpa, all the supporting files will automatically be read as well; the background (if available) will be defined, as will the source and background response files.
Analysis Caveats
Analysis at the edges of ACIS CCDs
Users should be cautious about analyzing the data for sources near the edges of the ACIS CCDs.
-
For X-rays passing through the mirrors, the very bottom of each CCD is obscured by the frame store. As a result, some of the events in rows with CHIPY <= 8 are not detected. (The set of rows affected varies from CCD to CCD.) Since the CIAO tools do not compensate for this effect, the ARFs and exposure maps for sources in these regions may be inaccurate.
-
For sources within about thirty-two pixels of any edge of a CCD, the source may be dithered off the CCD during part of an observation. The aspect histogram, which is used to create ARFs and exposure maps, is designed to compensate for this effect.
-
An ARF calculated at the edge of a chip will not be accurate since the response tools for spectral extraction (specifically the ARF) assume that 100% of the PSF is enclosed - i.e. on the chip - all the time, which may not be the case. The amount of error introduced depends on how close the source is to the edge, the morphology of the source, and the characteristics of the PSF, which depends on the source spectrum.
-
A contaminant has accumulated on the optical-blocking filters of the ACIS detectors, as described in the ACIS QE Contamination why topic. Since there is a gradient in the temperature across the filters (the edges are colder), there is a gradient in the amount of material on the filters. (The contaminant is thicker at the edges.) Within about 100 pixels of the outer edges of the ACIS-I and ACIS-S arrays, the gradient is relatively steep. Therefore, the effective low-energy (' 1 keV) detection efficiency may vary within the dither pattern in this region. The ARF and instrument map tools are designed to read a calibration file which describes this spatial dependence.
Parameters for /home/username/cxcds_param/specextract.par infile = acisf00869_repro_evt2.fits[sky=region(simple.reg)] Source event file(s) outroot = simple Output directory path + root name for output files (bkgfile = acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)]) Background event file(s) (asp = ) Source aspect solution or histogram file(s) (dtffile = ) Input DTF files for HRC observations (mskfile = ) Maskfile (input to mkwarf) (rmffile = CALDB) rmffile input for CALDB (badpixfile = ) Bad pixel file for the observation (dafile = CALDB) Dead area file (input to mkwarf) (bkgresp = yes) Create background ARF and RMF? (weight = yes) Should response files be weighted? (weight_rmf = no) Should RMF also be weighted? (refcoord = ) RA and Dec of responses? (correctpsf = no) Apply point source aperture correction to ARF? (combine = no) Combine ungrouped output spectra and responses? (grouptype = NUM_CTS) Spectrum grouping type (same as grouptype in dmgroup) (binspec = 15) Spectrum grouping specification (NONE,1:1024:10,etc) (bkg_grouptype = NONE) Background spectrum grouping type (NONE, BIN, SNR, NUM_BINS, NUM_CTS, or ADAPTIVE) (bkg_binspec = ) Background spectrum grouping specification (NONE,10,etc) (energy = 0.3:11.0:0.01) Energy grid (channel = 1:1024:1) RMF binning attributes (energy_wmap = 300:2000) Energy range for (dmextract) WMAP input to mkacisrmf (binwmap = tdet=8) Binning factor for (dmextract) WMAP input to mkacisrmf (binarfwmap = 1) Binning factor for (sky2tdet) WMAP input to mkwarf (clobber = no) OK to overwrite existing output file? (verbose = 1) Debug Level(0-5) (mode = ql)
Parameters for /home/username/cxcds_param/dmextract.par infile = acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)][bin pi] Input event file outfile = simple_steps_bkg.pi Enter output file name (bkg = ) Background region file or fixed background (counts/pixel/s) subtraction (error = gaussian) Method for error determination(gaussian|gehrels|<variance file>) (bkgerror = gaussian) Method for background error determination(gaussian|gehrels|<variance file>) (bkgnorm = 1.0) Background normalization (exp = ) Exposure map image file (bkgexp = ) Background exposure map image file (sys_err = 0) Fixed systematic error value for SYS_ERR keyword (opt = pha1) Output file type (defaults = ${ASCDS_CALIB}/cxo.mdb -> /soft/ciao/data/cxo.mdb) Instrument defaults file (wmap = [energy=300:2000][bin tdet=8]) WMAP filter/binning (e.g. det=8 or default) (clobber = no) OK to overwrite existing output file(s)? (verbose = 0) Verbosity level (mode = ql)
Parameters for /home/username/cxcds_param/asphist.par infile = @acisf00869_asol1.lis Aspect Solution List Files outfile = simple_steps.asphist Aspect Histogram Output File evtfile = acisf00869_repro_evt2.fits[ccd_id=7] Event List Files dtffile = Live Time Correction List Files for HRC (geompar = geom) Parameter file for Pixlib Geometry files (res_xy = 0.5) Aspect Resolution x and y in arcsec (res_roll = 600.) Aspect Resolution roll in arcsec (max_bin = 10000.) Maximal number of bins (clobber = no) Clobber output (verbose = 0) Verbose (mode = ql)
Parameters for /home/username/cxcds_param/sky2tdet.par infile = acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)][energy=300:2000][bin sky=1] Input image in sky (x,y) coordinates asphistfile = simple_steps.asphist Input aspect histogram file outfile = simple_steps_bkg_tdet.fits[wmap] Output TDET WMAP file (bin = 1) Binning factor (geompar = geom) Pixlib geometry file (verbose = 0) Verbosity (clobber = no) Remove existing files? (mode = ql)
Parameters for /home/username/cxcds_param/mkwarf.par infile = simple_steps_bkg_tdet.fits[wmap] Input detector WMAP outfile = simple_steps_bkg.arf Output weighted ARF file weightfile = simple_steps_bkg.wfef Output FEF weights spectrumfile = Input Spectral weighting file (<filename>|NONE) egridspec = 0.3:11.0:0.01 Output energy grid [kev] pbkfile = Parameter block file (threshold = 0) Percent threshold cut for FEF regions (feffile = CALDB) FEF file (mskfile = acisf00869_000N005_msk1.fits) Mask file (asolfile = ) Stack of aspect solution files (mirror = HRMA) ARDLIB Mirror specification (detsubsysmod = ) Detector sybsystem modifier (dafile = CALDB) Dead area file (ardlibpar = ardlib) Parameter file for ARDLIB files (geompar = geom) Parameter file for Pixlib Geometry files (clobber = no) Clobber existing outputs (verbose = 0) Tool chatter level (mode = ql)
Parameters for /home/username/cxcds_param/mkacisrmf.par infile = CALDB scatter/rsp matrix file outfile = simple_steps_bkg_mkacisrmf.rmf RMF output file wmap = simple_steps_bkg.pi[WMAP] WMAP file energy = 0.3:11.0:0.01 energy grid in keV (lo:hi:bin) channel = 1:1024:1 channel grids in pixel (min:max:bin) chantype = PI channel type ccd_id = filter CCD-ID chipx = filter chipx in pixel chipy = filter chipy in pixel gain = CALDB gain file (asolfile = ) aspect solution file or a stack of asol files (obsfile = )wmap -> simple_steps_bkg.pi[WMAP]) obs file (logfile = ) log file (contlvl = 100) # contour level (geompar = geom) pixlib geometry parameter file (thresh = 1e-06) low threshold of energy cut-off probability (clobber = no) overwrite existing output file (yes|no)? (verbose = 0) verbosity level (0 = no display) (mode = ql)
Parameters for /home/username/cxcds_param/mkrmf.par infile = CALDB name of FEF input file outfile = simple_steps_bkg_mkrmf.rmf name of RMF output file axis1 = energy=0:1 axis-1(name=lo:hi:btype) axis2 = pi=1:1024:1 axis-2(name=lo:hi:btype) (logfile = STDOUT) name of log file (weights = simple_steps_bkg.wfef) name of weight file (thresh = 1e-5) low threshold of energy cut-off probability (outfmt = legacy) RMF output format (legacy|cxc) (clobber = no) overwrite existing output file (yes|no)? (verbose = 0) verbosity level (0 = no display) (axis3 = none) axis-3(name=lo:hi:btype) (axis4 = none) axis-4(name=lo:hi:btype) (axis5 = none) axis-5(name=lo:hi:btype) (mode = ql)
Parameters for /home/username/cxcds_param/dmgroup.par infile = simple_steps.pi Input dataset name outfile = simple_steps_grp.pi Output dataset name grouptype = NUM_CTS Grouping type grouptypeval = 15 Grouping type value binspec = Binning specification xcolumn = channel Name of x-axis ycolumn = counts Name of y-axis (tabspec = ) Tab specification (tabcolumn = ) Name of tab column (stopspec = ) Stop specification (stopcolumn = ) Name of stop column (errcolumn = ) Name of error column (clobber = no) Clobber existing output file? (verbose = 0) Verbosity level (maxlength = 0) Maximum size of groups (in channels) (mode = ql)
Parameters for /home/username/cxcds_param/specextract.par infile = @multi_src.lis Source event file(s) outroot = spec/1842,spec/1843 Output directory path + root name for output files (bkgfile = @multi_bg.lis) Background event file(s) (asp = ) Source aspect solution or histogram file(s) (dtffile = ) Input DTF files for HRC observations (mskfile = ) Maskfile (input to mkwarf) (rmffile = CALDB) rmffile input for CALDB (badpixfile = ) Bad pixel file for the observation (dafile = CALDB) Dead area file (input to mkwarf) (bkgresp = yes) Create background ARF and RMF? (weight = yes) Should response files be weighted? (weight_rmf = no) Should RMF also be weighted? (refcoord = ) RA and Dec of responses? (correctpsf = no) Apply point source aperture correction to ARF? (combine = no) Combine ungrouped output spectra and responses? (grouptype = NUM_CTS) Spectrum grouping type (same as grouptype in dmgroup) (binspec = 15) Spectrum grouping specification (NONE,1:1024:10,etc) (bkg_grouptype = NONE) Background spectrum grouping type (NONE, BIN, SNR, NUM_BINS, NUM_CTS, or ADAPTIVE) (bkg_binspec = ) Background spectrum grouping specification (NONE,10,etc) (energy = 0.3:11.0:0.01) Energy grid (channel = 1:1024:1) RMF binning attributes (energy_wmap = 300:2000) Energy range for (dmextract) WMAP input to mkacisrmf (binwmap = tdet=8) Binning factor for (dmextract) WMAP input to mkacisrmf (binarfwmap = 1) Binning factor for (sky2tdet) WMAP input to mkwarf (clobber = no) OK to overwrite existing output file? (verbose = 1) Debug Level(0-5) (mode = ql)
History
01 Feb 2006 | new for CIAO 3.3 |
15 Feb 2006 | created Running mkacisrmf Independently section |
31 Mar 2006 | specextract use update added to Overview |
05 Apr 2006 | In light of the specextract usage change, the thread has been rewritten to use extended sources in the examples |
14 Apr 2006 | added Analysis at the edges of ACIS CCDs caveat |
24 May 2006 | added new information to Using the ACIS "Blank-Sky" Background Files caveat |
14 Jun 2006 | corrected link in "Calibration Updates"; clarified information on GRADED mode data |
18 Dec 2006 | updated for CIAO 3.4: new calibration files in CALDB 3.3.0; Extracting Multiple Spectra section uses a stack of output file roots (new feature in CIAO 3.4); output files in one-output case no longer have "src1" or "bkg1" included in the filename; mkrmf no longer prints messages at verbose=0; CIAO version in warnings |
06 Mar 2007 | added ACIS dead area correction section |
22 Jan 2008 | updated for CIAO 4.0: ACIS dead area correction parameters added to the specextract.par file: pbkfile and dafile (dead area correction is turned on by default); new ACIS blank-sky background file in CALDB 3.4.0 eliminate the header keyword issue; available links point to the Sherpa Beta website; removed outdated calibration updates |
31 Mar 2008 | updated for CALDB 3.4.3: use mkacisrmf for -110 BI chips if TGAIN calibration has been applied |
26 Jun 2008 | updated Analysis at the edges of ACIS CCDs caveats (aspect information not taken into account by specextract) |
04 Feb 2009 | updated for CIAO 4.1: images are inline; Sherpa link updated to 4.1 website; screen output changed with CALDB 4; input data must have a CTI_APP keyword |
17 Feb 2009 | added "for Extended Sources" to the title |
12 Jan 2010 | updated for CIAO 4.2: specextract uses a CALDB query to decide which RMF tool should be used; calibration update - the ACIS QE contamination model has been upgraded to vN0005. |
05 Mar 2010 | added additional information to the Choosing a background file section |
09 Mar 2010 | The ACIS detector is calibrated over the range 0.224004 - 12 keV; choosing values outside this range will result in errors from specextract. |
22 Mar 2010 | added "Special case: -110 C data on a front-illuminated chip" to the Creating RMFs: mkrmf vs mkacisrmf section |
15 Dec 2010 | updated for CIAO 4.3: new ACIS contamination calibration file; specextract is part of the scripts package; the script has been updated to run the sky2tdet tool when creating weighted responses, so an aspect histogram must be provided; the mask file is needed as input for mkwarf |
22 Dec 2010 | specextract bug fixes released in 22 Dec 2010 scripts package update |
25 Feb 2011 | updated for 25 Feb scripts package release: the asp parameter can take aspect solution files directly (don't have to run asphist make an aspect histogram first); if the bad pixel file is supplied in the badpixfile parameter, the ardlib parameters are set by the script |
01 Mar 2011 | CALDB 4.4.2 release: fix to the header of the ACIS QE contamination file. Prior to this release, CIAO would fail when trying to look up the contamination model correction for chips ACIS-8 (S4) and ACIS-9 (S5). |
04 Apr 2011 | updated for 04 Apr scripts package release: acis_fef_lookup script prints the version at verbose > 0. specextract: user is prompted for the bkgfile parameter (previously was hidden); new parameter, bkgresp, determines whether a background ARF and RMF should be created; additional changes are outlined in detail in the script release notes for 04 Apr 2011. |
12 Apr 2011 | updated for 12 Apr scripts package release: specextract bug fix - regions may be specified on the command line or in a file, i.e. "sky=circle(344,435,10)" vs. "sky=region(src.reg)". |
26 Apr 2011 | install version 2 of the tools package for CIAO 4.3 to fix the mkrmf bug; updated for 25 Apr scripts package release: specextract runs the remove_extra_time_keywords script to work around a sky2tdet bug |
21 Jun 2011 | added a "step-by-step" section |
06 Jul 2011 | specextract bug fixes were released in the 07 July 2011 scripts package; required software updates are listed in Synopsis |
06 Sep 2011 | a specextract bug fix was released in the 06 Sep 2011 scripts package |
06 Oct 2011 | a specextract bug fix and a workaround for a known CIAO bug were released in the 06 Oct 2011 scripts package |
15 Dec 2011 | reviewed for CIAO 4.4: the remove_extra_time_keywords script is no longer needed, since the sky2tdet bug is fixed in CIAO 4.4 |
16 Feb 2012 | a specextract bug fix was released in the 16 Feb 2012 scripts package: the script will not error out if you set outroot to be a stack and have correct=yes and weight=no. |
03 Dec 2012 | Review for CIAO 4.5; minor formatting |
08 Aug 2013 | Updated for the contributed scripts 4.5.4 release: the "ancillary" files - e.g. aspect solution and bad-pixel files - can now be picked up from information in the event file (in many cases). The Extracting Multiple Spectra section has also been updated. |
12 Dec 2013 | Review for CIAO 4.6: by default weighted RMF files are no-longer generated; the pbkfile parameter has been removed from specextract and is no-longer used by tools like mkwarf; the response files are now always named with the .rmf and .arf suffix, even when weighted; the multiple spectra section now uses reproject_obs to align the observations before source extraction; although not shown in this thread, specextract can now be used with HRC-I data sets. |
17 Dec 2014 | Reviewed for CIAO 4.7; removed "New in CIAO4.6" sections. Minor edits. |
08 201 2019 | Updated sherpa plot to use matplotlib. |
18 Jan 2022 | Review for CIAO 4.14. Updated for Repro5 (coordinates) and CALDB 4.9.6. |