Last modified: 18 Jan 2022

URL: https://cxc.cfa.harvard.edu/ciao/threads/extended/

Extract Spectrum and Response Files for an Extended Source

CIAO 4.16 Science Threads


Overview

Synopsis:

Using a combination of CIAO tools, we extract source and background spectra for an extended source. The background spectrum is grouped, if desired. The appropriate Response Matrix Files (RMFs) and Ancillary Response Files (ARFs) are created for both source and background.

The specextract script automates these steps for extended and pointlike sources observed with the ACIS detector.

[CAUTION]
Warm ACIS observations taken after 2022-11-01

Due to on going thermal challenges, some ACIS observations will be taken with the focal plane temperature, FP_TEMP, warmer than the currently calibrated high temperature limit of -109 C (164.15 K). Use the following command to get the median FP_TEMP in degrees Kelvin for the observation:

unix% dmkeypar acis_repro_evt2.fits FP_TEMP echo+
170.2

Users fitting ACIS imaging spectra that fulfill all the following three conditions may see line-centroids shifted by 1-2% from systematic offsets due to calibration uncertainty:

  1. Spectra with more than 1000 counts on a front-illuminated (FI) chip or more than 2000 counts on a back-illuminated (BI) chip (CCD_ID 5,7).
  2. More than half of source counts were acquired above the current temperature limit of -109 C.
  3. More than half of source counts are in emission or absorption lines.

Under these conditions, users may benefit from trying to identify time periods when the focal plane temperature was within the currently calibrated range, below -109C, using the following thread: Removing Warm ACIS Data.

Purpose:

To generate source and background spectra of an extended ACIS source and build the proper RMFs and ARFs.

Related Links:

Last Update: 18 Jan 2022 - Review for CIAO 4.14. Updated for Repro5 (coordinates) and CALDB 4.9.6.


Contents


Get Started

Download the sample data: 869 (ACIS-S, ARP 220)

Re-process it with chandra_repro:

unix% cd 869
unix% punlearn chandra_repro
unix% chandra_repro mode=h
Output from acis_process_events:
Output from acis_process_events:
# acis_process_events (CIAO): The following error occurred 81 times:
	dsAPEPULSEHEIGHTERR -- WARNING: pulse height is less than split threshold when performing serial CTI adjustment.

As only a small number of events are reported, the warning can be ignored.

The files we are going to use are in the repro/ directory, shown in bold (note that only the event file needs to be explicitly specified since the remaining files can be found automatically by specextract):

unix% cd repro/
unix% ls
acisf00869_000N005_bpix1.fits   acisf00869_asol1.lis
acisf00869_000N005_fov1.fits    acisf00869_repro_bpix1.fits
acisf00869_000N005_msk1.fits    acisf00869_repro_evt2.fits
acisf00869_000N005_mtl1.fits    acisf078247287N005_pbk0.fits
acisf00869_000N005_stat1.fits   pcadf00869_000N001_asol1.fits

How specextract Works

specextract runs the following tools for the extended source case:

  • dmextract: to extract source and (optionally) background spectra. This tool also creates the WMAP used as input to mkacisrmf.
  • sky2tdet: to create the WMAP input for mkwarf.
  • mkwarf: to create weighted ARF(s).
  • mkrmf or mkacisrmf: to build the RMF(s), depending on which is appropriate for the data and the calibration; see the Creating ACIS RMFs why topic for details.
  • dmgroup: to group the source spectrum and/or background spectrum.
  • dmhedit: to update the BACKFILE, RESPFILE and ANCRFILE keys in the source and background spectrum files.

Responses

Weighted ARFs are generated by default, since weight=yes. For point-source analysis, or when you are not interested in the spatial variation of the effective area, unweighted ARFs can be calculated by setting weight=no.

RMFs are - by default - generated without any weighting, since this can considerably speed up the running of the script and - for most cases - does not significantly change the result, since the RMF variation with position is small. If you do want to create weighted RMF files, then set weight_rmf=yes and weight=yes.

Users can also specify a location at which to calculate unweighted responses - using the refcoord parameter - rather than using the location of the source and background regions.


Build Source and Background Regions

We need to define two regions, one for the source and another for the background. To do this, first display the file:

unix% ds9 acisf00869_repro_evt2.fits &

Refer to the Using CIAO Regions thread for information on creating region files. The CIAO-format files for this example are:

unix% cat simple.reg 
# Region file format: CIAO version 1.0
ellipse(4026.2341,4125.3693,50,40,0)

unix% cat simple_bkg.reg
# Region file format: CIAO version 1.0
circle(3955.5,4252.5,50)

The regions are shown displayed on the event file in Figure 1.

Figure 1: Extraction regions on the event file

[Thumbnail image: The source region and the background region are each a single ellipse.]

[Version: full-size]

[Print media version: The source region and the background region are each a single ellipse.]

Figure 1: Extraction regions on the event file

unix% ds9 acisf00869_repro_evt2.fits -bin filter 'energy=500:7000' \
    -bin about 4025 4140 -cmap b -scale log -region simple.reg \
    -region simple_bkg.reg

The background was chosen from a source-free area of the same chip for this example, but it may also be chosen from a different chip or different event file.

Choosing a background file

The examples in this thread use a background region that was chosen from a source-free area of the same chip as the source region. The source or field of sources may cover the chip, however, making it impossible to select a local background. There are several options in this case:

  • Choose the background from another same-type chip that was on in the observation. If your source is on ACIS-S3 and ACIS-S1 (the other back-illuminated chip) was also turned on, define the background on ACIS-S1.

  • If using another chip from the observation is not an option, use one of the ACIS "blank-Sky" background files which are distributed in the CALDB.

    If you plan on using one of the ACIS "blank-sky" background files from the CALDB and intend to subtract the background spectrum (i.e. not fit it), then you do not need to create a background RMF and ARF. Set the bkgresp parameter to "no" and the script will only extract a background spectrum.

  • Omit the background completely (i.e. leave the bkgfile blank).

Using specextract

1. Set the input parameters

specextract can determine the ancillary files (aspect solution, bad pixel, mask, DTF if HRC data) from the header of the event file, which means that you only need give the source file, output directory, and - if desired - the background file:

unix% punlearn specextract
unix% pset specextract infile="acisf00869_repro_evt2.fits[sky=region(simple.reg)]"
unix% pset specextract outroot=simple
unix% pset specextract bkgfile="acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)]"

2. Run the script

Running the tool is then just (the lines in bold indicate the ancillary files that have been picked up automatically):

unix% specextract mode=h
Running specextract
Version: 18 November 2021

Aspect solution file repro/pcadf00869_000N001_asol1.fits found.

Bad pixel file repro/acisf00869_repro_bpix1.fits found.

Mask file repro/acisf00869_000N005_msk1.fits found.

Setting bad pixel file 

Setting bad pixel file 

Extracting src spectra 

Extracting bkg spectra 

Using mkacisrmf...

Using mkacisrmf...

Creating bkg ARF 

Using mkacisrmf...

Creating bkg RMF 

Creating src ARF 

Using mkacisrmf...

Creating src RMF 

Grouping src spectrum 

Updating header of simple.pi with RESPFILE and ANCRFILE keywords.

Updating header of simple_grp.pi with RESPFILE and ANCRFILE keywords.

Updating header of simple.pi with BACKFILE keyword.

Updating header of simple_grp.pi with BACKFILE keyword.

Updating header of simple_bkg.pi with RESPFILE and ANCRFILE keywords.

The contents of the parameter file may be checked with plist specextract.

[NOTE]
Note

If you want more details on what the script is doing, set verbose=2 before running specextract.

Ancillary files

If you want to explicitly give the location of the ancillary files then, in this case, you would add the following before the call to specextract (note that the asp parameter accepts a stack for those observations with multiple aspect solution files):

unix% pset specextract asp=pcadf00869_000N001_asol1.fits

or

unix% pset specextract asp=@acisf00869_asol1.lis

and for the other files:

unix% pset specextract mskfile=acisf00869_000N005_msk1.fits
unix% pset specextract badpixfile=acisf00869_repro_bpix1.fits

Background responses

The above call created an ARF and RMF for the background region; if you only intend to subtract the background, rather than fitting it, you do not need these responses so can save time (particularly if the background area is large) by setting bkgresp=no when running specextract.

Grouping the output spectra

We choose to use the default grouping values: the source spectrum will be grouped to a minimum number of 15 counts per new channel and the background spectrum will be ungrouped. The grouptype and binspec parameters are used to specify the source spectrum grouping, and the bkg_grouptype and bkg_binspec parameters specify the background spectrum grouping.

Setting the energy range for the WMAP

The energy_wmap parameter determines the energy band used to create the weight map when weight=yes.

If you use the soft band (e.g. the default value of 300:2000 eV), then you are weighting by the emission from the source (unless you have a very hard spectrum). To weight by the effective area of the telescope (again, unless you have a hard source), set the range to, for example, 2000:7000 eV.

Unless the spectrum of your source is unusual, adjusting the WMAP energy range will not affect the ARF - or, therefore, the spectral fit - noticeably.


3. Examining the Output Files

The number of files created depends on if a background event file was provided and if source and/or background grouping was specified. In this case, the output files are:

Source:

File Description
simple.pi ungrouped spectrum
simple.arf ARF (weighted)
simple.rmf RMF (weighted)
simple_grp.pi grouped spectrum

Background:

File Description
simple_bkg.pi ungrouped spectrum
simple_bkg.arf weighted ARF
simple_bkg.rmf weighted RMF

Step-by-Step Guide

This section explains how to manually run the analysis steps that are used by specextract.

Make sure that you have set up ardlib to use the bad pixel file for your observation before following this section of the thread.

1. Extract Source and Background Spectra

First, we extract spectra from the events enclosed by the source and background regions.

To create a WMAP block in a PHA file, dmextract must be given a binning specification for its wmap option. This WMAP will be used as input to mkacisrmf. It must be in detector coordinates, and the suggested binning factor is 8; an optional energy filter is included for the WMAP.

unix% punlearn dmextract
unix% pset dmextract infile="acisf00869_repro_evt2.fits[sky=region(simple.reg)][bin pi]"
unix% pset dmextract outfile=simple_steps.pi
unix% pset dmextract wmap="[energy=300:2000][bin tdet=8]"
unix% dmextract mode=h

And for the background spectrum:

unix% pset dmextract infile="acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)][bin pi]"
unix% pset dmextract outfile=simple_steps_bkg.pi
unix% dmextract mode=h

You can check the parameter file that was used with plist dmextract.


2. Calculate the ARFs

Compute the Aspect Histogram (asphist)

We now need to create the aspect histogram, which is a binned representation of aspect motion during the observation.

In few cases, there will be more than one aspect solution file (pcad_asol1.fits) for an observation. All the files must be input to the infile parameter, either as a list or as a stack. Here we use the stack file generated by chandra_repro:

unix% cat acisf00869_asol1.lis
/data/ciao/869/primary/pcadf00869_000N001_asol1.fits

We also need to know what chip the source and background regions are on:

unix% dmstat "acisf00869_repro_evt2.fits[sky=region(simple.reg)][cols ccd_id]"
ccd_id
    min:	7 	      @:	1 
    max:	7 	      @:	1 
   mean:	7 
  sigma:	0 
    sum:	17906 
   good:	2558 
   null:	0 
   
unix% dmstat "acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)][cols ccd_id]"
ccd_id
    min:	7 	      @:	1 
    max:	7 	      @:	1 
   mean:	7 
  sigma:	0 
    sum:	10276 
   good:	1468 
   null:	0 

Each region is on ccd_id=7. This means we only need to make one aspect histogram; if the regions were on different chips, we would make one for each chip. Now run the tool:

unix% punlearn asphist
unix% pset asphist infile=@acisf00869_asol1.lis
unix% pset asphist outfile=simple_steps.asphist
unix% pset asphist evtfile="acisf00869_repro_evt2.fits[ccd_id=7]"
unix% asphist mode=h

The contents of the parameter file may be checked with plist asphist.

Create a WMAP for the ARF (sky2tdet)

The WMAP created by sky2tdet properly weights the ARF based on how much of the source flux fell onto the bad pixels, columns, or a node boundary, and which bad pixels are actually exposed. Without accounting for these effects, the ARF is over-estimated.

unix% punlearn sky2tdet
unix% pset sky2tdet infile="acisf00869_repro_evt2.fits[sky=region(simple.reg)][energy=300:2000][bin sky=1]"
unix% pset sky2tdet asphistfile="simple_steps.asphist"
unix% pset sky2tdet outfile="simple_steps_tdet.fits[wmap]"
unix% sky2tdet mode=h

If you made a separate aspect histogram for the background, set it in the asphistfile parameter before running the tool for the background spectrum.

unix% pset sky2tdet infile="acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)][energy=300:2000][bin sky=1]"
unix% pset sky2tdet outfile="simple_steps_bkg_tdet.fits[wmap]"
unix% sky2tdet mode=h

You can check the parameter file that was used with plist sky2tdet.

Compute the ARFs (mkwarf)

mkwarf creates a weighted ARF file based on the WMAP that was created with sky2tdet. It also produces a set of weights that are used by mkrmf to create the associated weighted RMF.

Run mkwarf for the source:

unix% punlearn mkwarf
unix% pset mkwarf infile="simple_steps_tdet.fits[wmap]"
unix% pset mkwarf outfile=simple_steps.arf
unix% pset mkwarf weightfile=simple_steps.wfef
unix% pset mkwarf egridspec=0.3:11.0:0.01 
unix% pset mkwarf mskfile=acisf00869_000N005_msk1.fits
unix% mkwarf mode=h

and for the background:

unix% pset mkwarf infile="simple_steps_bkg_tdet.fits[wmap]"
unix% pset mkwarf outfile=simple_steps_bkg.arf
unix% pset mkwarf weightfile=simple_steps_bkg.wfef
unix% mkwarf mode=h

You can check the parameter file that was used with plist mkwarf.


3. Calculate the RMFs

The observation used in this thread (ObsID 869) was taken at the -120 C focal plane temperature. Therefore, mkacisrmf should be used to create the RMF file.

The syntax for both mkacisrmf and mkrmf are given in this section. Users should choose the appropriate tool for the data and calibration.

Using mkacisrmf

The mkacisrmf why topic has more details on using the mkacisrmf tool.

The RMF doesn't change quickly enough to require the detail of the sky2tdet WMAP, and its precision will cause mkacisrmf to run very slowly. Nearly identical results are obtained with the dmextract WMAP with a more reasonable runtime.

For the source:

unix% punlearn mkacisrmf
unix% pset mkacisrmf infile=CALDB
unix% pset mkacisrmf outfile=simple_steps_mkacisrmf.rmf
unix% pset mkacisrmf energy=0.3:11.0:0.01
unix% pset mkacisrmf channel=1:1024:1
unix% pset mkacisrmf wmap="simple_steps.pi[WMAP]"
unix% mkacisrmf mode=h


Total 18 regions to be processed:
    1> reg# 1263  processed
    2> reg# 1264  processed
    3> reg# 1265  processed
    4> reg# 1266  processed
    5> reg# 1294  processed
    6> reg# 1295  processed
    7> reg# 1296  processed
    8> reg# 1297  processed
    9> reg# 1298  processed
   10> reg# 1326  processed
   11> reg# 1327  processed
   12> reg# 1328  processed
   13> reg# 1329  processed
   14> reg# 1330  processed
   15> reg# 1359  processed
   16> reg# 1360  processed
   17> reg# 1361  processed
   18> reg# 1391  processed

For the background, repeat the command with the background spectrum:

unix% pset mkacisrmf outfile=simple_steps_bkg_mkacisrmf.rmf
unix% pset mkacisrmf wmap="simple_steps_bkg.pi[WMAP]"
unix% mkacisrmf mode=h


Total 17 regions to be processed:
    1> reg# 1393  processed
    2> reg# 1394  processed
    3> reg# 1395  processed
    4> reg# 1424  processed
    5> reg# 1425  processed
    6> reg# 1426  processed
    7> reg# 1427  processed
    8> reg# 1456  processed
    9> reg# 1457  processed
   10> reg# 1458  processed
   11> reg# 1459  processed
   12> reg# 1488  processed
   13> reg# 1489  processed
   14> reg# 1490  processed
   15> reg# 1491  processed
   16> reg# 1521  processed
   17> reg# 1522  processed

You can check the parameter file that was used with plist mkacisrmf.

You can now skip to the Update the Spectrum Files section.

Using mkrmf

The mkrmf tool uses the weightfile from mkwarf to create the weighted RMF. The tool queries the CALDB for the correct FEF file and the energy axis grid is ignored since the grid is read from the weights file. Note that although the axis1 parameter is ignored, a syntactically-correct value must be entered; this example uses axis1="energy=0:1".

For the source:

unix% punlearn mkrmf
unix% pset mkrmf infile=CALDB
unix% pset mkrmf outfile=simple_steps_mkrmf.rmf
unix% pset mkrmf axis1="energy=0:1"
unix% pset mkrmf axis2="pi=1:1024:1"
unix% pset mkrmf weights=simple_steps.wfef
unix% mkrmf mode=h

and for the background:

unix% pset mkrmf outfile=simple_steps_bkg_mkrmf.rmf
unix% pset mkrmf weights=simple_steps_bkg.wfef
unix% mkrmf mode=h

You can check the parameter file that was used with plist mkrmf.


4. Update the Spectrum Files

Group the Source Spectrum (dmgroup)

Sherpa has dynamic grouping capabilities so you can easily set or change the grouping of a dataset at the Sherpa prompt. For other data analysis packages it can be useful to create a grouped version of the spectrum as a seprate file.

The source spectrum is grouped to have a minimum number of 15 counts per new channel:

unix% punlearn dmgroup
unix% pset dmgroup infile=simple_steps.pi
unix% pset dmgroup outfile=simple_steps_grp.pi
unix% pset dmgroup grouptype=NUM_CTS 
unix% pset dmgroup grouptypeval=15
unix% pset dmgroup xcolumn=channel
unix% pset dmgroup ycolumn=counts
unix% dmgroup mode=h

You can check the parameter file that was used with plist dmgroup.

Update File Headers (dmhedit)

Finally, update the appropriate header keywords in the both the ungrouped and grouped source files.

Note that the ungrouped background file name is used as the BACKFILE header keyword value. The source grouping is applied to the background grouping when fitting in Sherpa. For fitting only background data, or simultaneous fitting of source and background data, Sherpa can group background dynamically with the group functions; see the Fitting section of this thread for more information.

unix% punlearn dmhedit
unix% pset dmhedit operation=add filelist="" mode=h
unix% dmhedit simple_steps.pi key=BACKFILE value=simple_steps_bkg.pi
unix% dmhedit simple_steps.pi key=RESPFILE value=simple_steps_mkacisrmf.rmf
unix% dmhedit simple_steps.pi key=ANCRFILE value=simple_steps_corr.arf
unix% dmhedit simple_steps_grp.pi key=BACKFILE value=simple_steps_bkg.pi
unix% dmhedit simple_steps_grp.pi key=RESPFILE value=simple_steps_mkacisrmf.rmf
unix% dmhedit simple_steps_grp.pi key=ANCRFILE value=simple_steps.arf

And for the ungrouped background file:

unix% dmhedit simple_steps_bkg.pi operation=add key=RESPFILE value=simple_steps_bkg_mkacisrmf.rmf
unix% dmhedit simple_steps_bkg.pi operation=add key=ANCRFILE value=simple_steps_bkg.arf

Extracting Multiple Spectra

In this example, we show how specextract can create multiple output spectra from a single run of the script.

Build Source Regions and Background Regions

This example uses two observations of G21.5-09. Both observations will be processed by specextract at the same time, producing two sets of output files; this is explained further in the Run specextract section.

We download and reprocess the data:

unix% download_chandra_obsid 1842,1843
...
unix% punlearn chandra_repro
unix% chandra_repro indir=1842,1843 mode=h

and then use reproject_obs to reproject the data sets to a common tangent-plane position (i.e. so that the SKY coordinate system is the same in the two event files).

unix% punlearn reproject_obs
unix% reproject_obs 1842,1843 reproj/
Running reproject_obs
Version: 15 November 2021

Found 1842/repro/acisf01842_repro_evt2.fits
Found 1843/repro/acisf01843_repro_evt2.fits
Verifying 2 observations.
Calculating new tangent point.
New tangent point: RA=18h 32m 21.347s Dec=-10d 34' 38.44"

Observations to be reprojected:

  Obsid  Obs Date   Exp    DETNAM     SIM_Z    FP   Sepn   PA  
                   (ks)                (mm)    (K)   (')  (deg)
---------------------------------------------------------------
1 1843  2000-09-02   7.9 ACIS-012367 -245.857 153.4   2.6   -79
2 1842  2000-09-02   7.4 ACIS-012367 -231.239 153.4   2.6  +101

Running tasks in parallel with 4 processors.
Reprojecting 2 event files to a common tangent point.
Merging reprojected events files to multi_reproj/merged_evt.fits

The following files were created:

The reprojected FOV files:
     multi_reproj/1843.fov
     multi_reproj/1842.fov

The combined FOV file:
     multi_reproj/merged.fov

The reprojected event files:
     multi_reproj/1843_reproj_evt.fits
     multi_reproj/1842_reproj_evt.fits

The merged event file:
     multi_reproj/merged_evt.fits

Warning: the merged event file multi_reproj/merged_evt.fits
   should not be used to create ARF/RMF/exposure maps because
      the RA_NOM keyword varies by 0.0851 (limit is 0.0003)
      the DEC_NOM keyword varies by 0.0159 (limit is 0.0003)
      the ROLL_NOM keyword varies by 2.5 (limit is 1.0)
      the SIM_Z keyword varies by 14.6 (limit is 0.1)

As the screen output describes, the merged event file - in this case reproj/merged_eve.fits - should not be used for spectral analysis.

We have defined a source and background region for each observation (so that different radii and, for the backgrounds, centers can be used):

unix% cat 1842_src.reg
# Region file format: CIAO version 1.0
circle(1943.3,4164.2,102)
unix% cat 1843_src.reg
# Region file format: CIAO version 1.0
circle(1943.3,4164.2,135)
unix% cat 1842_bg.reg
# Region file format: CIAO version 1.0
circle(2259.3,4072.3,100)
unix% cat 1843_bg.reg
# Region file format: CIAO version 1.0
circle(2361.5,4104.8,140)

The regions are shown displayed on the event files in Figure 2.

Figure 2: Extraction regions on the event files

[Thumbnail image: The two event files used for spectral extraction are displayed in ds9 with the source and background regions overlaid.]

[Version: full-size]

[Print media version: The two event files used for spectral extraction are displayed in ds9 with the source and background regions overlaid.]

Figure 2: Extraction regions on the event files

unix% ds9 reproj/1842_reproj_evt.fits -bin filter 'energy=500:7000' \
  -bin about 2000 4150 -bin factor 2 -cmap b -region 1842_src.reg -region 1842_bg.reg \
  reproj/1843_reproj_evt.fits -bin filter 'energy=500:7000' \
  -bin about 2000 4150 -bin factor 2 -cmap b -region 1843_src.reg \
  -region 1843_bg.reg -tile row

ObsID 1842 is displayed in the top frame and ObsID 1843 is displayed in the bottom frame.


Run specextract

The event files are input to the script as a stack; this syntax means that a separate spectrum will be created for each of the file/region pairs (since we set combine=no):

unix% cat multi_src.lis
reproj/1842_reproj_evt.fits[sky=region(1842_src.reg)]
reproj/1843_reproj_evt.fits[sky=region(1843_src.reg)]

When working with stack inputs to specextract, the source and background stacks must contain the same number of items, unless you are not extracting background spectra. Make sure that the background file definitions are in the same order as the source files:

unix% cat multi_bg.lis
reproj/1842_reproj_evt.fits[sky=region(1842_bg.reg)]
reproj/1843_reproj_evt.fits[sky=region(1843_bg.reg)]

Using an external file for a stack - such as multi_src.lis and multi_bg.lis - is useful when either the contents are going to be used multiple times or the contents are long, but you can also use a comma-separated list of values, which is what we do for the stack of output names (the outroot parameter). If you prefer, you may just give a single value for outroot and specextract will create output files designated as "src1", "src2", "bkg1", "bkg2", etc. The default values for combine, weight, and correctpsf (no, yes, and no respectively) are used. In the output below the bold lines show the ancillary files that were selected by the script.

unix% mkdir spec
unix% punlearn specextract
unix% pset specextract infile=@multi_src.lis
unix% pset specextract outroot=spec/1842,spec/1843
unix% pset specextract bkgfile=@multi_bg.lis
unix% specextract mode=h
Running specextract
Version: 18 November 2021

Aspect solution file multi_reproj/1842.asol found.

Aspect solution file multi_reproj/1843.asol found.

Bad pixel file multi_reproj/1842.bpix found.

Bad pixel file multi_reproj/1843.bpix found.

Mask file multi_reproj/1842.mask found.

Mask file multi_reproj/1843.mask found.

Setting bad pixel file [1 of 2]

Setting bad pixel file [2 of 2]

Setting bad pixel file [1 of 2]

Setting bad pixel file [2 of 2]

Extracting src spectra [1 of 2]

Extracting bkg spectra [2 of 2]

Extracting bkg spectra [1 of 2]

Extracting src spectra [2 of 2]

Using mkacisrmf...

Using mkacisrmf...

Using mkacisrmf...

Using mkacisrmf...

Creating bkg ARF [1 of 2]

Using mkacisrmf...

Creating bkg RMF [1 of 2]

Creating bkg ARF [2 of 2]

Using mkacisrmf...

Creating bkg RMF [2 of 2]

Creating src ARF [1 of 2]

Creating src ARF [2 of 2]

Using mkacisrmf...

Creating src RMF [1 of 2]

Using mkacisrmf...

Creating src RMF [2 of 2]

Grouping src spectrum [1 of 2]

Grouping src spectrum [2 of 2]

Updating header of spec/1842.pi with RESPFILE and ANCRFILE keywords.

Updating header of spec/1842_grp.pi with RESPFILE and ANCRFILE keywords.

Updating header of spec/1842.pi with BACKFILE keyword.

Updating header of spec/1842_grp.pi with BACKFILE keyword.

Updating header of spec/1843.pi with RESPFILE and ANCRFILE keywords.

Updating header of spec/1843_grp.pi with RESPFILE and ANCRFILE keywords.

Updating header of spec/1843.pi with BACKFILE keyword.

Updating header of spec/1843_grp.pi with BACKFILE keyword.

Updating header of spec/1842_bkg.pi with RESPFILE and ANCRFILE keywords.

Updating header of spec/1843_bkg.pi with RESPFILE and ANCRFILE keywords.

The contents of the parameter file may be checked with plist specextract.

Ancillary files

If you want to explicitly give the location of the ancillary files then, in this case, you would add the following before the call to specextract. The files should be listed in the same order as the infile paramter. So, for the bad-pixel, mask, and aspect solution files:

unix% cat multi_bpix.lis
repro/1842.bpix
repro/1843.bpix

unix% cat multi_mask.lis
repro/1842.mask
repro/1843.mask

unix% cat multi_asp.lis
repro/1842.asol
repro/1843.asol

Examining the Output Files

The number of files created depends on if a background event file was provided and how the source and background groupings were specified. In this case we have:

Grouped source spectra

unix% ls -1 spec/*grp.pi
spec/1842_grp.pi
spec/1843_grp.pi

Ungrouped source spectra

unix% ls -1 spec/184?.pi
spec/1842.pi
spec/1843.pi

Background spectra

These are ungrouped since the default setting bkg_grouptype=NONE was used:

unix% ls -1 spec/*bkg.pi
spec/1842_bkg.pi
spec/1843_bkg.pi

Responses

These responses are weighted and there are responses for the background spectra, since weight=yes and bkgresp=yes.

unix% ls -1 spec/*arf
spec/1842.arf
spec/1842_bkg.arf
spec/1843.arf
spec/1843_bkg.arf
unix% ls -1 spec/*rmf
spec/1842.rmf
spec/1842_bkg.rmf
spec/1843.rmf
spec/1843_bkg.rmf

Setting outroot to a single value

If the run had used outroot=spec/multi instead then the files would have been labelled as "src1" and "src2"; so the grouped spectra would have been called

unix% ls -1 spec/*grp.pi
spec/multi_src1_grp.pi
spec/multi_src2_grp.pi

Fitting

Below we show the data from the single source example being read into Sherpa and displayed (Figure 3):

unix% sherpa

sherpa In[]: %matplotlib
Using matplotlib backend: TkAgg

sherpa In[]: load_data('simple.pi')
read ARF file simple.warf
read RMF file simple.wrmf
read ARF (background) file simple_bkg.warf
read RMF (background) file simple_bkg.wrmf
read background file simple_bkg.pi
sherpa In[]: notice(0.3, 7)
sherpa In[]: group_counts(20)
sherpa In[]: plot_data()
sherpa In[]: subtract()
sherpa In[]: plot_data(overplot=True)
sherpa In[]: import matplotlib.pylab as plt
sherpa In[]: plt.xscale("log")
sherpa In[]: ply.yscale("log")

Figure 3: Spectrum from ObsId 869

[Thumbnail image: There is plenty of signal to fit!]

[Version: full-size]

[Print media version: There is plenty of signal to fit!]

Figure 3: Spectrum from ObsId 869

The orange curve shows the data after background subtraction whereas the blue curve includes the background contribution.

If you would like to fit the background-subtracted source spectrum using a common RMF and ARF for source and background, simply read the source spectrum FITS file into Sherpa, subtract the background, and fit. See the Introduction to Fitting PHA Spectra thread for details.

To fit source and background spectra simultaneously with distinct RMFs and ARFs, follow the Independent Background Responses thread.

The RESPFILE and ANCRFILE header keywords in the source and background spectra have been updated, as well as the BACKFILE in the source spectra. This means that when the spectra are read into Sherpa, all the supporting files will automatically be read as well; the background (if available) will be defined, as will the source and background response files.


Analysis Caveats

Analysis at the edges of ACIS CCDs

Users should be cautious about analyzing the data for sources near the edges of the ACIS CCDs.

  1. For X-rays passing through the mirrors, the very bottom of each CCD is obscured by the frame store. As a result, some of the events in rows with CHIPY <= 8 are not detected. (The set of rows affected varies from CCD to CCD.) Since the CIAO tools do not compensate for this effect, the ARFs and exposure maps for sources in these regions may be inaccurate.

  2. For sources within about thirty-two pixels of any edge of a CCD, the source may be dithered off the CCD during part of an observation. The aspect histogram, which is used to create ARFs and exposure maps, is designed to compensate for this effect.

  3. An ARF calculated at the edge of a chip will not be accurate since the response tools for spectral extraction (specifically the ARF) assume that 100% of the PSF is enclosed - i.e. on the chip - all the time, which may not be the case. The amount of error introduced depends on how close the source is to the edge, the morphology of the source, and the characteristics of the PSF, which depends on the source spectrum.

  4. A contaminant has accumulated on the optical-blocking filters of the ACIS detectors, as described in the ACIS QE Contamination why topic. Since there is a gradient in the temperature across the filters (the edges are colder), there is a gradient in the amount of material on the filters. (The contaminant is thicker at the edges.) Within about 100 pixels of the outer edges of the ACIS-I and ACIS-S arrays, the gradient is relatively steep. Therefore, the effective low-energy (' 1 keV) detection efficiency may vary within the dither pattern in this region. The ARF and instrument map tools are designed to read a calibration file which describes this spatial dependence.



Parameters for /home/username/cxcds_param/specextract.par


        infile = acisf00869_repro_evt2.fits[sky=region(simple.reg)] Source event file(s)
       outroot = simple           Output directory path + root name for output files
      (bkgfile = acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)]) Background event file(s)
          (asp = )                Source aspect solution or histogram file(s)
      (dtffile = )                Input DTF files for HRC observations
      (mskfile = )                Maskfile (input to mkwarf)
      (rmffile = CALDB)           rmffile input for CALDB
   (badpixfile = )                Bad pixel file for the observation
       (dafile = CALDB)           Dead area file (input to mkwarf)
      (bkgresp = yes)             Create background ARF and RMF?
       (weight = yes)             Should response files be weighted?
   (weight_rmf = no)              Should RMF also be weighted?
     (refcoord = )                RA and Dec of responses?
   (correctpsf = no)              Apply point source aperture correction to ARF?
      (combine = no)              Combine ungrouped output spectra and responses?
    (grouptype = NUM_CTS)         Spectrum grouping type (same as grouptype in dmgroup)
      (binspec = 15)              Spectrum grouping specification (NONE,1:1024:10,etc)
(bkg_grouptype = NONE)            Background spectrum grouping type (NONE, BIN, SNR, NUM_BINS, NUM_CTS, or ADAPTIVE)
  (bkg_binspec = )                Background spectrum grouping specification (NONE,10,etc)
       (energy = 0.3:11.0:0.01)   Energy grid
      (channel = 1:1024:1)        RMF binning attributes
  (energy_wmap = 300:2000)        Energy range for (dmextract) WMAP input to mkacisrmf
      (binwmap = tdet=8)          Binning factor for (dmextract) WMAP input to mkacisrmf
   (binarfwmap = 1)               Binning factor for (sky2tdet) WMAP input to mkwarf
      (clobber = no)              OK to overwrite existing output file?
      (verbose = 1)               Debug Level(0-5)
         (mode = ql)
    


Parameters for /home/username/cxcds_param/dmextract.par


        infile = acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)][bin pi] Input event file
       outfile = simple_steps_bkg.pi Enter output file name
          (bkg = )                Background region file or fixed background (counts/pixel/s) subtraction
        (error = gaussian)        Method for error determination(gaussian|gehrels|<variance file>)
     (bkgerror = gaussian)        Method for background error determination(gaussian|gehrels|<variance file>)
      (bkgnorm = 1.0)             Background normalization
          (exp = )                Exposure map image file
       (bkgexp = )                Background exposure map image file
      (sys_err = 0)               Fixed systematic error value for SYS_ERR keyword
          (opt = pha1)            Output file type
     (defaults = ${ASCDS_CALIB}/cxo.mdb -> /soft/ciao/data/cxo.mdb) Instrument defaults file
         (wmap = [energy=300:2000][bin tdet=8]) WMAP filter/binning (e.g. det=8 or default)
      (clobber = no)              OK to overwrite existing output file(s)?
      (verbose = 0)               Verbosity level
         (mode = ql)
    


Parameters for /home/username/cxcds_param/asphist.par


        infile = @acisf00869_asol1.lis Aspect Solution List Files
       outfile = simple_steps.asphist Aspect Histogram Output File
       evtfile = acisf00869_repro_evt2.fits[ccd_id=7] Event List Files
       dtffile =                  Live Time Correction List Files for HRC
      (geompar = geom)            Parameter file for Pixlib Geometry files
       (res_xy = 0.5)             Aspect Resolution x and y in arcsec
     (res_roll = 600.)            Aspect Resolution roll in arcsec
      (max_bin = 10000.)          Maximal number of bins
      (clobber = no)              Clobber output
      (verbose = 0)               Verbose
         (mode = ql)
    


Parameters for /home/username/cxcds_param/sky2tdet.par


        infile = acisf00869_repro_evt2.fits[sky=region(simple_bkg.reg)][energy=300:2000][bin sky=1] Input image in sky (x,y) coordinates
   asphistfile = simple_steps.asphist Input aspect histogram file
       outfile = simple_steps_bkg_tdet.fits[wmap] Output TDET  WMAP file
          (bin = 1)               Binning factor
      (geompar = geom)            Pixlib geometry file
      (verbose = 0)               Verbosity
      (clobber = no)              Remove existing files?
         (mode = ql)
    


Parameters for /home/username/cxcds_param/mkwarf.par


        infile = simple_steps_bkg_tdet.fits[wmap] Input detector WMAP
       outfile = simple_steps_bkg.arf Output weighted ARF file
    weightfile = simple_steps_bkg.wfef Output FEF weights
  spectrumfile =                  Input Spectral weighting file (<filename>|NONE)
     egridspec = 0.3:11.0:0.01    Output energy grid [kev]
       pbkfile =                  Parameter block file
    (threshold = 0)               Percent threshold cut for FEF regions
      (feffile = CALDB)           FEF file
      (mskfile = acisf00869_000N005_msk1.fits) Mask file
     (asolfile = )                Stack of aspect solution files
       (mirror = HRMA)            ARDLIB Mirror specification
 (detsubsysmod = )                Detector sybsystem modifier
       (dafile = CALDB)           Dead area file
    (ardlibpar = ardlib)          Parameter file for ARDLIB files
      (geompar = geom)            Parameter file for Pixlib Geometry files
      (clobber = no)              Clobber existing outputs
      (verbose = 0)               Tool chatter level
         (mode = ql)
    


Parameters for /home/username/cxcds_param/mkacisrmf.par


        infile = CALDB            scatter/rsp matrix file
       outfile = simple_steps_bkg_mkacisrmf.rmf RMF output file
          wmap = simple_steps_bkg.pi[WMAP] WMAP file
        energy = 0.3:11.0:0.01    energy grid in keV (lo:hi:bin)
       channel = 1:1024:1         channel grids in pixel (min:max:bin)
      chantype = PI               channel type
        ccd_id =                  filter CCD-ID
         chipx =                  filter chipx in pixel
         chipy =                  filter chipy in pixel
          gain = CALDB            gain file
     (asolfile = )                aspect solution file or a stack of asol files
      (obsfile = )wmap -> simple_steps_bkg.pi[WMAP]) obs file
      (logfile = )                log file
      (contlvl = 100)             # contour level
      (geompar = geom)            pixlib geometry parameter file
       (thresh = 1e-06)           low threshold of energy cut-off probability
      (clobber = no)              overwrite existing output file (yes|no)?
      (verbose = 0)               verbosity level (0 = no display)
         (mode = ql)
    


Parameters for /home/username/cxcds_param/mkrmf.par


        infile = CALDB            name of FEF input file
       outfile = simple_steps_bkg_mkrmf.rmf name of RMF output file
         axis1 = energy=0:1       axis-1(name=lo:hi:btype)
         axis2 = pi=1:1024:1      axis-2(name=lo:hi:btype)
      (logfile = STDOUT)          name of log file
      (weights = simple_steps_bkg.wfef) name of weight file
       (thresh = 1e-5)            low threshold of energy cut-off probability
       (outfmt = legacy)          RMF output format (legacy|cxc)
      (clobber = no)              overwrite existing output file (yes|no)?
      (verbose = 0)               verbosity level (0 = no display)
        (axis3 = none)            axis-3(name=lo:hi:btype)
        (axis4 = none)            axis-4(name=lo:hi:btype)
        (axis5 = none)            axis-5(name=lo:hi:btype)
         (mode = ql)
    


Parameters for /home/username/cxcds_param/dmgroup.par


        infile = simple_steps.pi  Input dataset name
       outfile = simple_steps_grp.pi Output dataset name
     grouptype = NUM_CTS          Grouping type
  grouptypeval = 15               Grouping type value
       binspec =                  Binning specification
       xcolumn = channel          Name of x-axis
       ycolumn = counts           Name of y-axis
      (tabspec = )                Tab specification
    (tabcolumn = )                Name of tab column
     (stopspec = )                Stop specification
   (stopcolumn = )                Name of stop column
    (errcolumn = )                Name of error column
      (clobber = no)              Clobber existing output file?
      (verbose = 0)               Verbosity level
    (maxlength = 0)               Maximum size of groups (in channels)
         (mode = ql)
    


Parameters for /home/username/cxcds_param/specextract.par


        infile = @multi_src.lis   Source event file(s)
       outroot = spec/1842,spec/1843 Output directory path + root name for output files
      (bkgfile = @multi_bg.lis)   Background event file(s)
          (asp = )                Source aspect solution or histogram file(s)
      (dtffile = )                Input DTF files for HRC observations
      (mskfile = )                Maskfile (input to mkwarf)
      (rmffile = CALDB)           rmffile input for CALDB
   (badpixfile = )                Bad pixel file for the observation
       (dafile = CALDB)           Dead area file (input to mkwarf)
      (bkgresp = yes)             Create background ARF and RMF?
       (weight = yes)             Should response files be weighted?
   (weight_rmf = no)              Should RMF also be weighted?
     (refcoord = )                RA and Dec of responses?
   (correctpsf = no)              Apply point source aperture correction to ARF?
      (combine = no)              Combine ungrouped output spectra and responses?
    (grouptype = NUM_CTS)         Spectrum grouping type (same as grouptype in dmgroup)
      (binspec = 15)              Spectrum grouping specification (NONE,1:1024:10,etc)
(bkg_grouptype = NONE)            Background spectrum grouping type (NONE, BIN, SNR, NUM_BINS, NUM_CTS, or ADAPTIVE)
  (bkg_binspec = )                Background spectrum grouping specification (NONE,10,etc)
       (energy = 0.3:11.0:0.01)   Energy grid
      (channel = 1:1024:1)        RMF binning attributes
  (energy_wmap = 300:2000)        Energy range for (dmextract) WMAP input to mkacisrmf
      (binwmap = tdet=8)          Binning factor for (dmextract) WMAP input to mkacisrmf
   (binarfwmap = 1)               Binning factor for (sky2tdet) WMAP input to mkwarf
      (clobber = no)              OK to overwrite existing output file?
      (verbose = 1)               Debug Level(0-5)
         (mode = ql)
    

History

01 Feb 2006 new for CIAO 3.3
15 Feb 2006 created Running mkacisrmf Independently section
31 Mar 2006 specextract use update added to Overview
05 Apr 2006 In light of the specextract usage change, the thread has been rewritten to use extended sources in the examples
14 Apr 2006 added Analysis at the edges of ACIS CCDs caveat
24 May 2006 added new information to Using the ACIS "Blank-Sky" Background Files caveat
14 Jun 2006 corrected link in "Calibration Updates"; clarified information on GRADED mode data
18 Dec 2006 updated for CIAO 3.4: new calibration files in CALDB 3.3.0; Extracting Multiple Spectra section uses a stack of output file roots (new feature in CIAO 3.4); output files in one-output case no longer have "src1" or "bkg1" included in the filename; mkrmf no longer prints messages at verbose=0; CIAO version in warnings
06 Mar 2007 added ACIS dead area correction section
22 Jan 2008 updated for CIAO 4.0: ACIS dead area correction parameters added to the specextract.par file: pbkfile and dafile (dead area correction is turned on by default); new ACIS blank-sky background file in CALDB 3.4.0 eliminate the header keyword issue; available links point to the Sherpa Beta website; removed outdated calibration updates
31 Mar 2008 updated for CALDB 3.4.3: use mkacisrmf for -110 BI chips if TGAIN calibration has been applied
26 Jun 2008 updated Analysis at the edges of ACIS CCDs caveats (aspect information not taken into account by specextract)
04 Feb 2009 updated for CIAO 4.1: images are inline; Sherpa link updated to 4.1 website; screen output changed with CALDB 4; input data must have a CTI_APP keyword
17 Feb 2009 added "for Extended Sources" to the title
12 Jan 2010 updated for CIAO 4.2: specextract uses a CALDB query to decide which RMF tool should be used; calibration update - the ACIS QE contamination model has been upgraded to vN0005.
05 Mar 2010 added additional information to the Choosing a background file section
09 Mar 2010 The ACIS detector is calibrated over the range 0.224004 - 12 keV; choosing values outside this range will result in errors from specextract.
22 Mar 2010 added "Special case: -110 C data on a front-illuminated chip" to the Creating RMFs: mkrmf vs mkacisrmf section
15 Dec 2010 updated for CIAO 4.3: new ACIS contamination calibration file; specextract is part of the scripts package; the script has been updated to run the sky2tdet tool when creating weighted responses, so an aspect histogram must be provided; the mask file is needed as input for mkwarf
22 Dec 2010 specextract bug fixes released in 22 Dec 2010 scripts package update
25 Feb 2011 updated for 25 Feb scripts package release: the asp parameter can take aspect solution files directly (don't have to run asphist make an aspect histogram first); if the bad pixel file is supplied in the badpixfile parameter, the ardlib parameters are set by the script
01 Mar 2011 CALDB 4.4.2 release: fix to the header of the ACIS QE contamination file. Prior to this release, CIAO would fail when trying to look up the contamination model correction for chips ACIS-8 (S4) and ACIS-9 (S5).
04 Apr 2011 updated for 04 Apr scripts package release: acis_fef_lookup script prints the version at verbose > 0. specextract: user is prompted for the bkgfile parameter (previously was hidden); new parameter, bkgresp, determines whether a background ARF and RMF should be created; additional changes are outlined in detail in the script release notes for 04 Apr 2011.
12 Apr 2011 updated for 12 Apr scripts package release: specextract bug fix - regions may be specified on the command line or in a file, i.e. "sky=circle(344,435,10)" vs. "sky=region(src.reg)".
26 Apr 2011 install version 2 of the tools package for CIAO 4.3 to fix the mkrmf bug; updated for 25 Apr scripts package release: specextract runs the remove_extra_time_keywords script to work around a sky2tdet bug
21 Jun 2011 added a "step-by-step" section
06 Jul 2011 specextract bug fixes were released in the 07 July 2011 scripts package; required software updates are listed in Synopsis
06 Sep 2011 a specextract bug fix was released in the 06 Sep 2011 scripts package
06 Oct 2011 a specextract bug fix and a workaround for a known CIAO bug were released in the 06 Oct 2011 scripts package
15 Dec 2011 reviewed for CIAO 4.4: the remove_extra_time_keywords script is no longer needed, since the sky2tdet bug is fixed in CIAO 4.4
16 Feb 2012 a specextract bug fix was released in the 16 Feb 2012 scripts package: the script will not error out if you set outroot to be a stack and have correct=yes and weight=no.
03 Dec 2012 Review for CIAO 4.5; minor formatting
08 Aug 2013 Updated for the contributed scripts 4.5.4 release: the "ancillary" files - e.g. aspect solution and bad-pixel files - can now be picked up from information in the event file (in many cases). The Extracting Multiple Spectra section has also been updated.
12 Dec 2013 Review for CIAO 4.6: by default weighted RMF files are no-longer generated; the pbkfile parameter has been removed from specextract and is no-longer used by tools like mkwarf; the response files are now always named with the .rmf and .arf suffix, even when weighted; the multiple spectra section now uses reproject_obs to align the observations before source extraction; although not shown in this thread, specextract can now be used with HRC-I data sets.
17 Dec 2014 Reviewed for CIAO 4.7; removed "New in CIAO4.6" sections. Minor edits.
08 201 2019 Updated sherpa plot to use matplotlib.
18 Jan 2022 Review for CIAO 4.14. Updated for Repro5 (coordinates) and CALDB 4.9.6.