Skip to the navigation links
Last modified: 16 Aug 2010
Where are the PDFs?

Coadding Spectra and Weighted Responses

CIAO 4.2 Science Threads



Overview

Last Update: 16 Aug 2010 - the combine_spectra script is a new, better method for summing multiple imaging spectra and response files

Synopsis:

[New] the combine_spectra script sums multiple imaging source PHA spectra, and the associated background PHA spectra and ARF files. combine_spectra should be used in combination with the specextract script in place of acisspec. To use the script, install the contrib tarfile; read the combine_spectra help file for examples.

Users are encouraged to use specextract in place of acisspec for new analyses, noting that specextract does not yet offer the option to coadd spectra as acisspec does; refer to the specextract thread for more information.

In this thread, acisspec is used to weight the spectra for their respective exposures and (optionally) group the output spectra. It also creates weighted responses (WARF and WRMF).

Purpose:

To describe two ways to run the acisspec script on a point source (the zero-order source of two HETG/ACIS-S observations):

  1. To use the acisspec script on two point sources to create a source spectra, coadd them, and make the corresponding weighted responses.
  2. To use spectra and WMAPs as input to create coadded and weighted output spectra.

Read this thread if:

you are working with an ACIS imaging observation or the 0th-order of a grating observation and would like to perform one of the analyses described in the Purpose (above).

Calibration Updates:




Contents



Getting Started

Sample ObsIDs used: 9100 (ACIS-S, SNR 0104-72.3); 9810 (ACIS-S, SNR 0104-72.3)

File types needed: evt2

The ACIS dead area correction

When coadding files with acisspec it is not appropriate to apply the ACIS dead area correction. Since coadded data is provided as input to mkwarf, there is not an accurate single file to provide for the pbkfile parameter. Therefore, the correction is turned off in the examples in this thread.

Refer to the ACIS Dead Area Correction why topic for more information.



Downloading acisspec

This thread uses the acisspec script, which is part of the CIAO Scripts distribution. The CIAO scripts package should be the following version or newer:

unix% cat $ASCDS_CONTRIB/VERSION.CIAO_scripts
17 Apr 2009

Please check that you have at least this version of the scripts package installed before continuing. If you do not have the scripts installed or need to update to a newer version, refer to the Scripts page.



Defining Source & Background Regions

First we need to define regions from which the source and background spectra will be created. In this example, the same source and background regions are used for both files:

unix% cat src.reg
# Region file format: CIAO version 1.0
circle(4473.5,3611.5,40)

unix% cat bgd.reg
# Region file format: CIAO version 1.0
circle(4353.0001,3680.7499,20)
circle(4343,3599.7499,20)
circle(4422.5001,3737.2499,20)
circle(4530.2501,3728.4999,20)

Make sure that you save the regions in CIAO format so that they are fully compatible with the analysis tools; see the Using CIAO Regions thread for more information.

The regions are shown displayed on the event files in Figure 1.

[Thumbnail image: The two event files used for spectral extraction are displayed in ds9 with the source and background regions overlaid.]

[Version: full-size]

[Print media version: The two event files used for spectral extraction are displayed in ds9 with the source and background regions overlaid.]

Figure 1: Extraction regions on the event files

ObsID 9100 is displayed in the top frame and ObsID 9810 is displayed in the bottom frame. Both event files have a white source region (src.reg) and green background regions (bgd.reg) overlaid.



Calculating the Weights

In order to apply a weighting factor to the spectra before coadding them, we need to calculate the ratio of each observation's exposure to the total exposure (sum of the two).

First, find the individual exposures:

unix% dmkeypar acisf09100N001_evt2.fits EXPOSURE echo+
54466.269883243

unix% dmkeypar acisf09810N001_evt2.fits EXPOSURE echo+
55815.36769867

Now calculate the total exposure and the ratios:

total exposure: 24527.46+24845.44 = 110281.64

weight1: 54466.27/110281.64 = 0.49

weight2: 55815.34/110281.64 = 0.51


Running the Script

The thread shows two sample runs of acisspec: one beginning at the event files and the other using spectra and WMAPs as input.

Beginning at the event files

This run of the script takes the original event files as input, creating and coadding the spectra and weighted responses. In this example, we opt to create both source and background spectra.

We need to set the input source and background files - including the regions with the appropriate DM filter - as well as the root and weight parameters. The pbkfile and dafile are set such that the ACIS dead area correction is not applied.

unix% punlearn acisspec
unix% pset acisspec soufile1="acisf09100N001_evt2.fits[sky=region(src.reg)]"
unix% pset acisspec soufile2="acisf09810N001_evt2.fits[sky=region(src.reg)]"
unix% pset acisspec bgfile1="acisf09100N001_evt2.fits[sky=region(bgd.reg)]"
unix% pset acisspec bgfile2="acisf09810N001_evt2.fits[sky=region(bgd.reg)]"
unix% pset acisspec pbkfile="" dafile=NONE
unix% pset acisspec root=events
unix% pset acisspec weight1=0.49 weight2=0.51

Now execute the script:

unix% acisspec
Primary source file: events or spectrum (acisf09100N001_evt2.fits[sky=region(src.reg)]): 
Root name for output files (events): 
NONE, or name of the parameter block file (): 
BTIMDRFT values are different...FAIL...
BTIMNULL values are different...FAIL...
BTIMRATE values are different...FAIL...
warning: DS_IDENT has different value...Merged...
warning: OBS_ID has different value...Merged...
(messages are repeated for second file)

# mkwarf (CIAO 4.2): WARNING: Input image name was "events_sou12.wmap" instead of "WMAP".  
  Will attempt to use.

# mkwarf (CIAO 4.2): WARNING: Input image name was "events_bgd12.wmap" instead of "WMAP".
  Will attempt to use.

Atten: grid input,"energy=0.3:11.0:0.01", is ignored as a weights file, "events_sou12.wfef", is found.


Total 5 regions to be processed:
--- Region #1 processed
--- Region #2 processed
--- Region #3 processed
--- Region #4 processed
--- Region #5 processed



Atten: grid input,"energy=0.3:11.0:0.01", is ignored as a weights file, "events_bgd12.wfef", is found.


Total 13 regions to be processed:
--- Region #1 processed
--- Region #2 processed
--- Region #3 processed
...
--- Region #12 processed
--- Region #13 processed

See the Common Warnings section for an explanation of the messages printed to the screen.

Check the parameter file that was used with plist acisspec.

There are a couple things to note about this run:

  • The default values of the binwmap (det=8) and ewmap (300:2000) were used.

  • The spectra were not grouped in this case. There are four parameters that can be used to specify grouping for the source (gtype, gspec) and background (bggtype, bggspec) spectra, if desired.

You may now skip to the Fitting section.


Spectra and WMAPs as input

This run of the script takes source spectra as input to create and coadd the weighted spectra. When using a spectrum for either soufile parameter, it is required that the corresponding WMAP (an image of the field in detector coordinates) be supplied. This can easily be made with dmcopy; for details on this process, see the Weighting ARFs and RMFs thread, specifically the Create the WMAP: Using dmcopy section.

In this example, we opt to create only source spectra, so background WMAPs are not needed:

unix% dmcopy "acisf09100N001_evt2.fits[sky=region(src.reg)][energy=300:2000][bin det=8]" \
      src_1.wmap

unix% dmcopy "acisf09810N001_evt2.fits[sky=region(src.reg)][energy=300:2000][bin det=8]" \
      src_2.wmap

The input spectra were created with a basic dmextract command:

unix% dmextract "acisf09100N001_evt2.fits[sky=region(src.reg)][bin pi]" src_1.pi

unix% dmextract "acisf09810N001_evt2.fits[sky=region(src.reg)][bin pi]" src_2.pi

We can now set the necessary parameters. Again, the pbkfile and dafile are set so that the ACIS dead area correction is not applied.

unix% punlearn acisspec
unix% pset acisspec soufile1=src_1.pi soufile2=src_2.pi
unix% pset acisspec souwmap1=src_1.wmap souwmap2=src_2.wmap 
unix% pset acisspec pbkfile="" dafile=NONE
unix% pset acisspec root=spectra
unix% pset acisspec weight1=0.49 weight2=0.51

And execute the script:

unix% acisspec
Primary source file: events or spectrum (src_1.pi): 
Root name for output files (spectra): 
NONE, or name of the parameter block file (): 
BTIMDRFT values are different...FAIL...
BTIMNULL values are different...FAIL...
BTIMRATE values are different...FAIL...
warning: DS_IDENT has different value...Merged...
warning: OBS_ID has different value...Merged...
# mkwarf (CIAO 4.2): WARNING: Input image name was "spectra_sou12.wmap" instead of "WMAP".
  Will attempt to use.

Atten: grid input,"energy=0.3:11.0:0.01", is ignored as a weights file, "spectra_sou12.wfef", is found.


Total 5 regions to be processed:
--- Region #1 processed
--- Region #2 processed
--- Region #3 processed
--- Region #4 processed
--- Region #5 processed

See the Common Warnings section for an explanation of the messages printed to the screen.

Check the parameter file that was used with plist acisspec.

There are a couple things to note about this run:

  • The default values of the binwmap (det=8) and ewmap (300:2000) were used.

  • The spectra were not grouped in this case. There are two parameters that can be used to specify grouping for the source spectrum, if desired: gtype and gspec.



Fitting

The BACKFILE, RESPFILE, and ANCRFILE header keywords have been updated in the source spectrum files:

unix% dmlist events_sou12.pi header | grep FILE
0110 BACKFILE             events_bgd12.pi                String
0111 CORRFILE             none                           String
0112 RESPFILE             events_sou12.wrmf              String
0113 ANCRFILE             events_sou12.warf              String

Since no background spectrum was created for the second run, the BACKFILE keyword is unaffected:

unix% dmlist spectra_sou12.pi header |grep FILE
0110 BACKFILE             none                           String
0111 CORRFILE             none                           String
0112 RESPFILE             spectra_sou12.wrmf             String
0113 ANCRFILE             spectra_sou12.warf             String

When the source spectra are read into Sherpa, the background file (when applicable) and source-related responses will automatically be read in as well. Note that the RESPFILE and ANCRFILE header keywords are not updated in the background spectrum; to fit the background, the background response files will have to be loaded into Sherpa separately.

If you would like to fit the background-subtracted source spectrum using a common RMF and ARF for source and background, simply read the source spectrum FITS file into Sherpa, subtract the background, and fit it. See the Introduction to Fitting PHA Spectra thread for details.

To fit source and background spectra simultaneously with distinct RMFs and ARFs, follow the Independent Background Responses thread.



Common Warnings

There are several warning messages that you may see printed to the screen during the acisspec run:

  • Originating tool: dmmerge

    BTIMDRFT values are different...FAIL...
    BTIMNULL values are different...FAIL...
    BTIMRATE values are different...FAIL...
    omit - DEC_NOM values different more than 0.000300
    warning: DETNAM has different value...Merged...
    warning: OBJECT has different value...Merged...
    warning: OBSERVER has different value...Merged...
    warning: OBS_ID has different value...Merged...
    omit - RA_NOM values different more than 0.000300
    omit - ROLL_NOM values different more than 1.000000
    warning: SEQ_NUM has different value...Merged...
    warning: TITLE has different value...Merged...
    

    The messages are related to how the tool merges the header information in the input files. The merging_rules ahelp file explains the rules and how they affect the output file header.

  • Originating tool: mkwarf

    # mkwarf (CIAO 4.2): WARNING: Input image name was "spectra_sou12.wmap" instead of "WMAP".  Will attempt to use.
    

    There are two ways to create a WMAP, with dmextract or with dmcopy; acisspec uses the dmcopy method. In this process, the primary block of the WMAP file is name EVENTS_IMAGE, as opposed to WMAP. Since we know that sources.wmap8 was created as a WMAP image (i.e. the detector coordinate system was used), the warning message can be ignored.

  • Originating tool: mkwarf

    Couldn't determine chip position for pixel: (4076.500000,4052.500000) with value=3.000000. Skipping pixel
    

    If the source is near a chip boundary, mkwarf may not be able to determine the chip location of every pixel in the WMAP. This has to do with the lack of SIM info in the image; details are available in this FAQ.

    In the vast majority of cases, the number of counts ignored (i.e. the sum of the "value" of each ignored pixel) is much smaller than the total signal in the WMAP. However, this may not be the case for small regions near chip boundaries. dmstat may be used to determine the sum of all pixels in the WMAP for comparison to the sum of the ignore pixel values.

  • Originating tool: mkrmf

     Atten: grid input,"energy=0.3:11.0:0.01", is ignored as a weights file, "es_1447_sou.wfef", is found.
    

    The energy axis grid is ignored since the grid is read from the weights file. Since the value entered for the energy axis (axis1 in this case) must be syntactically correct for the tool to parse over it properly, acisspec uses "energy=0.3:11.0:0.0" as a placeholder.



Analysis Caveats

Users should be cautious about analyzing the data for sources near the edges of the ACIS CCDs.

  1. For X-rays passing through the mirrors, the very bottom of each CCD is obscured by the frame store. As a result, some of the events in rows with CHIPY <= 8 are not detected. (The set of rows affected varies from CCD to CCD.) Since the CIAO tools do not compensate for this effect, the ARFs and exposure maps for sources in these regions may be inaccurate.

  2. For sources within about thirty-two pixels of any edge of a CCD, the source may be dithered off the CCD during part of an observation. The aspect histogram, which is used to create ARFs and exposure maps, is designed to compensate for this effect.

  3. An ARF calculated at the edge of a chip will not be accurate. The response tools for spectral extraction (specifically the ARF) assume that 100% of the PSF is enclosed - i.e. on the chip - all the time, which may not be the case. The amount of error introduced depends on how close the source is to the edge, the morphology of the source, and the characteristics of the PSF, which depends on the source spectrum.

  4. A contaminant has accumulated on the optical-blocking filters of the ACIS detectors, as described in the ACIS QE Contamination why topic. Since there is a gradient in the temperature across the filters (the edges are colder), there is a gradient in the amount of material on the filters. (The contaminant is thicker at the edges.) Within about 100 pixels of the outer edges of the ACIS-I and ACIS-S arrays, the gradient is relatively steep. Therefore, the effective low-energy (' 1 keV) detection efficiency may vary within the dither pattern in this region. The ARF and instrument map tools are designed to read a calibration file which describes this spatial dependence.




Parameters for /home/username/cxcds_param/acisspec.par


      soufile1 = acisf09100N001_evt2.fits[sky=region(src.reg)] Primary source file: events or spectrum
          root = events           Root name for output files
       pbkfile =                  NONE, or name of the parameter block file
      (bgfile1 = acisf09100N001_evt2.fits[sky=region(bgd.reg)]) Primary background file: events or spectrum
     (soufile2 = acisf09810N001_evt2.fits[sky=region(src.reg)]) Secondary source file: events or spectrum
      (bgfile2 = acisf09810N001_evt2.fits[sky=region(bgd.reg)]) Secondary background file: events or spectrum
     (souwmap1 = )                Primary source WMAP
      (bgwmap1 = )                Primary background WMAP
     (souwmap2 = )                Secondary source WMAP
      (bgwmap2 = )                Secondary background WMAP
      (binwmap = det=8)           Binning factor for WMAPs
        (ewmap = 300:2000)        Energy Range for WMAPs
      (weight1 = 0.49)            Weight for the primary source spectrum
      (weight2 = 0.51)            Weight for the secondary source spectrum
        (gtype = )                Source spectrum grouping type (NONE, BIN, SNR, NUM_BINS, NUM_CTS, or ADAPTIVE)
        (gspec = )                Source spectrum grouping specs (NONE,10,etc)
      (bggtype = )                Background spectrum grouping type (NONE, BIN, SNR, NUM_BINS, NUM_CTS, or ADAPTIVE)
      (bggspec = )                Background spectrum grouping specs (NONE,10,etc)
       (dafile = NONE)            CALDB, NONE, or name of ACIS dead-area calibration file
      (clobber = no)              Clobber existing file
      (verbose = 0)               Debug Level(0-5)
         (mode = ql)              




Parameters for /home/username/cxcds_param/acisspec.par


      soufile1 = src_1.pi         Primary source file: events or spectrum
          root = spectra          Root name for output files
       pbkfile =                  NONE, or name of the parameter block file
      (bgfile1 = )                Primary background file: events or spectrum
     (soufile2 = src_2.pi)        Secondary source file: events or spectrum
      (bgfile2 = )                Secondary background file: events or spectrum
     (souwmap1 = src_1.wmap)      Primary source WMAP
      (bgwmap1 = )                Primary background WMAP
     (souwmap2 = src_2.wmap)      Secondary source WMAP
      (bgwmap2 = )                Secondary background WMAP
      (binwmap = det=8)           Binning factor for WMAPs
        (ewmap = 300:2000)        Energy Range for WMAPs
      (weight1 = 0.49)            Weight for the primary source spectrum
      (weight2 = 0.51)            Weight for the secondary source spectrum
        (gtype = )                Source spectrum grouping type (NONE, BIN, SNR, NUM_BINS, NUM_CTS, or ADAPTIVE)
        (gspec = )                Source spectrum grouping specs (NONE,10,etc)
      (bggtype = )                Background spectrum grouping type (NONE, BIN, SNR, NUM_BINS, NUM_CTS, or ADAPTIVE)
      (bggspec = )                Background spectrum grouping specs (NONE,10,etc)
       (dafile = NONE)            CALDB, NONE, or name of ACIS dead-area calibration file
      (clobber = no)              Clobber existing file
      (verbose = 0)               Debug Level(0-5)
         (mode = ql)              



History

14 Dec 2004 updated for CIAO 3.2: created Using Consistent Calibration section; can be run on 0th order grating in CIAO 3.2
25 Feb 2005 acisspec updated to version 3.4: better error handling (see the script update history for a complete description of the changes)
23 Jun 2005 CIAO 3.2.2 patch: new calibration for mkacisrmf is available; see the Using Consistent Calibration: mkrmf vs mkacisrmf section
16 Dec 2005 updated for CIAO 3.3: the acisspec script has been updated to version 3.5; note about specextract in the Overview
01 Feb 2006 added link to specextract thread
14 Feb 2006 correction: weight1 was given in some places as 0.58, when it should be 0.59
16 Feb 2006 acisspec updated to version 3.6: improved the conditional statement for recognizing weighted vs. unweighted cases.
05 Apr 2006 specextract use update added to Overview
14 Jun 2006 corrected link in "Calibration Updates"; clarified information on GRADED mode data
01 Dec 2006 updated for CIAO 3.4: acisspec updated to version 3.7 (removed "punlearn mkwarf" to allow new CIAO 3.4 mkwarf parameters (asolfile, pbkfile, dafile) to be set before running the script, if desired); CIAO version in warnings
02 Feb 2007 updated for CALDB 3.3.0.1 patch
26 Feb 2007 acisspec v4.0: Dead area correction parameters added to the acisspec.par file: pbkfile and dafile (added ACIS dead area correction section). Script prints a warning if RMF should be remade with mkacisrmf.
23 Jan 2008 updated for CIAO 4.0: acisspec v4.1 (parameter updates for applying ACIS dead area correction, which is turned on by default); links point to Sherpa Beta website; removed outdated calibration updates; filename and screen output updated for reprocessed data (version N002 event file for 2463)
31 Mar 2008 updated for CALDB 3.4.3: use mkacisrmf for -110 BI chips if TGAIN calibration has been applied
03 Feb 2009 updated for CIAO 4.1: thread rewritten with ObsIDs 9100 and 9810; image converted to inline; "ARDLIB warning ... Assuming the first "interesting" extension." no longer printed; input data must have a CTI_APP keyword
19 Feb 2009 renamed "Using the Output" section to "Fitting" and included additional links to Sherpa threads
06 May 2009 check the version of the CIAO scripts package instead of the individual script
12 Jan 2010 updated for CIAO 4.2: calibration update - the ACIS QE contamination model has been upgraded to vN0005.
16 Aug 2010 the combine_spectra script is a new, better method for summing multiple imaging spectra and response files

Return to Threads Page: Top | All | Imag Spec

Where are the PDFs?
Last modified: 16 Aug 2010