Skip to the navigation links
Last modified: 12 Jan 2010
Where are the PDFs?

Using merge_all to Compute ACIS Exposure Maps and Fluxed Images

CIAO 4.2 Science Threads



Overview

Last Update: 12 Jan 2010 - updated for CIAO 4.2: calibration update - the ACIS QE contamination model has been upgraded to vN0005.

Synopsis:

mkexpmap generates an exposure map which may be used to convert a counts image of a source to an image in flux units. The computed exposure map is essentially an image of the effective area at each sky position, accounting for the effects of dither motion which are especially important near the edges of the detector.

Purpose:

To create a wide variety of exposure maps using the script merge_all.

Read this thread if:

you are working with an ACIS imaging observation and would like to create an exposure map. If you are interested in having an accurate exposure map at the very edge of a CCD, subarray or window, this thread cannot be used. Instead, run one of the step-by-step exposure map threads that include the use of the mask file.

merge_all will produce incorrect results if one (or more) of the observations was done with an offset. For details and a workaround, refer to the merge_all bugs page.

Calibration Updates:

Related Links:




Contents



Getting Started

Sample ObsIDs used: 1842 (ACIS-I, G21.5-09); 1843 (ACIS-I, G21.5-09)

File types needed: evt2; asol1

This thread uses the merge_all script; for information about the script, consult the help file ("ahelp merge_all"). The script is part of the CIAO Scripts distribution. The CIAO scripts package should be the following version or newer:

unix% cat $ASCDS_CONTRIB/VERSION.CIAO_scripts
01 Jun 2009

Please check that you have at least this version of the scripts package installed before continuing. If you do not have the scripts installed or need to update to a newer version, refer to the Scripts page.

Also, make sure that you have set up ardlib to use the bad pixel file for your observation before following this thread. Note that ardlib.par will only accept the bad pixel file for a single observation. If you are merging two or more observations which do not have similar bad pixel files (e.g. Faint vs. Very Faint mode observations), you will have to create the merged exposure map manually. This can be accomplished by following the Multiple Chip example and using reproject_image to combine the final products.

We begin by restricting the energy range from 0.3 keV to 10 keV:

unix% dmcopy "acisf01842N002_evt2.fits[energy=300:10000]" 1842_evt2.fits
unix% dmcopy "acisf01843N002_evt2.fits[energy=300:10000]" 1843_evt2.fits

Caveat: extra GTIs in merged event files

When event files that have different ranges of exposure numbers (expno column) are merged, the output file will have more than one GTI block for each chip. This causes incorrect exposure values later in the analysis, as the CIAO tools cannot use multiple GTI blocks for a single chip. Further details are available on the dmmerge bug page.

A "[subspace -expno]" filter was added to the dmmerge command in merge_all v3.6 as a workaround for this problem.

Note that the addition of the subspace filter means that any user who intends to create lightcurves binned on exposure number from the merged output cannot use merge_all, since that information is eliminated from the subspace. (In general, lightcurves are binned on time.)


Additional exposure map and fluxed image threads

This thread does not explain all the subtleties in creating exposure maps and fluxed images. For a more detailed explanation of the process, refer to the following threads:



Create a Single-Chip, Single Observation Exposure Map and Fluxed Image

Sample ObsID used in this section: 1843, chip I3 (ccd_id=3) only

Since we are only creating an exposure map and fluxed image for a single observation, we do not need to set the parameters refcoord or merged. The xygrid parameter is chosen so that the resulting exposure map includes the region of the I3 chip and is effectively binned by 2.

Finally, we choose to make the map monoenergetic with an energy of 1 keV (see the Single Chip ACIS Exposure Map Step-by-Step thread for details).

Set the necessary parameters:

unix% punlearn merge_all
unix% pset merge_all evtfile=1843_evt2.fits 
unix% pset merge_all asol=pcadf084271087N002_asol1.fits
unix% pset merge_all chip=3
unix% pset merge_all xygrid=2480.5:3680.5:#600,3940.5:5140.5:#600
unix% pset merge_all energy=1
unix% pset merge_all expmap=1843_I3_expmap.fits
unix% pset merge_all expcorr=1843_I3_expcorr.fits

Run the script:

unix% merge_all
Input event file(s) (1843_evt2.fits): 
Input asol file(s); time-sorted (pcadf084271087N002_asol1.fits): 
Input Live Time Correction list files for HRC-I (): 
ACIS CCD ID(s), comma-separated or range; or 'HRC-I' or 'HRC-S' (3): 
Reference coordinates or evt2 file for reproject_events (): 
XY grid (e.g., 0.5:8192.5:#1024,0.5:8192.5:#1024) (2480.5:3680.5:#600,3940.5:5140.5:#600): 
Energy or spectral weights file for mkinstmap (1): 
Output merged event file (): 
Output merged exposure map (1843_I3_expmap.fits): 
Output exposure-corrected image (1843_I3_expcorr.fits): 

Creating aspect histogram for CCD 3...

Making instrument map for CCD 3...

Making exposure map for CCD 3...
Exposure map limits: 0.000000e+00, 2.840771e+06

Creating image

Normalizing image

All done!

In many cases, there will be more than one aspect solution file (pcad_asol1.fits) for an observation. All the files must be input to the asol parameter, either as a list or as a stack.

The output exposure map is shown in Figure 1 and the output fluxed image is shown in Figure 2.

Check the parameter file that was used with plist merge_all.



Create a Multiple-Chip, Single Observation Exposure Map and Fluxed Image

Sample ObsID used in this section: 1843, all chips:

unix% dmkeypar 1843_evt2.fits DETNAM echo+
ACIS-012367

Since we are only using a single observation, the parameters refcoord and merged do not need to be set. The xygrid parameter is chosen so that the resulting exposure map is effectively binned by 8.

Finally, we choose to make the map monoenergetic with an energy of 1 keV (see the Create a Single Chip ACIS Exposure Map Step-by-Step thread for details).

Set the necessary parameters:

unix% punlearn merge_all
unix% pset merge_all evtfile=1843_evt2.fits 
unix% pset merge_all asol=pcadf084271087N002_asol1.fits
unix% pset merge_all chip=0,1,2,3,6,7
unix% pset merge_all xygrid=0.5:8192.5:#1024,0.5:8192.5:#1024
unix% pset merge_all energy=1
unix% pset merge_all expmap=1843_all_expmap.fits
unix% pset merge_all expcorr=1843_all_expcorr.fits

Run the script:

unix% merge_all
Input event file(s) (1843_evt2.fits): 
Input asol file(s); time-sorted (pcadf084271087N002_asol1.fits): 
Input Live Time Correction list files for HRC-I (): 
ACIS CCD ID(s), comma-separated or range; or 'HRC-I' or 'HRC-S' (0,1,2,3,6,7): 
Reference coordinates or evt2 file for reproject_events (): 
XY grid (e.g., 0.5:8192.5:#1024,0.5:8192.5:#1024) (0.5:8192.5:#1024,0.5:8192.5:#1024): 
Energy or spectrum file for mkinstmap (1): 
Output merged event file (): 
Output merged exposure map (1843_all_expmap.fits): 
Output exposure-corrected image (1843_all_expcorr.fits): 

Creating aspect histogram for CCD 0...
Creating aspect histogram for CCD 1...
Creating aspect histogram for CCD 2...
Creating aspect histogram for CCD 3...
Creating aspect histogram for CCD 6...
Creating aspect histogram for CCD 7...

Making instrument map for CCD 0...

Making exposure map for CCD 0...
Exposure map limits: 0.000000e+00, 2.923175e+06
Making exposure map for CCD 1...
Exposure map limits: 0.000000e+00, 2.962581e+06
Making exposure map for CCD 2...
Exposure map limits: 0.000000e+00, 2.869983e+06
Making exposure map for CCD 3...
Exposure map limits: 0.000000e+00, 2.839312e+06
Making exposure map for CCD 6...
Exposure map limits: 0.000000e+00, 2.445186e+06
Making exposure map for CCD 7...
Exposure map limits: 0.000000e+00, 3.433622e+06

Merging exposure maps

Creating image

Normalizing image

All done!

In many cases, there will be more than one aspect solution file (pcad_asol1.fits) for an observation. All the files must be input to the asol parameter, either as a list or as a stack.

The output exposure map is shown in Figure 3 and the output fluxed image is shown in Figure 4.

Check the parameter file that was used with plist merge_all.



Create a Multiple-Chip, Multiple Observation Exposure Map and Fluxed Image

Sample ObsIDs used in this section: 1842, 1843, chips I0-I3 (ccd_id=0,1,2,3) for both

The script is now used to merge the two observations and to create an exposure map and fluxed image. The second observation (ObsID 1842) is reprojected to the coordinate system of the first. Instead, one could choose to reproject both file to a new new tangent point in R.A. and Dec. [degrees].

The xygrid parameter is chosen so that the resulting exposure map is effectively binned by 8.

Finally, we choose to make the map monoenergetic with an energy of 1 keV (see the Create a Single Chip ACIS Exposure Map Step-by-Step thread for details).

In many cases, there will be more than one aspect solution file (pcad_asol1.fits) for an observation. All the files must be input to the asol parameter, either as a list or as a stack.

Set the necessary parameters:

unix% punlearn merge_all
unix% pset merge_all evtfile=1843_evt2.fits,1842_evt2.fits
unix% pset merge_all asol=pcadf084271087N002_asol1.fits,pcadf084280882N002_asol1.fits
unix% pset merge_all chip=0:3
unix% pset merge_all refcoord=1843_evt2.fits
unix% pset merge_all xygrid=0.5:8192.5:#1024,0.5:8192.5:#1024
unix% pset merge_all energy=1
unix% pset merge_all merged=merged_evt2.fits
unix% pset merge_all expmap=merged_expmap.fits
unix% pset merge_all expcorr=merged_expcorr.fits

Run the script:

unix% merge_all
Input event file(s) (1843_evt2.fits,1842_evt2.fits): 
Input asol file(s); time-sorted (pcadf084271087N002_asol1.fits,pcadf084280882N002_asol1.fits): 
Input Live Time Correction list files for HRC-I (): 
ACIS CCD ID(s), comma-separated or range; or 'HRC-I' or 'HRC-S' (0:3): 
Reference coordinates or evt2 file for reproject_events (1843_evt2.fits): 
XY grid (e.g., 0.5:8192.5:#1024,0.5:8192.5:#1024) (0.5:8192.5:#1024,0.5:8192.5:#1024): 
Energy or spectrum file for mkinstmap (1): 
Output merged event file (merged_evt2.fits): 
Output merged exposure map (merged_expmap.fits): 
Output exposure-corrected image (merged_expcorr.fits): 

Reprojecting 1842_evt2.fits to 1843_evt2.fits

Merging data...

SIM_Z of event file: -245.85729313244

SIM_Z of pcadf084271087N002_asol1.fits: -245.857293132441
Difference is -9.9475983006414e-13 mm: SIM_Z differs by less than ~1/4 pixel.

SIM_Z of pcadf084280882N002_asol1.fits: -231.239063969172
Difference is 14.618229163268 mm: SIM_Z differs by more than ~1/4 pixel.
Modifying DZ in pcadf084280882N002_asol1.fits

Creating aspect histogram for CCD 0...
Creating aspect histogram for CCD 1...
Creating aspect histogram for CCD 2...
Creating aspect histogram for CCD 3...

Making instrument map for CCD 0...
Making instrument map for CCD 1...
Making instrument map for CCD 2...
Making instrument map for CCD 3...

Making exposure map for CCD 0...
Exposure map limits: 0.000000e+00, 5.616112e+06
Making exposure map for CCD 1...
Exposure map limits: 0.000000e+00, 5.706218e+06
Making exposure map for CCD 2...
Exposure map limits: 0.000000e+00, 5.502165e+06
Making exposure map for CCD 3...
Exposure map limits: 0.000000e+00, 5.458836e+06

Merging exposure maps

Creating image

Normalizing image

All done!

The combined evt2 file is shown in Figure 5, the exposure map is shown in Figure 6, and the fluxed image is shown in Figure 7. Note that for these observations, the SIM_Z value (position of the Scientific Instrument Module with respect to the aimpoint) differed significantly. This is the reason for the modification of DZ that the script made, but will not be the case for all of observations.

Check the parameter file that was used with plist merge_all.



Calculate the Source Flux

Since the units of the fluxed image are [photon/cm2/s/pixel], adding up the pixel values around a source results in the source flux in [photon/cm2/s]. Note that this flux is an approximation - as discussed in An Introduction to Exposure Maps (PS, 12pp) - since a spectral shape was assumed when using mkinstmap (in this thread, a monochromatic source).

The Single Chip ACIS Exposure Map Step-by-Step thread shows how to approximate the source flux from the normalized image.



Analysis Caveats

Users should be cautious about analyzing the data for sources near the edges of the ACIS CCDs.

  1. For X-rays passing through the mirrors, the very bottom of each CCD is obscured by the frame store. As a result, some of the events in rows with CHIPY <= 8 are not detected. (The set of rows affected varies from CCD to CCD.) Since the CIAO tools do not compensate for this effect, the ARFs and exposure maps for sources in these regions may be inaccurate.

  2. For sources within about thirty-two pixels of any edge of a CCD, the source may be dithered off the CCD during part of an observation. The aspect histogram, which is used to create ARFs and exposure maps, is designed to compensate for this effect.

  3. An ARF calculated at the edge of a chip will not be accurate. The response tools for spectral extraction (specifically the ARF) assume that 100% of the PSF is enclosed - i.e. on the chip - all the time, which may not be the case. The amount of error introduced depends on how close the source is to the edge, the morphology of the source, and the characteristics of the PSF, which depends on the source spectrum.

  4. A contaminant has accumulated on the optical-blocking filters of the ACIS detectors, as described in the ACIS QE Contamination why topic. Since there is a gradient in the temperature across the filters (the edges are colder), there is a gradient in the amount of material on the filters. (The contaminant is thicker at the edges.) Within about 100 pixels of the outer edges of the ACIS-I and ACIS-S arrays, the gradient is relatively steep. Therefore, the effective low-energy (' 1 keV) detection efficiency may vary within the dither pattern in this region. The ARF and instrument map tools are designed to read a calibration file which describes this spatial dependence.




Parameters for /home/username/cxcds_param/merge_all.par


##
## Parameter file for merge_all script
##
       evtfile = 1843_evt2.fits   Input event file(s)
          asol = pcadf084271087N002_asol1.fits Input asol file(s); time-sorted
       dtffile =                  Input Live Time Correction list files for HRC-I
          chip = 3                ACIS CCD ID(s), comma-separated or range; or 'HRC-I' or 'HRC-S'
      refcoord =                  Reference coordinates or evt2 file for reproject_events
        xygrid = 2480.5:3680.5:#600,3940.5:5140.5:#600 XY grid (e.g., 0.5:8192.5:#1024,0.5:8192.5:#1024)
        energy = 1                Energy or spectral weights file for mkinstmap
        merged =                  Output merged event file
        expmap = 1843_I3_expmap.fits Output merged exposure map
       expcorr = 1843_I3_expcorr.fits Output exposure-corrected image
       (intdir = /tmp)            Directory for intermediate files
      (clobber = no)              OK to overwrite existing output file(s)?
         (mode = ql)              
    


Parameters for /home/username/cxcds_param/merge_all.par


##
## Parameter file for merge_all script
##
       evtfile = 1843_evt2.fits   Input event file(s)
          asol = pcadf084271087N002_asol1.fits Input asol file(s); time-sorted
       dtffile =                  Input Live Time Correction list files for HRC-I
          chip = 0,1,2,3,6,7      ACIS CCD ID(s), comma-separated or range; or 'HRC-I' or 'HRC-S'
      refcoord =                  Reference coordinates or evt2 file for reproject_events
        xygrid = 0.5:8192.5:#1024,0.5:8192.5:#1024 XY grid (e.g., 0.5:8192.5:#1024,0.5:8192.5:#1024)
        energy = 1                Energy or spectral weights file for mkinstmap
        merged =                  Output merged event file
        expmap = 1843_all_expmap.fits Output merged exposure map
       expcorr = 1843_all_expcorr.fits Output exposure-corrected image
       (intdir = /tmp)            Directory for intermediate files
      (clobber = no)              OK to overwrite existing output file(s)?
         (mode = ql)              
    


Parameters for /home/username/cxcds_param/merge_all.par


##
## Parameter file for merge_all script
##
       evtfile = 1843_evt2.fits,1842_evt2.fits Input event file(s)
          asol = pcadf084271087N002_asol1.fits,pcadf084280882N002_asol1.fits Input asol file(s); time-sorted
       dtffile =                  Input Live Time Correction list files for HRC-I
          chip = 0:3              ACIS CCD ID(s), comma-separated or range; or 'HRC-I' or 'HRC-S'
      refcoord = 1843_evt2.fits   Reference coordinates or evt2 file for reproject_events
        xygrid = 0.5:8192.5:#1024,0.5:8192.5:#1024 XY grid (e.g., 0.5:8192.5:#1024,0.5:8192.5:#1024)
        energy = 1                Energy or spectral weights file for mkinstmap
        merged = merged_evt2.fits Output merged event file
        expmap = merged_expmap.fits Output merged exposure map
       expcorr = merged_expcorr.fits Output exposure-corrected image
       (intdir = /tmp)            Directory for intermediate files
      (clobber = no)              OK to overwrite existing output file(s)?
         (mode = ql)              
    

History

04 Jan 2005 updated for CIAO 3.2: added Caveats section
12 Jan 2005 usage warning added to Overview; replaced Create a Multiple-Chip, Multiple Observation Exposure Map and Fluxed Image section with warning note
23 Jun 2005 CIAO 3.2.2 patch: bug fix to asphist means it is again possible to use the merge_all script to create exposure maps for multiple observations; updated images
19 Dec 2005 updated for CIAO 3.3: the merge_all script has been updated to version 3.3 for the new asphist tool syntax; corresponding changes to screen output
01 Dec 2006 updated for CIAO 3.4: version 3.4 of merge_all (removed "punlearn mkinstmap" to allow new CIAO 3.4 mkinstmap parameters (pbkfile, dafile) to be set before running the script, if desired; parameters supplied on the command line will not be prompted)
26 Feb 2007 merge_all v3.5: added a test to determine whether "energy" value is monochromatic or a spectrum file; temporary filenames have a timestamp value as the root, rather than hardwired names.
03 Mar 2007 added Caveat: extra GTIs in merged event files; added ACIS dead area correction section
16 Apr 2007 merge_all v3.6: "[subspace -expno]" filter was added to the dmmerge command in merge_all as a workaround for the extra GTIs in merged event files problem
18 Jan 2008 updated for CIAO 4.0: merge_all v3.8 (outdated ChIPS workaround replaced by dmtcalc, few instances of dmlist were replaced by dmkeypar), minor screen update to match; ACIS QE contamination link moved to "Related Links" section; filenames and screen output updated for reprocessed data (version N002 files); added "Additional exposure map and fluxed image threads" section; removed outdated calibration updates
03 Feb 2009 updated for CIAO 4.1: images are inline; "ARDLIB warning ... Assuming the first "interesting" extension." no longer printed; input data must have a CTI_APP keyword
06 May 2009 check the version of the CIAO scripts package instead of the individual script
01 Jun 2009 merge_all updated in 01 Jun 2009 scripts package: check that intdir directory exists and is writeable, run punlearn before running mkinstmap
25 Jun 2009 information on bug with offset observations added to Synopsis
12 Jan 2010 updated for CIAO 4.2: calibration update - the ACIS QE contamination model has been upgraded to vN0005.

Return to Threads Page: Top | All | Imag

Where are the PDFs?
Last modified: 12 Jan 2010