Chandra X-Ray Observatory
	(CXC)
Skip to the navigation links
Last modified: 26 October 2016

URL: http://cxc.harvard.edu/ciao/why/combine_image.html

Why combine data?

Due to scheduling constraints - that are getting more severe as Chandra ages - a long observation is likely to be performed as several, smaller, observations (each with a separate Observation Id). Many targets have also been observed by multiple observers, whether to go deeper, monitor sources, or to cover a larger area than possible with a single observation. In both cases, it would make life easier for you, as a user, to be able to combine all the observations into one, and use this for your analysis. Unfortunately, as described below and elsewhere on this site - e.g. the How & Why (or Why Not) to Combine Data document - this is often not the best solution, and can lead to incorrect results.

The time-varying response

The reason that data can not - in general - be simply combined with the dmmerge tool is that the response for a particular chip coordinate (i.e. where the photon strikes the detector) depends on time:

  1. within an observation, the telescope dither pattern means that the area of sky seen by each detector (pixels for ACIS, grid/channels for the HRC) varies with time;

  2. between observations, the telescope and detector properties can change - for instance the ACIS time-dependent gain and contamination model, or the HRC-I and HRC-S gain settings;

  3. the detector properties may even vary during an observation (due to scheduling constraints, the focal-plane temperature may vary significantly within an observation, which can change the ACIS detector response);

  4. the background may vary (e.g. due to changes in the intensity or spectrum of the particle background along Chandra's orbit).

For more information on instrumental issues please review the Chandra Data Caveats page.

Since the detector response - that is, terms Am(E), Rm(E,h), and Bm(h) in equation 1 of How & Why (or Why Not) to Combine Data - varies with time, then this variation would need to be accounted for when combining the responses (c.f. equation 3 Ibid.). Although tools like mkwarf and addresp can combine responses (whether implicitly or explicitly), they should not be used to create a response from a combined event file, as some of the information needed to track down all the time dependencies has been lost.

In the following we show the difference between "combining and then calculating the response" - as used in merge_all (tool withdrawn) - and "calculate the individual responses and then combine" - as used by merge_obs - when calculating exposure-corrected ACIS images. For simplicity we focus on the CSC broad band, that is 0.5 to 7.0 keV band with the exposure map evaluated at 2.3 keV.

The test data

In the following we shall use the 5 Chandra observations of 3C186, all ACIS-S, since

  1. the observations span a large time range - May 2002 to December 2007 - so the response of the ACIS detector changes significantly between them (in particular the change in the contamination layer);

  2. the second set, taken in December 2007, represent a 200 ks observation that had to be split up;

  3. for the combined 200 ks data, the first three observations - 9407, 9774, and 9775 - have the same SIM position, but the last one (9408) is slightly shifted, so there may be issues with combining data from all four observations;

  4. and the first observation (3098) was taken using only part of ACIS-S3 (as it was taken using the sub-array mode).

The analysis presented below was performed with CIAO 4.4 Tuesday, June 5, 2012, that is the CIAO 4.4.1 release, using CALDB version 4.5.3.

The data can be retrieved and reprocessed by saying:

unix% find_chandra_obsid 3c186
# obsid  sepn   inst grat   time    obsdate        piname    target
3098      0.0 ACIS-S NONE   37.9 2002-05-16 Siemiginowska Q0740+380
9407      0.0 ACIS-S NONE   67.2 2007-12-03 Siemiginowska  "3C 186"
9408      0.0 ACIS-S NONE   40.1 2007-12-11 Siemiginowska  "3C 186"
9774      0.0 ACIS-S NONE   76.1 2007-12-06 Siemiginowska  "3C 186"
9775      0.0 ACIS-S NONE   16.1 2007-12-08 Siemiginowska  "3C 186"
unix% find_chandra_obsid 3c186 download=all
... a lot of output ...
unix% chandra_repro 3098,9407,9408,9774,9775
Output directory (default = $indir/repro) (): 
# acis_process_events (CIAO 4.4): The following error occurred 13 times:
	dsAPEPULSEHEIGHTERR -- WARNING: pulse height is less than split threshold when performing serial CTI adjustment.
Output from acis_process_events:
# acis_process_events (CIAO 4.4): The following error occurred 4 times:
	dsAPEPULSEHEIGHTERR -- WARNING: pulse height is less than split threshold when performing serial CTI adjustment.
Output from acis_process_events:
# acis_process_events (CIAO 4.4): The following error occurred 13 times:
	dsAPEPULSEHEIGHTERR -- WARNING: pulse height is less than split threshold when performing serial CTI adjustment.
Output from acis_process_events:
# acis_process_events (CIAO 4.4): The following error occurred 3 times:
	dsAPEPULSEHEIGHTERR -- WARNING: pulse height is less than split threshold when performing serial CTI adjustment.
unix% punlearn reproject_obs
unix% reproject_obs '*/repro/*evt*' reproj/
Running reproject_obs
Version: 17 October 2012

Verifying 5 observations.
Calculating new tangent point.
New tangent point: RA=7h 44m 16.584s Dec=37d 54' 9.612"

Observations to be reprojected:

  Obsid  Obs Date   Exp    DETNAM     SIM_Z    FP   Sepn   PA  
                   (ks)                (mm)    (K)   (')  (deg)
---------------------------------------------------------------
1 3098  2002-05-16  34.4 ACIS-7      -190.143 153.3   1.5  +177
2 9407  2007-12-03  66.3 ACIS-23567  -190.140 154.1   0.4    -2
3 9774  2007-12-06  75.1 ACIS-23567  -190.140 154.4   0.4    -2
4 9775  2007-12-08  15.9 ACIS-23567  -190.140 153.6   0.4    -3
5 9408  2007-12-11  39.6 ACIS-23567  -190.143 153.8   0.4    -6

WARNING - EXPTIME values differ:
  Obsid 3098 has EXPTIME=0.4 and the rest have 3.1

Running tasks in parallel with 4 processors.
Reprojecting 5 event files to a common tangent point.
Merging reprojected events files to: reproj/merged_evt.fits

The following files were created:

The reprojected event files:
     reproj/3098_reproj_evt.fits
     reproj/9407_reproj_evt.fits
     reproj/9774_reproj_evt.fits
     reproj/9775_reproj_evt.fits
     reproj/9408_reproj_evt.fits

The merged event file:
     reproj/merged_evt.fits

Warning: the merged event file reproj/merged_evt.fits
   should not be used to create ARF/RMF/exposure maps because
      the RA_NOM keyword varies by 0.00256923286001 (limit is 0.0003)
      the DEC_NOM keyword varies by 0.030215177097 (limit is 0.0003)
      the ROLL_NOM keyword varies by 167.915572499 (limit is 1.0)
      the EXPTIME keyword contains: 0.4 3.1
        which means that the DTCOR value, and hence LIVETIME/EXPOSURE
        keywords are wrong
      the aim points fall on CCDs: 7
        which means that the ONTIME/LIVETIME/EXPOSURE keywords
        do not reflect the full observation length.

Not shown directly above - although it is mentioned in the output of reproject_obs - is the variation in the roll between the observations. As shown below, 3098 is rotated by almost 180 degrees from the other observations; 9407, 9774, and 9775 are very close; and 9408 is slightly different.

unix% foreach o ( 3098 9407 9774 9775 9408 )
foreach? echo "$o `dmkeypar $o/repro/acisf0${o}_repro_evt2.fits roll_nom echo+`" 
foreach? end
3098 270.57447599566
9407 102.65890349618
9774 102.65890960359
9775 102.65892363247
9408 103.70777695244

We use a combination of reproject_obs and flux_obs in the following - rather than just using merge_obs - to avoid repeated work, namely the re-projection of the event files.

The merged event file looks like the following, which shows the 0.5 to 7.0 keV energy range, and the sub-array used for ObsId 3098 is highlighted by the green polygon:

[The merged observations for 3C186, with a rectangle showing the location of ObsId 3098]
Figure 1:
unix% ds9 reproj/merged_evt.fits -smooth -scale log -bin factor 4 \
       -bin about 3900 4800 -zoom 0.5 -cmap b \
       -bin filter 'energy=500:7000' \
       -region 3098/primary/acisf03098_001N003_fov1.fits

Although the merged event file can be used for visualizing the data, it should not be used for data extraction (e.g. creating spectra) when knowledge of the telescope response is required.

Combining "identical" observations

We start by trying to combine the three observations that are "identical", namely 9407, 9774, and 9775. We start by using flux_obs to take the reprojected event files created by reproject_obs above, and create a combined exposure-corrected image:

unix% punlearn flux_obs
unix% pset flux_obs bin=4
unix% flux_obs 'reproj/9407_reproj_evt.fits,reproj/97*_reproj_evt.fits' simple/
Running flux_obs
Version: 17 October 2012

Verifying 3 observations.
Using CSC ACIS broad science energy band.
Calculating the output grid

The merged images will have 774 by 825 pixels, a pixel size of 1.968 arcsec,
    and cover x=2020.5:5116.5:4, y=3196.5:6496.5:4.

Creating the fluxed images for 3 observations in parallel.
Creating aspect histograms for obsid 9775
Creating aspect histograms for obsid 9774
Creating aspect histograms for obsid 9407
Creating 5 instrument maps for obsid 9775
Creating 5 instrument maps for obsid 9407
Creating 5 instrument maps for obsid 9774
Creating 5 exposure maps for obsid 9775
Exposure map limits: 0.000000e+00, 4.661458e+06
Writing exposure map to simple/9775_5_broad.expmap
Exposure map limits: 0.000000e+00, 3.929060e+06
Writing exposure map to simple/9775_3_broad.expmap
Exposure map limits: 0.000000e+00, 4.524344e+06
Writing exposure map to simple/9775_6_broad.expmap
Exposure map limits: 0.000000e+00, 3.939074e+06
Writing exposure map to simple/9775_2_broad.expmap
Exposure map limits: 0.000000e+00, 5.937810e+06
Writing exposure map to simple/9775_7_broad.expmap
Creating 5 exposure maps for obsid 9407
Exposure map limits: 0.000000e+00, 2.469366e+07
Writing exposure map to simple/9407_7_broad.expmap
Exposure map limits: 0.000000e+00, 1.881372e+07
Writing exposure map to simple/9407_6_broad.expmap
Exposure map limits: 0.000000e+00, 1.938741e+07
Writing exposure map to simple/9407_5_broad.expmap
Exposure map limits: 0.000000e+00, 1.637222e+07
Writing exposure map to simple/9407_2_broad.expmap
Exposure map limits: 0.000000e+00, 1.634182e+07
Writing exposure map to simple/9407_3_broad.expmap
Exposure map limits: 0.000000e+00, 2.800297e+07
Writing exposure map to simple/9774_7_broad.expmap
Exposure map limits: 0.000000e+00, 2.198827e+07
Writing exposure map to simple/9774_5_broad.expmap
Creating 5 exposure maps for obsid 9774
Exposure map limits: 0.000000e+00, 1.852836e+07
Writing exposure map to simple/9774_3_broad.expmap
Combining 5 exposure maps for obsid 9775
Combining 5 exposure maps for obsid 9407
Thresholding data for obsid 9775
Thresholding data for obsid 9407
Exposure-correcting image for obsid 9775
Exposure-correcting image for obsid 9407
Exposure map limits: 0.000000e+00, 1.857118e+07
Writing exposure map to simple/9774_2_broad.expmap
Exposure map limits: 0.000000e+00, 2.133700e+07
Writing exposure map to simple/9774_6_broad.expmap
Combining 5 exposure maps for obsid 9774
Thresholding data for obsid 9774
Exposure-correcting image for obsid 9774

Combining 3 observations.

The following files were created:

The co-added clipped counts image is:
     simple/broad_thresh.img

The co-added clipped exposure map is:
     simple/broad_thresh.expmap

The co-added exposure-corrected image is:
     simple/broad_flux.img

and compare it to the merge_all results (which we make match the positioning and size of the flux_obs output, and for which we decide to use the bad-pixel table from the 9774 observation, since it is the longest one):

unix% cat simple_all.evt2
9407/repro/acisf09407_repro_evt2.fits[energy=500:7000]
9774/repro/acisf09774_repro_evt2.fits[energy=500:7000]
9775/repro/acisf09775_repro_evt2.fits[energy=500:7000]
unix% cat simple_all.asol1 
9407/repro/pcadf313081138N002_asol1.fits
9774/repro/pcadf313307273N002_asol1.fits
9775/repro/pcadf313532704N002_asol1.fits
unix% acis_set_ardlib 9774/repro/acisf09774_repro_bpix1.fits 
Updated ardlib parameter file: /data/param/ardlib.par
  AXAF_ACIS0_BADPIX_FILE -> CALDB
  AXAF_ACIS1_BADPIX_FILE -> CALDB
  AXAF_ACIS2_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX2]
  AXAF_ACIS3_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX3]
  AXAF_ACIS4_BADPIX_FILE -> CALDB
  AXAF_ACIS5_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX5]
  AXAF_ACIS6_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX6]
  AXAF_ACIS7_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX7]
  AXAF_ACIS8_BADPIX_FILE -> CALDB
  AXAF_ACIS9_BADPIX_FILE -> CALDB
unix% get_sky_limits simple/broad_flux.img verbose=0
unix% pget get_sky_limits xygrid
2020.5:5116.5:#774,3196.5:6496.5:#825
unix% mkdir simple_all
unix% punlearn merge_all
unix% pset merge_all chip=2,3,5,6,7 energy=2.3
unix% pset merge_all refcoord=reproj/merged_evt.fits
unix% pset merge_all xygrid=2020.5:5116.5:#774,3196.5:6496.5:#825
unix% pset merge_all merged=simple_all/merged_event.fits
unix% pset merge_all expmap=simple_all/broad.expmap
unix% pset merge_all expcorr=simple_all/broad_flux.img
unix% merge_all @simple_all.evt2 @simple_all.asol1 dtffile= mode=h

Reprojecting 9407/repro/acisf09407_repro_evt2.fits to reproj/merged_evt.fits

Reprojecting 9774/repro/acisf09774_repro_evt2.fits to reproj/merged_evt.fits

Reprojecting 9775/repro/acisf09775_repro_evt2.fits to reproj/merged_evt.fits

Merging data...

SIM_Z of event file: -190.14006604987

SIM_Z of 9407/repro/pcadf313081138N002_asol1.fits: -190.14006604987
Difference is 0 mm: SIM_Z differs by less than ~1/4 pixel.

SIM_Z of 9774/repro/pcadf313307273N002_asol1.fits: -190.14006604987
Difference is 0 mm: SIM_Z differs by less than ~1/4 pixel.

SIM_Z of 9775/repro/pcadf313532704N002_asol1.fits: -190.14006604987
Difference is 0 mm: SIM_Z differs by less than ~1/4 pixel.

Creating aspect histogram for CCD 2...
Creating aspect histogram for CCD 3...
Creating aspect histogram for CCD 5...
Creating aspect histogram for CCD 6...
Creating aspect histogram for CCD 7...

Making instrument map for CCD 2...
Making instrument map for CCD 3...
Making instrument map for CCD 5...
Making instrument map for CCD 6...
Making instrument map for CCD 7...

Making exposure map for CCD 2...
Exposure map limits: 0.000000e+00, 4.044487e+07
Making exposure map for CCD 3...
Exposure map limits: 0.000000e+00, 4.028789e+07
Making exposure map for CCD 5...
Exposure map limits: 0.000000e+00, 4.601305e+07
Making exposure map for CCD 6...
Exposure map limits: 0.000000e+00, 4.613702e+07
Making exposure map for CCD 7...
Exposure map limits: 0.000000e+00, 5.862443e+07

Merging exposure maps

Creating image

Normalizing image

All done!

As a reminder, the exposure-corrected image was created by:

Below we use chips to view the exposure-corrected images created by both approaches, after applying light smoothing, logarithmic scaling, and using the same display range (the combine_image.py Python script defines the compare_data and plot_difference routines used below);

unix% chips combine_image.py
-----------------------------------------
Welcome to ChIPS: CXC's Plotting Package
-----------------------------------------
CIAO 4.4 ChIPS version 1 Tuesday, June 5, 2012

chips-2> compare_data('simple')
Warning: divide by zero encountered in log10
Warning: divide by zero encountered in log10
chips-3> print_window('compare_simple.png')
chips-4> quit()
[Other than bright pixels at the edge of the merge_all version, the images look the same]

Figure 2: The bright pixels at the edges of the chips in the merge_all output are where the exposure varies strongly within a pixel; they do not show up in the flux_obs output since they have been filtered out by the expmapthresh parameter. Other than this difference, the two images look identical, but there are some systematic differences, as shown in Figure 3.

and we can look at the percentage difference between the two, calculated as 100 * (flux_obs - merge_all) / flux_obs so that a positive value means that the flux_obs flux is larger than the merge_all one:

unix% chips combine_image.py
-----------------------------------------
Welcome to ChIPS: CXC's Plotting Package
-----------------------------------------
CIAO 4.4 ChIPS version 1 Tuesday, June 5, 2012

chips-2> plot_difference('simple')
Warning: divide by zero encountered in divide
chips-3> print_window('difference_simple.png')
chips-4> quit()
[Overall the results are similar, but there are systematic differences]

Figure 3: The results look similar, for instance most of ACIS-S3, which contains the cluster, is close to no difference, but you can see some systematic features, including:

If acis_set_ardlib (or manually setting the ardlib parameter file) is not used before calling merge_all, then the bad-pixel table used by merge_all will be taken from the CALDB, and the difference in results is shown in Figure 4:

[The major difference now is the difference in the bad columns]

Figure 4: The differences to Figure 3 are due to the fact that the CALDB bad-pixel table does not match the per-observation one. The most obvious difference is the bad columns, where there are more in the per-observation table, leading to an increased flux in the flux_obs version, but there are also individual bad pixels missing in the CALDB version, which give rise to the small squares that can be seen on all chips.

Combining similar observations

Here we extend the analysis to include the 9408 observation, which was taken only three days after 9775 but has a slightly-different SIM_Z value.

unix% punlearn flux_obs
unix% flux_obs 'reproj/9*_evt.fits' simz/ bin=4
Running flux_obs
Version: 17 October 2012

Verifying 4 observations.
Using CSC ACIS broad science energy band.
Calculating the output grid

The merged images will have 786 by 826 pixels, a pixel size of 1.968 arcsec,
    and cover x=2012.5:5156.5:4, y=3192.5:6496.5:4.

Creating the fluxed images for 4 observations in parallel.
Creating aspect histograms for obsid 9408
... a lot more of the same ...
Exposure-correcting image for obsid 9775

Combining 4 observations.

The following files were created:

The co-added clipped counts image is:
     simz/broad_thresh.img

The co-added clipped exposure map is:
     simz/broad_thresh.expmap

The co-added exposure-corrected image is:
     simz/broad_flux.img

and (again using the bad-pixel table from ObsId 9774):

unix% cat simz_all.evt2 
9407/repro/acisf09407_repro_evt2.fits[energy=500:7000]
9774/repro/acisf09774_repro_evt2.fits[energy=500:7000]
9775/repro/acisf09775_repro_evt2.fits[energy=500:7000]
9408/repro/acisf09408_repro_evt2.fits[energy=500:7000]
unix% cat simz_all.asol1 
9407/repro/pcadf313081138N002_asol1.fits
9774/repro/pcadf313307273N002_asol1.fits
9775/repro/pcadf313532704N002_asol1.fits
9408/repro/pcadf313765973N002_asol1.fits
unix% acis_set_ardlib 9774/repro/acisf09774_repro_bpix1.fits 
Updated ardlib parameter file: /data/param/ardlib.par
  AXAF_ACIS0_BADPIX_FILE -> CALDB
  AXAF_ACIS1_BADPIX_FILE -> CALDB
  AXAF_ACIS2_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX2]
  AXAF_ACIS3_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX3]
  AXAF_ACIS4_BADPIX_FILE -> CALDB
  AXAF_ACIS5_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX5]
  AXAF_ACIS6_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX6]
  AXAF_ACIS7_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX7]
  AXAF_ACIS8_BADPIX_FILE -> CALDB
  AXAF_ACIS9_BADPIX_FILE -> CALDB
unix% get_sky_limits simz/broad_flux.img verbose=0
unix% pget get_sky_limits xygrid
2012.5:5156.5:#786,3192.5:6496.5:#826
unix% mkdir simz_all
unix% punlearn merge_all
unix% pset merge_all chip=2,3,5,6,7 energy=2.3
unix% pset merge_all refcoord=reproj/merged_evt.fits
unix% pset merge_all xygrid=2012.5:5156.5:#786,3192.5:6496.5:#826
unix% pset merge_all merged=simz_all/merged_event.fits
unix% pset merge_all expmap=simz_all/broad.expmap
unix% pset merge_all expcorr=simz_all/broad_flux.img
unix% merge_all @simz_all.evt2 @simz_all.asol1 dtffile= mode=h

Reprojecting 9407/repro/acisf09407_repro_evt2.fits to reproj/merged_evt.fits

Reprojecting 9774/repro/acisf09774_repro_evt2.fits to reproj/merged_evt.fits

Reprojecting 9775/repro/acisf09775_repro_evt2.fits to reproj/merged_evt.fits

Reprojecting 9408/repro/acisf09408_repro_evt2.fits to reproj/merged_evt.fits

Merging data...

SIM_Z of event file: -190.14006604987

SIM_Z of 9407/repro/pcadf313081138N002_asol1.fits: -190.14006604987
Difference is 0 mm: SIM_Z differs by less than ~1/4 pixel.

SIM_Z of 9774/repro/pcadf313307273N002_asol1.fits: -190.14006604987
Difference is 0 mm: SIM_Z differs by less than ~1/4 pixel.

SIM_Z of 9775/repro/pcadf313532704N002_asol1.fits: -190.14006604987
Difference is 0 mm: SIM_Z differs by less than ~1/4 pixel.

SIM_Z of 9408/repro/pcadf313765973N002_asol1.fits: -190.14258036517
Difference is -0.00251431530000445 mm: SIM_Z differs by less than ~1/4 pixel.

Creating aspect histogram for CCD 2...
# asphist (CIAO 4.4): ERROR: *_PNT keywords are required in file 'simz_all/merged_event.fits[ccd_id=2]'.  Will assume PNT == NOM which may produce bad results for reprojected datasets and multi-obi obsids.

Creating aspect histogram for CCD 3...
# asphist (CIAO 4.4): ERROR: *_PNT keywords are required in file 'simz_all/merged_event.fits[ccd_id=3]'.  Will assume PNT == NOM which may produce bad results for reprojected datasets and multi-obi obsids.

Creating aspect histogram for CCD 5...
# asphist (CIAO 4.4): ERROR: *_PNT keywords are required in file 'simz_all/merged_event.fits[ccd_id=5]'.  Will assume PNT == NOM which may produce bad results for reprojected datasets and multi-obi obsids.

Creating aspect histogram for CCD 6...
# asphist (CIAO 4.4): ERROR: *_PNT keywords are required in file 'simz_all/merged_event.fits[ccd_id=6]'.  Will assume PNT == NOM which may produce bad results for reprojected datasets and multi-obi obsids.

Creating aspect histogram for CCD 7...
# asphist (CIAO 4.4): ERROR: *_PNT keywords are required in file 'simz_all/merged_event.fits[ccd_id=7]'.  Will assume PNT == NOM which may produce bad results for reprojected datasets and multi-obi obsids.


Making instrument map for CCD 2...
Making instrument map for CCD 3...
Making instrument map for CCD 5...
Making instrument map for CCD 6...
Making instrument map for CCD 7...

Making exposure map for CCD 2...
Exposure map limits: 0.000000e+00, 5.058647e+07
Making exposure map for CCD 3...
Exposure map limits: 0.000000e+00, 5.043441e+07
Making exposure map for CCD 5...
Exposure map limits: 0.000000e+00, 5.753366e+07
Making exposure map for CCD 6...
Exposure map limits: 0.000000e+00, 5.775658e+07
Making exposure map for CCD 7...
Exposure map limits: 0.000000e+00, 7.338729e+07

Merging exposure maps

Creating image

Normalizing image

All done!

which produces Figures 5 and 6 below.

unix% chips combine_image.py
-----------------------------------------
Welcome to ChIPS: CXC's Plotting Package
-----------------------------------------
CIAO 4.4 ChIPS version 1 Tuesday, June 5, 2012

chips-2> compare_data('simz')
Warning: divide by zero encountered in log10
Warning: divide by zero encountered in log10
chips-3> print_window('compare_simz.png')
chips-4> clear()
chips-5> plot_difference('simz')
Warning: divide by zero encountered in divide
chips-6> print_window('difference_simz.png')
chips-7> quit()
[This looks similar to Figure 2]

Figure 5: The image is very similar to Figure 2.

[There are differences to Figure 3 at the chip edges]

Figure 6: Overall this looks similar to Figure 3, as we would expect, but you can see some small, systematic, differences at the edges of the chips, caused by the small difference in SIM position and roll angle between ObsId 9408 and the other observations, as noted in the screen output created by merge_all.

So, when combining data which is "similar" - in that it was taken close enough together in time that the same calibration products can be used and the instrument details are very similar - e.g. the SIM position - then you can combine data before creating the response products, as merge_all does, with only small-scale, but systematic, differences. How much of an issue this is depends on your science goals.

Combining all the observations

Now we add in the 3098 observation, which we expect to cause problems for merge_all since it is significantly different to the other observations. In Figure 7 we show the merged event file, focusing just on ACIS-S3, which clearly shows the sub-array used in the first observation:

[The merged observations for 3C186, concentrating on just the ACIS-S3 chip]
Figure 7:
unix% ds9 reproj/merged_evt.fits -smooth -scale log -bin factor 4 \
       -bin about 4100 4000 -cmap b \
       -bin filter 'energy=500:7000,ccd_id=7'
unix% punlearn flux_obs
unix% flux_obs 'reproj/*_reproj_evt.fits' combined/ bin=4
Running flux_obs
Version: 17 October 2012

Verifying 5 observations.
Using CSC ACIS broad science energy band.
Calculating the output grid

The merged images will have 786 by 826 pixels, a pixel size of 1.968 arcsec,
    and cover x=2012.5:5156.5:4, y=3192.5:6496.5:4.

Creating the fluxed images for 5 observations in parallel.
Creating aspect histograms for obsid 9408
... more output ...
Exposure-correcting image for obsid 9774

Combining 5 observations.

The following files were created:

The co-added clipped counts image is:
     combined/broad_thresh.img

The co-added clipped exposure map is:
     combined/broad_thresh.expmap

The co-added exposure-corrected image is:
     combined/broad_flux.img

and

unix% cat combined_all.evt2 
3098/repro/acisf03098_repro_evt2.fits[energy=500:7000]
9407/repro/acisf09407_repro_evt2.fits[energy=500:7000]
9774/repro/acisf09774_repro_evt2.fits[energy=500:7000]
9775/repro/acisf09775_repro_evt2.fits[energy=500:7000]
9408/repro/acisf09408_repro_evt2.fits[energy=500:7000]
unix% cat combined_all.asol1 
3098/repro/pcadf137943492N003_asol1.fits
9407/repro/pcadf313081138N002_asol1.fits
9774/repro/pcadf313307273N002_asol1.fits
9775/repro/pcadf313532704N002_asol1.fits
9408/repro/pcadf313765973N002_asol1.fits
unix% acis_set_ardlib 9774/repro/acisf09774_repro_bpix1.fits 
Updated ardlib parameter file: /data/param/ardlib.par
  AXAF_ACIS0_BADPIX_FILE -> CALDB
  AXAF_ACIS1_BADPIX_FILE -> CALDB
  AXAF_ACIS2_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX2]
  AXAF_ACIS3_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX3]
  AXAF_ACIS4_BADPIX_FILE -> CALDB
  AXAF_ACIS5_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX5]
  AXAF_ACIS6_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX6]
  AXAF_ACIS7_BADPIX_FILE -> /data/9774/repro/acisf09774_repro_bpix1.fits[BADPIX7]
  AXAF_ACIS8_BADPIX_FILE -> CALDB
  AXAF_ACIS9_BADPIX_FILE -> CALDB
unix% get_sky_limits combined/broad_flux.img verbose=0
unix% pget get_sky_limits xygrid
2012.5:5156.5:#786,3192.5:6496.5:#826
unix% mkdir combined_all
unix% punlearn merge_all
unix% pset merge_all chip=2,3,5,6,7 energy=2.3
unix% pset merge_all refcoord=reproj/merged_evt.fits
unix% pset merge_all xygrid=2012.5:5156.5:#786,3192.5:6496.5:#826
unix% pset merge_all merged=combined_all/merged_event.fits
unix% pset merge_all expmap=combined_all/broad.expmap
unix% pset merge_all expcorr=combined_all/broad_flux.img
unix% merge_all @combined_all.evt2 @combined_all.asol1 dtffile= mode=h

Reprojecting 3098/repro/acisf03098_repro_evt2.fits to reproj/merged_evt.fits

Reprojecting 9407/repro/acisf09407_repro_evt2.fits to reproj/merged_evt.fits

Reprojecting 9774/repro/acisf09774_repro_evt2.fits to reproj/merged_evt.fits

Reprojecting 9775/repro/acisf09775_repro_evt2.fits to reproj/merged_evt.fits

Reprojecting 9408/repro/acisf09408_repro_evt2.fits to reproj/merged_evt.fits

Merging data...

SIM_Z of event file: -190.14258036517

SIM_Z of 3098/repro/pcadf137943492N003_asol1.fits: -190.14258036517
Difference is 0 mm: SIM_Z differs by less than ~1/4 pixel.

SIM_Z of 9407/repro/pcadf313081138N002_asol1.fits: -190.14006604987
Difference is 0.00251431530000445 mm: SIM_Z differs by less than ~1/4 pixel.

SIM_Z of 9774/repro/pcadf313307273N002_asol1.fits: -190.14006604987
Difference is 0.00251431530000445 mm: SIM_Z differs by less than ~1/4 pixel.

SIM_Z of 9775/repro/pcadf313532704N002_asol1.fits: -190.14006604987
Difference is 0.00251431530000445 mm: SIM_Z differs by less than ~1/4 pixel.

SIM_Z of 9408/repro/pcadf313765973N002_asol1.fits: -190.14258036517
Difference is 0 mm: SIM_Z differs by less than ~1/4 pixel.

Creating aspect histogram for CCD 2...
# asphist (CIAO 4.4): ERROR: *_PNT keywords are required in file 'combined_all/merged_event.fits[ccd_id=2]'.  Will assume PNT == NOM which may produce bad results for reprojected datasets and multi-obi obsids.

# asphist (CIAO 4.4): WARNING: '3098/repro/pcadf137943492N003_asol1.fits' has no records fall in event gti's.

Creating aspect histogram for CCD 3...
# asphist (CIAO 4.4): ERROR: *_PNT keywords are required in file 'combined_all/merged_event.fits[ccd_id=3]'.  Will assume PNT == NOM which may produce bad results for reprojected datasets and multi-obi obsids.

# asphist (CIAO 4.4): WARNING: '3098/repro/pcadf137943492N003_asol1.fits' has no records fall in event gti's.

Creating aspect histogram for CCD 5...
# asphist (CIAO 4.4): ERROR: *_PNT keywords are required in file 'combined_all/merged_event.fits[ccd_id=5]'.  Will assume PNT == NOM which may produce bad results for reprojected datasets and multi-obi obsids.

# asphist (CIAO 4.4): WARNING: '3098/repro/pcadf137943492N003_asol1.fits' has no records fall in event gti's.

Creating aspect histogram for CCD 6...
# asphist (CIAO 4.4): ERROR: *_PNT keywords are required in file 'combined_all/merged_event.fits[ccd_id=6]'.  Will assume PNT == NOM which may produce bad results for reprojected datasets and multi-obi obsids.

# asphist (CIAO 4.4): WARNING: '3098/repro/pcadf137943492N003_asol1.fits' has no records fall in event gti's.

Creating aspect histogram for CCD 7...
# asphist (CIAO 4.4): ERROR: *_PNT keywords are required in file 'combined_all/merged_event.fits[ccd_id=7]'.  Will assume PNT == NOM which may produce bad results for reprojected datasets and multi-obi obsids.


Making instrument map for CCD 2...
Making instrument map for CCD 3...
Making instrument map for CCD 5...
Making instrument map for CCD 6...
Making instrument map for CCD 7...

Making exposure map for CCD 2...
Exposure map limits: 0.000000e+00, 4.700833e+07
Making exposure map for CCD 3...
Exposure map limits: 0.000000e+00, 4.688738e+07
Making exposure map for CCD 5...
Exposure map limits: 0.000000e+00, 5.334176e+07
Making exposure map for CCD 6...
Exposure map limits: 0.000000e+00, 5.343070e+07
Making exposure map for CCD 7...
Exposure map limits: 0.000000e+00, 8.069480e+07

Merging exposure maps

Creating image

Normalizing image

All done!
unix% chips combine_image.py 
-----------------------------------------
Welcome to ChIPS: CXC's Plotting Package
-----------------------------------------
CIAO 4.4 ChIPS version 1 Tuesday, June 5, 2012

chips-2> compare_data('combined')
Warning: divide by zero encountered in log10
Warning: divide by zero encountered in log10
chips-3> print_window('compare_combined.png')
chips-4> clear()
chips-5> plot_difference('combined', loval=-20, hival=20)
Warning: divide by zero encountered in divide
chips-6> print_window('difference_combined.png')
chips-7> quit()
[Significant differences are now visible]

Figure 8: Unlike previous versions (e.g. Figure 2 and 5), the merge_all version shows significant differences to the flux_obs one, namely:

  1. a spatial offset which makes the exposure map not match up to the chip boundaries, causing the obvious banding between chips and at the edge of chips;

  2. the exposure map for 3098 includes the whole chip, rather than just the sub-array, in the merge_all version, which causes the area around the cluster to have an artificially lowered flux (in this figure this is most noticeable as the change in the brighness of the chip boundary in the ACIS-S2 and ACIS-S3 chips to the East of the cluster).

[There are now significant differences]

Figure 9: Here we see the problems discussed in Figure 8 - the spatial shift and the drastic change in the flux close to the cluster because of ObsId 3098 - but you can also see that there is an overall difference in the normalisation of the merge_all result, resulting in it producing fluxes 5 to 10 percent less than merge_obs. This is because the exposure maps created by merge_all use the response information for the first observation (3098) for the whole data set, and so does not include the reduced effective area due to the ACIS contamination layer for the 2007 observations, which form the bulk of the observing time.

This example shows some of the problems that can occur when trying to create response products - in this case the instrument and exposure maps - from a combined dataset (as was done by merge_all). The responses should be created per observation and then - if desired and possible - combined, as done by merge_obs and flux_obs.


Last modified: 26 October 2016
Smithsonian Institute Smithsonian Institute

The Chandra X-Ray Center (CXC) is operated for NASA by the Smithsonian Astrophysical Observatory. 60 Garden Street, Cambridge, MA 02138 USA.   Email:   cxchelp@head.cfa.harvard.edu Smithsonian Institution, Copyright © 1998-2017. All rights reserved.