Computing Average HRC Dead Time Corrections
CIAO 4.16 Science Threads
Overview
Synopsis:
HRC deadtime corrections are determined as a function of time from detector total event and valid event counters (usually sampled every 2.05 seconds but may differ, depending on telemetry format) and written to a deadtime factor (dtf1) file. The average deadtime correction (DTCOR) for an observation is computed from the dtf1 file, filtered by the relevant good time intervals, and is applied to the corresponding ONTIME to compute the LIVETIME (and EXPOSURE) of the observation. If the user applies different time filters then the DTCOR, LIVETIME, and EXPOSURE keyword values should be updated to match.
Purpose:
To recompute deadtime statistics from an HRC deadtime factor file (dtf1.fits) with different time filters from those used in standard processing, and to use them to update the DTCOR, LIVETIME, and EXPOSURE of the observation.
This thread is needed to correct the count rates (counts/sec) when doing photometry. Light curves and exposure maps take the dtf1.fits file in as a separate input and do not require the DTCOR keyword value be updated in this way.
Run this thread if:
The event file was filtered on time, changing the good time intervals (GTIs), e.g. as a result of application of a background. Time filters may also be applied if the deadtime factors have been flagged as variable in the standard deadtime statistics (std_dtfstat1) file.
This thread is also required if your data was processed with software version 7.6.4 through 7.6.7: a bug in HRC standard processing led to the use of incorrect good time intervals (GTIs) in the calculation of DTCOR in the dtfstats file, and hence the LIVETIME and EXPOSURE. Get Started shows how to determine what version of the software has been applied to your data.
Related Links:
- About the Instrument: HRC
- Caveat: HRC Deadtime Correction with Telemetry Saturation
Last Update: 24 Jan 2022 - Review for CIAO 4.14. Updated for Repro-5 and CALDB 4.9.6
Contents
- Get Started
- Examine the deadtime corrections
- Recompute the deadtime statistics
- Update the Event File
- Parameter files:
- History
- Images
Get Started
Download the sample data: 3698 (HRC-I, CAS A)
unix% download_chandra_obsid 3698 evt2,dtf
This thread should be done after any necessary reprocessing is completed (e.g. running chandra_repro).
In this thread, we assume that all relevant files are in the same working directory.
The software version is stored in the ASCDSVER header keyword:
unix% dmkeypar hrcf03698_000N006_dtf1.fits ASCDSVER echo+ 10.10
This example will use a time filter on the event file.
unix% dmcopy "hrcf03698N006_evt2.fits[time=164259750:164261850]" filter_evt2.fits
The time range corresponds to a period of quiescent background. We can check that the time filter updated the ONTIME and related keywords; however, the DTCOR value is not automatically updated
unix% dmlist hrcf03698N006_evt2.fits header,clean | egrep 'DTCOR|ONTIME|LIVETIME|EXPOSURE' ONTIME 5171.3814679682 [s] DTCOR 0.755343479530 Dead time correction LIVETIME 3906.1692719921 Ontime multiplied by DTCOR EXPOSURE 3906.1692719921 Total exposure time, with all known corr. appl. unix% dmlist filter_evt2.fits header,clean | egrep 'DTCOR|ONTIME|LIVETIME|EXPOSURE' ONTIME 2100.0 [s] DTCOR 0.755343479530 Dead time correction LIVETIME 1586.2213070130 Ontime multiplied by DTCOR EXPOSURE 1586.2213070130 Total exposure time, with all known corr. appl.
Examine the deadtime corrections
We can find the average, mean, deadtime correction in the dtf1.fits file using dmstat:
unix% dmstat "hrcf03698_000N006_dtf1.fits[cols dtf]" DTF min: 0 @: 1 max: 1 @: 2 mean: 0.76564416062 sigma: 0.270005164 sum: 2081.0208286 good: 2718 null: 0
This value, 0.766, is slightly different from the value in the header of the event file
unix% dmkeypar filter_evt2.fits DTCOR echo+ 0.75534347953
The value in the event file is computed with hrc_dtfstats which uses data during the good time intervals and is computed using a good-time weighted average.
The other thing to note is the large standard deviation (sigma) reported by dmstat. This indicates that the DTF values are highly variable. It can therefore be useful to examine the files in ChIPS:
unix% python >>> import matplotlib.pylab as plt >>> from pycrates import read_file >>> >>> infile="hrcf03698_000N006_dtf1.fits" >>> tab = read_file(infile) >>> tt = tab.get_column("time") >>> dd = tab.get_column("dtf") >>> >>> xlab = tt.name + (" ({})".format(tt.unit) if len(tt.unit) else "") >>> ylab = dd.name + (" ({})".format(dd.unit) if len(dd.unit) else "") >>> >>> plt.plot(tt.values, dd.values) >>> plt.xlabel(xlab) >>> plt.ylabel(ylab) >>> plt.title(infile) >>> plt.axvline(164259750, color="red") >>> plt.axvline(164261850, color="red") >>> >>> plt.savefig("plot.png")
The plot is shown Figure 1. The dead time factors are highly variable due to a combination of the bright extended source and mildly varying background.
[Version: full-size]
Figure 1: Plot of deadtime correction value vs. time
Since a time filter was applied over a time range where the DTF values are different from the mean, the DTCOR value must be recomputed.
Recompute the deadtime statistics
The new GTI created by the time filter needs to be used to recompute the deadtime statistics using the hrc_dtfstats tool. This tool takes the existing dtf1.fits file and creates a new file with the event GTI information applied.
unix% punlearn hrc_dtfstats unix% pset hrc_dtfstats infile=hrcf03698_000N006_dtf1.fits unix% pset hrc_dtfstats outfile=dtf.stats unix% pset hrc_dtfstats gtifile=filter_evt2.fits unix% hrc_dtfstats Input file (hrcf03698_000N006_dtf1.fits): Output file (dtf.stats): File containing GTI to filter on (<filename>|NONE) (filter_evt2.fits):
The contents of the parameter file may be checked using plist hrc_dtfstats.
The new dtfstats file has a value of DTCOR=0.964
unix% dmlist dtf.stats"[cols DTCOR]" data,clean # DTCOR 0.96380777655308
which agrees with the range of values plotted in Figure 1.
Update the Event File
Finally, we update the event list header with dmhedit.
unix% dmhedit filter_evt2.fits file= op=add key=DTCOR value=0.963807
The LIVETIME and ONTIME also need to be corrected for the change in DTCOR. They will automatically get updated whenever any time filter is applied to the file. We can use an empty filter to trigger the update
unix% dmcopy filter_evt2.fits"[time=:]" dtcorr_evt2.fits
We can check the results using dmdiff
unix% dmdiff filter_evt2.fits dtcorr_evt2.fits data=no comments=no Infile 1: filter_evt2.fits Infile 2: dtcorr_evt2.fits ---------------------------------------------------------------------- Compare Headers ---------------------------------------------------------------------- Compare Key Lists: Compare Keyword Details: Compare Keyword Values: Keyword: Message: Value(s): Diff: -------- -------------------------------------- ---------------------------------- ------------------------ CHECKSUM Values are not equal Y6eha3cfZ3cfa3cf 7e1R7b1Q7b1Q7b1Q HISTNUM Values are not equal 485 492 +7 (+1.44%) LIVETIME Values are not equal 1586.221307013 2023.9963307615 +437.775 (+27.6%) EXPOSURE Values are not equal 1586.221307013 2023.9963307615 +437.775 (+27.6%) ...
Which shows the expected change in the LIVETIME and the EXPOSURE keywords.
The corrected event list, dtcorr_evt2.fits, should be used for all further analysis.
Parameters for /home/username/cxcds_param/hrc_dtfstats.par infile = hrcf03698_000N006_dtf1.fits Input file outfile = dtf.stats Output file gtifile = filter_evt2.fits File containing GTI to filter on (<filename>|NONE) (lookupTab = ${ASCDS_CALIB}/dmmerge_header_lookup.txt -> /soft/ciao/data/dmmerge_header_lookup.txt) lookup table (maincol = DTF) Name of the deadtime factor column (errcol = DTF_ERR) Name of the deadtime factor error column (chisqlim = 5) Limit for the variability test (clobber = no) Clobber the output file if it exists (verbose = 0) Verbose level. (mode = ql)
History
01 Dec 2006 | new for CIAO 3.4 |
25 Jan 2008 | updated for CIAO 4.0: updated ChIPS plotting; added dmstat command; new lookupTab parameter for header merging rules |
24 Jun 2008 | updated image display to place figures inline with text |
06 Feb 2009 | updated for CIAO 4.1: Python and S-Lang syntax included for ChIPS plotting |
05 Feb 2010 | updated for CIAO 4.2: ChIPS version |
13 Jan 2011 | reviewed for CIAO 4.3: no changes |
15 Feb 2011 | added link to the HRC Deadtime Correction with Telemetry Saturation caveat |
28 Jun 2011 | The CIAO 4.3 version of hrc_dtfstats should not be run on data processed with SDP version DS 8.4 (released 28 June 2011) or higher |
15 Dec 2011 | reviewed for CIAO 4.4: the hrc_dtfstats tool computes a simple average instead of an error-weighted average; minor updates to screen output. This software update makes the tool consistent with standard data processing (SDP); running this thread on recently-processed datasets will no longer introduce a regression into the data processing. |
03 Dec 2012 | Review for CIAO 4.5 |
02 Dec 2013 | Review for CIAO 4.6 |
15 Jul 2014 | Updated to use different OBS_ID that shows time variation within an observation. |
18 Dec 2014 | Reviewed for CIAO 4.7; no changes. |
02 Apr 2019 | Update to use matplotlib plotting. |
24 Jan 2022 | Review for CIAO 4.14. Updated for Repro-5 and CALDB 4.9.6 |