Skip to the navigation links
Last modified: 28 Sept 2010
Where are the PDFs?

The ACIS "Blank-Sky" Background Files

CIAO 4.2 Science Threads

[Python Syntax]



Overview

Last Update: 28 Sep 2010 - updated for CALDB 4.3.1: calibration update - updated files for ACIS-8

Synopsis:

The ACIS calibration team has completed a detailed study of the ACIS background. Part of this study has produced a set of "blank-sky" datasets which can be used, for instance, for analyses where the object of interest fills the entire chip.

Purpose:

To create ACIS background spectra tailored to a specific observation for spectral fitting or images for spatial analyses.

Read this thread if:

you are analyzing an ACIS imaging observation and the source is so large that you can not estimate the background from your dataset.

Calibration Updates:

Related Links:




Contents



Get Started

Sample ObsIDs used: 1838 (ACIS-S, G21.5-09); 1712 (ACIS-S, 3C 273)

File types needed: evt2; asol1

The 1838 event file used in this thread has gone through Reprocessing III, so the newest gain and CTI corrections have been applied. In these examples, we shall restrict attention to the ACIS-S3 chip and the 0.5 to 7 keV energy range:

unix% dmcopy "acisf01838N002_evt2.fits[energy=500:7000,ccd_id=7]" evt2_c7.fits
unix% dmcopy "evt2_c7.fits[bin sky=8]" img8_c7.fits

ObsID 1712 is used as an example of a strong background flare; this is the same dataset used in the Filtering Lightcurves thread.

The acis_bkgrnd_lookup script requires that the input file contains a CCD_ID column and the CTI_APP, SIM_Z, and FP_TEMP header keywords. If your data has not been recently reprocessed, the FP_TEMP keyword may not be included in the level 2 event file. If it is missing from the file header, get the value from the evt1.fits file and use dmhedit to add it to the evt2.fits file, as explained in this FAQ. The ACIS CTI Correction why topic has detailed information on adding the CTI_APP keyword if it is missing.

Download the ACIS background files

The ACIS "blank-sky" background files are not included in the main CALDB download file, due to the substantial filesize. If you have not already installed them, you must download the background files tarfile from the CALDB Download page and install it before continuing with this thread.

The ACIS background files are installed in the $CALDB/data/chandra/acis/bkgrnd/ directory.


Download the scripts

This thread uses the acis_bkgrnd_lookup script and the lc_clean routine from the lightcurves.py module. The deflare script may also be used to run lc_clean.

The scripts are part of the CIAO Scripts distribution. The CIAO scripts package should be the following version or newer:

unix% cat $ASCDS_CONTRIB/VERSION.CIAO_scripts
05 Mar 2010

Please check that you have at least this version of the scripts package installed before continuing. If you do not have the scripts installed or need to update to a newer version, refer to the Scripts page.



Finding the Correct File

Naming scheme

The ACIS background datasets are stored in the CALDB at $CALDB/data/chandra/acis/bkgrnd/.

They are indexed by focal plane temperatures, aimpoint (i.e. ACIS-I or ACIS-S), and chip number:

unix% ls -1 $CALDB/data/chandra/acis/bkgrnd/
acis0D2000-12-01bgstow_ctiN0004.fits
acis0iD1999-09-16bkgrndN0002.fits
acis0iD2000-01-29bkgrnd_ctiN0005.fits
acis0iD2000-01-29bkgrndN0003.fits
acis0iD2000-12-01bkgrnd_ctiN0004.fits
acis0iD2000-12-01bkgrndN0002.fits
acis1D2000-12-01bgstow_ctiN0004.fits
acis1iD1999-09-16bkgrndN0002.fits
acis1iD2000-01-29bkgrnd_ctiN0005.fits
...

The naming scheme for CTI-corrected data is:

acis<chip><aimpoint>D<date>bkgrnd_ctiN<version>.fits

For uncorrected data, the naming is:

acis<chip><aimpoint>D<date>bkgrndN<version>.fits

The files which have "bgstow" in place of "bkgrnd" in the filename are particle-only background files created from data taken when the HRC was in the focal plane (e.g. ACIS was "stowed"). There is no ACIS-S/ACIS-I distinction for these data, so an "<aimpoint>" is not included in the filename.


Using acis_bkgrnd_lookup

The acis_bkgrnd_lookup script makes it easy to find an ACIS background file that matches your data. The script takes an input event file and returns a list of background files for all the ACIS chips in the file that have events on them:

unix% acis_bkgrnd_lookup evt2_c7.fits
/soft/ciao/CALDB/data/chandra/acis/bkgrnd/acis7sD2000-01-29bkgrnd_ctiN0001.fits

This file only has data on ACIS-7 (ACIS-S3), so a single file is returned. The path reflects where the CALDB is installed on your system; here, $CALDB is set to /soft/ciao/CALDB.

In addition to being printed to the screen, the list of background files is also stored in the outfile parameter:

unix% pget acis_bkgrnd_lookup outfile
/soft/ciao/CALDB/data/chandra/acis/bkgrnd/acis7sD2000-01-29bkgrnd_ctiN0001.fits

The input file can include a Data Model filter as long as the resulting file is still a table with a ccd_id column, such as "acisf01838N001_evt2.fits[ccd_id=7]". In this example, we use a spatial region to filter the original event file. The resulting background file(s) will be chosen to match the CCDs that are covered by the circle:

unix% acis_bkgrnd_lookup "acisf01838N002_evt2.fits[sky=circle(6072.5,4632.5,320)]"
/soft/ciao/CALDB/data/chandra/acis/bkgrnd/acis1sD2000-01-29bkgrnd_ctiN0005.fits
/soft/ciao/CALDB/data/chandra/acis/bkgrnd/acis3sD2000-01-29bkgrnd_ctiN0005.fits

Two files are returned, so the outfile parameter contains a comma-separated list:

unix% pget acis_bkgrnd_lookup outfile
/soft/ciao/CALDB/data/chandra/acis/bkgrnd/acis1sD2000-01-29bkgrnd_ctiN0005.fits,
/soft/ciao/CALDB/data/chandra/acis/bkgrnd/acis3sD2000-01-29bkgrnd_ctiN0005.fits

No matching file and identical lookup cases

There are a number of known cases where acis_bkgrnd_lookup cannot find a background file for the dataset and exits with this error:

# acis_bkgrnd_lookup: ERROR: Unable to find an ACIS background file for:
infile=<filename>
CCD_ID=<N> 

There are no background files for:

  • non-CTI-corrected data on ACIS-1

  • any data on ACIS-4

  • CTI-corrected data on ACIS-9

  • ACIS-0 data, when ACIS-S is in the focal plane (a SIM_Z limit is exceeded)

There are also a few cases that result in identical lookup results:

  • For the front-illuminated (FI) chips, there is no difference between:

    • CTI_APP = PPPPPNPNPP
    • CTI_APP = PPPPPBPBPP

    since the parallel CTI-correction is applied to all FI chips either way.

  • For the back-illuminated (BI) chips, there is no difference between any of these:

    • CTI_APP = NNNNNNNNNN
    • CTI_APP = PPPPPNPNPP
    • CTI_CORR = YES
    • CTI_CORR = NO

    since no CTI correction is applied to BI chips for any of those configurations.



Make a Local Copy of the Background File

The background file for the evt2_c7.fits dataset is $CALDB/data/chandra/acis/bkgrnd/acis7sD2000-01-29bkgrnd_ctiN0001.fits. Make a local copy of the file:

unix% cp \
      $CALDB/data/chandra/acis/bkgrnd/acis7sD2000-01-29bkgrnd_ctiN0001.fits \
      bgevt2_c7.fits

The tailor the background file to your dataset section explains what data processing is required in order to use the background in the analysis. A "cleaned" event file is needed for that step, so we first have to remove any background flares from the event data.



Remove Background Flares from the Event File

As discussed in the ACIS background page, the background datasets were created for quiescent-background periods. It is therefore important that you filter your dataset to remove periods of background flares. The selection of a "true" background can be critical, as discussed in M. Markevitch's memo (astro-ph/0205333).

We use the lc_clean routine to calculate a GTI for the dataset.

Remove bright sources

The background fields were filtered using a maximum rate that is 20 percent higher than the quiescent background level. It is therefore necessary to remove any bright sources from your dataset. For this example we have chosen the regions to exclude by eye, using ds9 and the image img8_c7.fits, saving the file in CIAO format. The resulting region file - sources.reg - is shown on the image in Figure 1 and listed below:

[Thumbnail image: The regions defined in the file sources.reg are overlaid on a ds9 display of the image]

[Version: full-size]

[Print media version: The regions defined in the file sources.reg are overlaid on a ds9 display of the image]

Figure 1: Source regions overlaid on the image

The regions defined in region file sources.reg overlaid on the image img8_c7.fits in ds9.

unix% cat sources.reg
# Region file format: CIAO version 1.0
circle(4072.5,4252.5,400)
circle(4604.5,4804.5,48)

If any flux is left from the source, you should reduce the max_scale parameter of lc_clean() to account for this residual flux. For example, compare the expected background rate from the background files with those of your dataset within the same spatial region.

Now we create a copy of the event file, excluding the sources, using the DM filtering syntax:

unix% dmcopy "evt2_c7.fits[exclude sky=region(sources.reg)]" evt2_c7_bg.fits

Create the lightcurve

Note: in this example, we do not apply any energy filter to the lightcurve; we use the full energy range of the dataset (0.3-10 keV here). This is not necessarily the optimal energy band to use for your observation.

The dmextract tool is used to create the lightcurve of the selected region. First, get the GTI interval from the event file:

unix% dmlist evt2_c7_bg.fits"[GTI7]" data

------------------------------------------------
Data for Table Block GTI7
------------------------------------------------

ROW    START                STOP

     1  84245789.1549994648  84253741.1550068706

These times will be used as the lower and upper limits for extracting the lightcurve. The same binning as the background dataset - 259.28 seconds - is used, so the filter is:

[bin time=84245789.155:84253741.155:259.28]

The filter could also be defined as "[bin time=::259.28]", in which case the TSTART and TSTOP header values are used for the endpoints. This approach is valid, though it will probably create some empty bins in the output lightcurve, since that time range is longer than the good time interval.

Run dmextract to create the lightcurve:

unix% punlearn dmextract
unix% pset dmextract infile="evt2_c7_bg.fits[bin time=84245789.155:84253741.155:259.28]"
unix% pset dmextract outfile=evt2_c7_bg.lc
unix% pset dmextract opt=ltc1
unix% dmextract
Input event file  (evt2_c7_bg.fits[bin time=84245789.155:84253741.155:259.28]):
Enter output file name (evt2_c7_bg.lc):

The file evt2_c7_bg.lc contains the lightcurve for the data. You can check the parameter file that was used with plist dmextract.


Use the lc_clean() routine

The lc_clean() routine from the Python module lightcurves.py selects those regions of the lightcurve that do not contain flares. The algorithm used is taken from the lc_clean program created by Maxim Markevitch, and is different from that used by the lc_sigma_clip() routine in the Filtering Lightcurves thread.

[New] The deflare script provides a command-line interface to lc_clean. There is an example of using deflare at the end of this section; see the ahelp file for more information.

The lightcurves module must be loaded into an interactive session of chips or ipython before it can be run (this step is only necessary once per session). In the example below we use ChIPS:

unix% chips
-----------------------------------------
Welcome to ChIPS: CXC's Plotting Package
-----------------------------------------
CIAO 4.2 Tuesday, July 6, 2010

chips> from lightcurves import *

If lc_clean() is called with just the name of the lightcurve file then a diagnostic plot will be produced (Figure 2) and output displayed on the screen:

chips> lc_clean("evt2_c7_bg.lc")
Parameters used to clean the lightcurve are:
  script version = CIAO 4.2 - 1.0
  mean           = None
  clip           = 3
  scale          = 1.2
  minfrac        = 0.1
  plot           = True
  rateaxis       = y
  color          = green

Total number of bins in lightcurve   = 31
Max length of one bin                = 255.997 s
Num. bins with a smaller exp. time   = 1
Num. bins with exp. time = 0         = 0
Calculated an initial mean (sigma-clipped) rate of 0.269808 ct/s
Lightcurve limits use a scale factor of 1.2 about this mean
Filtering lightcurve between rates of 0.22484 and 0.32377 ct/s
Number of good time bins = 24
Rate filter:  0.22484 <= count_rate < 0.32377 
Mean level of filtered lightcurve = 0.264842 ct/s
GTI limits calculated using
    (count_rate>0.22484 && count_rate<0.32377)

The corresponding times are:
  ((time >= 84246048.435000) && (time < 84247604.115000)) ; 1.54 ksec, bin 1
  ((time >= 84248641.235000) && (time < 84249159.795000)) ; 0.51 ksec, bin 2
  ((time >= 84249419.075000) && (time < 84252011.875000)) ; 2.56 ksec, bin 3
  ((time >= 84252271.155000) && (time < 84253826.835000)) ; 1.45 ksec, bin 4

  Exposure time of lightcurve = 7.85 ks
  Filtered exposure time      = 6.06 ks
  DTCOR value                 = 0.987337
[Thumbnail image: The top plot shows the light curve, with the time values on the abscissa, and the bottom plot a histogram of the count-rate values.]

[Version: full-size, PNG, postscript]

[Print media version: The top plot shows the light curve, with the time values on the abscissa, and the bottom plot a histogram of the count-rate values.]

Figure 2: Plot from lc_clean

The top plot shows the light curve of evt2_c7_bg.lc, where the time axis is displayed on the x axis as a relative value (the first bin in the light curve is used as the 0 point). Points that are in green are those that passed the sigma-clipping and the horizontal dotted line shows the mean value of these points (the numerical value is given in the plot title).

The bottom plot shows a histogram of the count rate values; the green version again shows the data that passes the sigma-clipping algorithm.

In order to create a GTI file, we need to specify an output filename; here we use "evt2_c7_bg.gti". The command also creates Figure 3.

chips> lc_clean("evt2_c7_bg.lc", outfile="evt2_c7_bg.gti") 
Parameters used to clean the lightcurve are:
  script version = CIAO 4.2 - 1.0
  mean           = None
  clip           = 3
  scale          = 1.2
  minfrac        = 0.1
  outfile        = evt2_c7_bg.gti
  plot           = True
  rateaxis       = y
  color          = green
  pattern        = solid
  pattern color  = red

Total number of bins in lightcurve   = 31
Max length of one bin                = 255.997 s
Num. bins with a smaller exp. time   = 1
Num. bins with exp. time = 0         = 0
Calculated an initial mean (sigma-clipped) rate of 0.269808 ct/s
Lightcurve limits use a scale factor of 1.2 about this mean
Filtering lightcurve between rates of 0.22484 and 0.32377 ct/s
Number of good time bins = 24
Rate filter:  0.22484 <= count_rate < 0.32377 
Mean level of filtered lightcurve = 0.264842 ct/s
GTI limits calculated using
    (count_rate>0.22484 && count_rate<0.32377)

The corresponding times are:
  ((time >= 84246048.435000) && (time < 84247604.115000)) ; 1.54 ksec, bin 1
  ((time >= 84248641.235000) && (time < 84249159.795000)) ; 0.51 ksec, bin 2
  ((time >= 84249419.075000) && (time < 84252011.875000)) ; 2.56 ksec, bin 3
  ((time >= 84252271.155000) && (time < 84253826.835000)) ; 1.45 ksec, bin 4

  Exposure time of lightcurve = 7.85 ks
  Filtered exposure time      = 6.06 ks
  DTCOR value                 = 0.987337

Creating GTI file
Created: evt2_c7_bg.gti
[Thumbnail image: The top plot includes the regions excluded by the GTI file.]

[Version: full-size, PNG, postscript]

[Print media version: The top plot includes the regions excluded by the GTI file.]

Figure 3: Plot from lc_clean when outfile is set

The top plot now indicates the regions excluded by the GTI file created by lc_clean - here evt2_c7_bg.gti - using a solid red pattern.

Since we specified an output file, the lc_clean() function has run dmgti, using the calculated range, and created the file evt2_c7_bg.gti. You can check the parameter file that was used with plist dmgti.

Running lc_clean via the deflare script

[New] The deflare script provides a command-line interface to lc_clean. Run deflare with method=clean to choose the lc_clean cleaning method; here we repeat the example from this section:

unix% deflare evt2_c7_bg.lc evt2_c7_bg_deflare.gti method=clean
Parameters used to clean the lightcurve are:
  script version = CIAO 4.2 - 1.1
  mean           = None
  clip           = 3
  scale          = 1.2
  minfrac        = 0.1
  outfile        = evt2_c7_bg_deflare.gti
  plot           = False

Total number of bins in lightcurve   = 31
Max length of one bin                = 255.997 s
Num. bins with a smaller exp. time   = 1
Num. bins with exp. time = 0         = 0
Calculated an initial mean (sigma-clipped) rate of 0.269808 ct/s
Lightcurve limits use a scale factor of 1.2 about this mean
Filtering lightcurve between rates of 0.22484 and 0.32377 ct/s
Number of good time bins = 24
Rate filter:  0.22484 <= count_rate < 0.32377 
Mean level of filtered lightcurve = 0.264842 ct/s

GTI limits calculated using a count-rate filter:
  (count_rate>0.22484 && count_rate<0.32377)

The corresponding times are:
  ((time >= 84246048.435000) && (time < 84247604.115000)) ; 1.54 ksec, bin 1
  ((time >= 84248641.235000) && (time < 84249159.795000)) ; 0.51 ksec, bin 2
  ((time >= 84249419.075000) && (time < 84252011.875000)) ; 2.56 ksec, bin 3
  ((time >= 84252271.155000) && (time < 84253826.835000)) ; 1.45 ksec, bin 4

  Exposure time of lightcurve = 7.85 ks
  Filtered exposure time      = 6.06 ks
  DTCOR value                 = 0.987337

Creating GTI file
Created: evt2_c7_bg_deflare.gti
Light curve cleaned using the lc_clean routine.
Created: evt2_c7_bg_deflare.gti

To create a plot of the filter, set plot=yes when running deflare.


Filter the event list

We now filter the event list using the new GTI information, and check to see how the exposure time has changed

chips> !dmcopy "evt2_c7.fits[@evt2_c7_bg.gti]" evt2_c7_clean.fits
chips> !dmkeypar evt2_c7.fits EXPOSURE echo+
7851.3069951925
chips> !dmkeypar evt2_c7_clean.fits EXPOSURE echo+
6059.3291110124

An example of a strong flare

The previous example showed an essentially stable background level. ObsID 1712, which is also used in the Filtering Lightcurves, contains a major flare event. First we create a lightcurve of the background of this observation:

unix% dmcopy "acisf01712N003_evt2.fits[energy=500:7000,ccd_id=7]" evt2_flare.fits

unix% cat exclude.reg 
# Region file format: CIAO version 1.0
rotbox(4200.3328,4137.9892,1129.5056,74.07019,24.22333)
circle(4076.5,4088.5,316)
circle(4296.5,5024.5,48)

unix% punlearn dmextract
unix% pset dmextract infile="evt2_flare.fits[exclude sky=region(exclude.reg)][bin time=77379634.145:77407432.550:259.28]"
unix% pset dmextract outfile=flare.lc
unix% pset dmextract opt=ltc1
unix% dmextract
Input event file  (evt2_flare.fits[exclude sky=region(exclude.reg)][bin time=77379634.145:77407432.550:259.28]):
Enter output file name (flare.lc):

When lc_clean() is run on this lightcurve, the routine finds that the filtered lightcurve contains less than ten percent of the original exposure, and so stops (this limit can be changed by setting the option minfrac to something other than 0.1). The diagnostic plot that is created (Figure 4) shows that the default options do not do a good job with this lightcurve. The log_scale calls are used to change the scaling of the rate axes in both plots.

chips> lc_clean("flare.lc")
Parameters used to clean the lightcurve are:
  script version = CIAO 4.2 - 1.0
  mean           = None
  clip           = 3
  scale          = 1.2
  minfrac        = 0.1
  plot           = True
  rateaxis       = y
  color          = green

Total number of bins in lightcurve   = 108
Max length of one bin                = 255.997 s
Num. bins with a smaller exp. time   = 2
Num. bins with exp. time = 0         = 0
Calculated an initial mean (sigma-clipped) rate of 1.40236 ct/s
Lightcurve limits use a scale factor of 1.2 about this mean
Filtering lightcurve between rates of 1.16863 and 1.68283 ct/s
Number of good time bins = 2
Rate filter:  1.16863 <= count_rate < 1.68283
Mean level of filtered lightcurve = 1.40236 ct/s

Warning: Default bin width of 0.01 count/s is too small as it produces
         1099 bins.
         Replacing with a width of 0.0109962 count/s
         This may indicate that the lightcurve contains strong flares that
         require manual filtering.

StandardError: Fraction of bins that are good (0.0185185, 2 of 108) is below limit of 0.1!

chips> log_scale(X_AXIS)
chips> log_scale("rate",Y_AXIS)
[Thumbnail image: The presence of the strong flare towards the end of the observation means that the analysis fails to find enough good points.]

[Version: full-size, PNG, postscript]

[Print media version: The presence of the strong flare towards the end of the observation means that the analysis fails to find enough good points.]

Figure 4: A lightcurve with a strong flare in it

The strong flare means that the routine does not pick up the "quiescent" level in this obeservation. Since the X axis of the bottom plot is linked to the Y axis of the top plot, the log_scale has changed the scaling in both plots (e.g. compare to Figure 2).

The rate histogram in Figure 4 suggests a value of 0.35 count/s, so we use the mean option to force this value:

chips> lc_clean("flare.lc", mean=0.35, outfile="flare.gti")
Parameters used to clean the lightcurve are:
  script version = CIAO 4.2 - 1.0
  mean           = 0.35
  clip           = 3
  scale          = 1.2
  minfrac        = 0.1
  outfile        = flare.gti
  plot           = True
  rateaxis       = y
  color          = green

Total number of bins in lightcurve   = 108
Max length of one bin                = 255.997 s
Num. bins with a smaller exp. time   = 2
Num. bins with exp. time = 0         = 0
Using a fixed mean rate of 0.35 ct/s
Lightcurve limits use a scale factor of 1.2 about this mean
Filtering lightcurve between rates of 0.291667 and 0.42 ct/s
Number of good time bins = 42
Rate filter:  0.291667 <= count_rate < 0.42 
Mean level of filtered lightcurve = 0.348777 ct/s

Warning: Default bin width of 0.01 count/s is too small as it produces
         1099 bins.
         Replacing with a width of 0.0109962 count/s
         This may indicate that the lightcurve contains strong flares that
         require manual filtering.

GTI limits calculated using a count-rate filter:
  (count_rate>0.291667 && count_rate<0.42)

The corresponding times are:
  ((time >= 77379893.425000) && (time < 77380930.545000)) ; 1.02 ksec, bin 1
  ((time >= 77382745.505000) && (time < 77383782.625000)) ; 1.02 ksec, bin 2
  ((time >= 77384041.905000) && (time < 77385338.305000)) ; 1.28 ksec, bin 3
  ((time >= 77385597.585000) && (time < 77387153.265000)) ; 1.54 ksec, bin 4
  ((time >= 77387671.825000) && (time < 77389486.785000)) ; 1.79 ksec, bin 5
  ((time >= 77390523.905000) && (time < 77390783.185000)) ; 0.26 ksec, bin 6
  ((time >= 77391561.025000) && (time < 77392338.865000)) ; 0.77 ksec, bin 7
  ((time >= 77394931.665000) && (time < 77395968.785000)) ; 1.02 ksec, bin 8
  ((time >= 77396228.065000) && (time < 77396487.345000)) ; 0.26 ksec, bin 9
  ((time >= 77396746.625000) && (time < 77397524.465000)) ; 0.77 ksec, bin 10
  ((time >= 77397783.745000) && (time < 77398561.585000)) ; 0.77 ksec, bin 11
  ((time >= 77406339.985000) && (time < 77406599.265000)) ; 0.26 ksec, bin 12

  Exposure time of lightcurve = 27.45 ks
  Filtered exposure time      = 10.75 ks
  DTCOR value                 = 0.987337

Creating GTI file
Created: flare.gti

chips> limits(X_AXIS, AUTO, 0.5)
chips> current_plot("rate")
chips> limits(Y_AXIS, AUTO, 0.5)

The resulting plot (Figure 5) shows that more data is included but that the data is still noisy, which suggests that care probably needs to be taken in interpreting the results.

[Thumbnail image: More points have been selected by the algorithm.]

[Version: full-size, PNG, postscript]

[Print media version: More points have been selected by the algorithm.]

Figure 5: Setting the mean option when filtering a light curve

Forcing the initial calculation to use a mean level of 0.35 count/s has resulted in more points being marked as good. However, the curve is still noisy and many times are excluded, as indicated by the regions of solid red color.

The use of AUTO for the minimum value in the limits call means that the limit is calculated from the data.

See the lc_clean help file for more information on changing the filtering scheme used by the routine. Maxim Markevitch's Cookbook contains advice on filtering lightcurves with strong flares.



Tailor the background file to your dataset

Analysing datasets with multiple chips

In CALDB 3.4.0, the background data files were divided so that there is one chip per file. This was done so that a unique background file is returned for each chip that contains data in the event file.

In order to make a background event file that contains more than one chip, it is necessary to run dmmerge to combine the files. For example:

unix% dmmerge \
      "acis6sD2000-01-29bkgrnd_ctiN0005.fits,acis7sD2000-01-29bkgrnd_ctiN0001.fits,acis8sD2000-01-29bkgrnd_ctiN0001.fits" \
      merge_bg.fits 

Be sure to read the caveat about the time keywords in merged headers, and correct your file.

Then continue with the Matching calibration to the event data step.


Matching calibration to the event data

The background files were generated using a gain file that may not match the one used for your observation or may not be up to date. The GAINFILE keyword in the header of the background files should match that of your source file:

unix% dmkeypar evt2_c7_clean.fits GAINFILE echo+
acisD2000-01-29gain_ctiN0006.fits

unix% dmkeypar bgevt2_c7.fits GAINFILE echo+
acisD2000-01-29gain_ctiN0006.fits

The two gain files should have the same date and version number in the filename, as shown here. If they do not match, reprocess the background file with acis_process_events, specifying the exact gainfile that was used on the data. You will also need to create a custom event definition that does not contain a time definition (e.g. take the standard eventdef and remove "d:time"). If the time specification is left in, acis_process_events will create a column where every event has a time equal to TSTART; this will cause problems later in the analysis.

unix% acis_process_events infile=bgevt2_c7.fits outfile=bgevt2_c7_newgain.fits \
      acaofffile=NONE stop="none" doevtgrade=no apply_cti=yes apply_tgain=no \
      calculate_pi=yes \
      gainfile=$CALDB/data/chandra/acis/det_gain/acisD2000-01-29gain_ctiN0006.fits \
      eventdef="{s:ccd_id,s:node_id,i:expno,s:chip,s:tdet,f:det,f:sky,s:phas,l:pha,l:pha_ro,f:energy,l:pi,s:fltgrade,s:grade,x:status}"

unix% dmkeypar bgevt2_c7_newgain.fits GAINFILE echo+
acisD2000-01-29gain_ctiN0006.fits

acis_process_events will print several warnings that may be ignored. Read the acis_process_events ahelp file for more details on using this tool.

CTI and TGAIN Calibration Files

The calibration files used to apply the CTI and time-dependent gain corrections should also match in the event and background data. In practice, though, it is not currently possible to apply newer CTI and TGAIN correction the background files in the CALDB; they lack some columns required by acis_process_events.

The error introduced by this mismatched calibration should not be very significant for the faint, extended objects for which people use these backgrounds.


Add the PNT header keywords

The pointing (PNT) header keyword values in the background files are set to zero:

unix% dmlist bgevt2_c7.fits header |grep PNT
0058 RA_PNT                                  0           Real8        Pointing RA
0059 DEC_PNT                                 0           Real8        Pointing Dec
0060 ROLL_PNT                                0 [deg]     Real8        Roll (fake)

These values indicate where the optical axis was during the observation. Having zero-values will cause the CIAO tool dmcoords to produce incorrect results when run with the background file.

To avoid this problem later in the analysis, the PNT values from the event file are copied into the background file header. This is done by creating a file of the event file header keywords with dmmakepar, extracting the PNT values, then adding them to the background file header with dmreadpar:

unix% dmmakepar evt2_c7_clean.fits event_header.par

unix% grep _pnt event_header.par > event_pnt.par

unix% cat event_pnt.par
ra_pnt,r,h,278.38657206626,,,"Pointing RA"
dec_pnt,r,h,-10.589269915682,,,"Pointing Dec"
roll_pnt,r,h,266.80752046013,,,"Pointing Roll"

unix% chmod +w bgevt2_c7.fits
unix% dmreadpar event_pnt.par "bgevt2_c7.fits[events]" clobber+

It is necessary to specify the "[events]" extension of the background file, or the keywords will be added to the NULL block header. The "clobber+" setting tells dmreadpar to overwrite the existing (zero) header PNT values.


Reproject the background data

The reproject_events tool can handle the special case of reprojecting a file which does not have an input time column, such as the ACIS background file. The tool takes the time range covered by the aspect solution of the observation and intersects it with the GTIs from the match file (i.e. the cleaned event file, evt2_c7_clean.fits), which means that the output file will only contain events for those chips contained in the match file. A random time within the range of the background data subspace is selected and assigned to the event for the reprojection. The time is used only for the reprojection; the output file does not have a TIME column in it.

Note that reproject_events does not alter the duration of the background file in any way in this process. The time-related header keywords (e.g. TSTART, TSTOP) are identical in the input and output files. This preserves the original background count rate, regardless of what observation is matched.

ObsID 1838 has only one aspect solution file, which we assume to be in the current working directory. It is possible to use multiple asol files by using a stack for the aspect parameter; see this FAQ for more information. The random parameter is set to a value other than -1. This tells the tool to apply a random aspect solution to each background event, thus sampling the dither of the observation.

unix% punlearn reproject_events
unix% pset reproject_events infile=bgevt2_c7.fits
unix% pset reproject_events outfile=bgevt2_c7_reproj.fits
unix% pset reproject_events aspect=pcadf084244404N001_asol1.fits
unix% pset reproject_events match=evt2_c7_clean.fits
unix% pset reproject_events random=0
unix% reproject_events
Input dataset/block specification (bgevt2_c7.fits):
Output dataset/block specification (bgevt2_c7_reproj.fits):
Match file (evt2_c7_clean.fits):

You can check the parameter file that was used with plist reproject_events.

The file bgevt2_c7_reproj.fits can now be used to estimate the background of your observation.

Once you have created a background spectrum or image, you may wish to change the value of the EXPOSURE keyword to account for the time dependence of the background. You may also wish to use the high energy data (above 10 keV) to see how the particle background normalization differs between your observation and the background.



Caveats

Working with VFAINT data

When working with observations performed in very faint (VFAINT) mode which have had the ACIS background cleaning performed, a similarly-filtered blank-sky background file may be used.

The period D background files are composed of VFAINT mode observations and have the "potential background event" status bit set to 1 (status bit 23). As no other status bits are set in the background files, it is simple to filter out these events:

dmcopy "bg.fits[status=0]" bg_cleaned.fits

Creating response files

If you are just going to subtract the background spectrum (i.e. not fit it), then you do not need a separate RMF and ARF for the background spectrum. You may simply use dmextract to create the spectrum.

If you do need an RMF file for the background region:

mkacisrmf should be used to make the RMF for any dataset which meets the requirements listed in mkacisrmf thread.

For all other data, use mkrmf. First run acis_fef_lookup to find the FEF file, giving it the source event file (not the background file) as the infile parameter. Find the coordinates of the source in chip coordinates (see the Step-by-Step Guide to Creating ACIS Spectra for Pointlike Sources thread for an example of how to do this). This filename should be used (including any DM filter) for the infile parameter of mkrmf.

The same FEF filename and filter should be used for the feffile parameter in mkwarf (i.e. instead of "CALDB").


Soft galactic flux and nH

The background datasets are constructed from a set of observations which are spread out across the sky. There is therefore no guarantee that the background at the position of your observation will match that in the background maps. An external check on the background can be made using the ROSAT All-Sky Survey R4-R5 band images. The Cookbook discusses these issues further.


Bad pixel maps

The background files were created using a more conservative list of bad pixels than the early processing of the data, but reprocessed Chandra data should match more closely.

The total area of the detector covered by hot pixels is very small, so different pixel filtering is not a concern, unless a bright hot pixel is left in the user's data. Bad columns are more important and ideally, the same bad pixel and column list should be applied to the data and the background, and also used for making the exposure maps. This excerpt was taken from the bad pixels and columns section of the Cookbook; refer to that document for more details.




Parameters for /home/username/cxcds_param/dmextract.par


#--------------------------------------------------------------------
#
# DMEXTRACT -- extract columns or counts from an event list
#
#--------------------------------------------------------------------
        infile = evt2_c7_bg.fits[bin time=84245789.155:84253741.155:259.28] Input event file 
       outfile = evt2_c7_bg.lc    Enter output file name
          (bkg = )                Background region file or fixed background (counts/pixel/s) subtraction
        (error = gaussian)        Method for error determination(poisson|gaussian|<variance file>)
     (bkgerror = gaussian)        Method for background error determination(poisson|gaussian|<variance file>)
      (bkgnorm = 1.0)             Background normalization
          (exp = )                Exposure map image file
       (bkgexp = )                Background exposure map image file
      (sys_err = 0)               Fixed systematic error value for SYS_ERR keyword
          (opt = ltc1)            Output file type: pha1 
     (defaults = ${ASCDS_CALIB}/cxo.mdb -> /soft/ciao/data/cxo.mdb) Instrument defaults file
         (wmap = )                WMAP filter/binning (e.g. det=8 or default)
      (clobber = no)              OK to overwrite existing output file(s)?
      (verbose = 0)               Verbosity level
         (mode = ql)              
    


Parameters for /home/username/cxcds_param/dmgti.par


        infile = evt2_c7_bg.lc    Input MTL file
       outfile = evt2_c7_bg.gti   Output GTI file
     userlimit = (count_rate>0.418866 && count_rate<0.603168) User defined limit string
      (mtlfile = none)            Optional output smoothed/filtered MTL file
     (lkupfile = none)            Lookup table defining which MTL columns to check against (NONE|none|<filename>)
       (smooth = yes)             Smooth the input MTL data?
      (clobber = yes)             Clobber output file if it exists?
      (verbose = 0)               Debug level
         (mode = h)               
    


Parameters for /home/username/cxcds_param/reproject_events.par


        infile = bgevt2_c7.fits Input dataset/block specification
       outfile = bgevt2_c7_reproj.fits Output dataset/block specification
         match = evt2_c7_clean.fits Match file
       (aspect = pcadf084244404N001_asol1.fits) Aspect file
       (random = 0)               random seed (0 use time, -1 no randomize)
      (geompar = geom)            Parameter file for Pixlib Geometry files
      (verbose = 0)               Debug Level(0-5)
      (clobber = no)              Clobber existing file
         (mode = ql)              
    

History

05 Jan 2005 updated for CIAO 3.2: updates for CALDB 3.0.0, added Download the Background Files section
12 Jul 2005 acis_bkgrnd_lookup updated to version 1.11. A bug has been fixed that prevented the script from storing the name of the selected ACIS background file, or files, in its parameter file.
20 Dec 2005 updated for CIAO 3.3: default value of dmextract error and bkgerror parameters is "gaussian"
16 Feb 2006 added using the background files with specextract caveat
01 Dec 2006 updated for CIAO 3.4: serial CTI released in CALDB 3.3.0 is not included in the CALDB background files; dmgti uses the value of the TIMEPIXR header keyword to adjust start and stop times (users may see a small shift in the time filter when compared to CIAO 3.3 because of this); kernel parameter removed from dmgti; CIAO version in errors and warnings; ChIPS version
01 May 2007 updated for CALDB 3.4.0: new ACIS background files; majority of thread rewritten to reflect the new background files
12 Jul 2007 acis_process_events command needs apply_cti=yes, to retain the CTI correction on the input file
14 Sep 2007 updated for CALDB 3.4.1: new ACIS background files
25 Jan 2008 updated for CIAO 4.0: acis_bkgrnd_lookup v1.12; lc_clean.sl v1.10 (plotting routines have been removed from the script until the ChIPS syntax is updated); slsh version 0.8.2-0/S-Lang version 2.1.3; screen output updated for reprocessed data; lightcurve extract uses GTI range, not header time range
06 May 2008 changed energy filter to 0.5 to 7 keV (500:7000)
24 Jun 2008 updated image display to place figures inline with text
15 Dec 2008 updated for CIAO 4.1: lc_clean.sl has been replaced by lightcurves.sl; added a Python version of the script; the method for specifying options to the command has changed; plotting capabilities have been restored; the location of the ACIS background files has changed
14 Jan 2009 acis_bkgrnd_lookup updated to version "CIAO 4.1 - 1.1" (additional functionality which does not affect how the script is used in this thread); updated error message when no matching file found
06 May 2009 check the version of the CIAO scripts package instead of the individual script
21 May 2009 added the Working with VFAINT data caveat
07 Jul 2009 updated for CALDB 4.1.3: correction to lookup of three background files.
20 Aug 2009 expanded the information on how the aspect solution is applied to the background file (Reproject the background data section)
27 Jan 2010 updated for CIAO 4.2: update the PNT header keywords in the background file header
05 Mar 2010 added information on using deflare script; updated script tarfile version to 05 Mar 2010
05 Apr 2010 updated for CALDB 4.2.1: calibration update - new ACIS blank-sky background files released
19 Apr 2010 updated for CALDB 4.2.2: calibration update - CALDB 4.2.1 contained one incorrect background file
28 Sep 2010 updated for CALDB 4.3.1: calibration update - updated files for ACIS-8

Return to Threads Page: Top | All | Data Prep | Imag | Imag Spec

Where are the PDFs?
Last modified: 28 Sept 2010