Skip to the navigation links
Last modified:

URL: http://cxc-newtest.cfa.harvard.edu/ciao4.2/combine_spectra.html
AHELP for CIAO 4.2

combine_spectra

Context: contrib

Synopsis

Combine multiple PHA imaging spectra and associated ARF responses

Syntax

combine_spectra src_spectra outroot [src_arfs] [src_rmfs] [bkg_spectra]
[bkg_arfs] [bkg_rmfs] [clobber] [verbose]

Description

The combine_spectra script sums multiple imaging source PHA spectra, and optionally, associated background PHA spectra and source and background ARF instrument responses. When a CIAO tool for combining RMF instrument responses becomes available, the script will also combine input RMF files. To run the script, it is only necessary to enter a list of PHA files in the 'src_spectra' parameter and specify a root name for output files in the 'outroot' parameter; in this case, background spectra and instrument response files will be combined based on information contained in the headers of the input source spectra. Please read the "Caveats" section below before conducting scientific analysis with the output of this script.

combine_spectra is a Python script which may be downloaded from the CIAO contributed scripts package, and executed on the Unix command line within the CIAO environment. The CIAO tools dmcopy, dmpaste, dmtcalc, dmarfadd, dmhedit, and dmappend are invoked by combine_spectra.

combine_spectra:

  • calculates a simple sum of input source PHA spectra to produce a single combined source spectrum
  • calculates an exposure-weighted sum of input or located source ARF responses to produce a single combined source ARF
  • calculates an area- and exposure-weighted sum of input or located background PHA spectra to produce a single combined background spectrum
  • calculates an expsoure-weighted sum of input or located background ARF responses to produce a single combined background ARF

Counts in the individual input source spectra are simply summed, without weighting, to produce the output combined source spectrum. Background spectra, however, are weighted before being summed in order to account for differences in exposure time and spectral extraction region area between each source and background spectrum. A combined source or background ARF instrument response output by the script is an exposure-weighted sum of the individual ARFs, appropriate for analysis with a corresponding combined spectrum which includes the total exposure time, and not the mean exposure. Input RMF responses are not yet combined by the script, but are accepted; see the "Caveats" section for details.

The required input to combine_spectra includes a list of at least two OGIP-compliant FITS PHA spectral data files, entered into the 'src_spectra' parameter, and a root name for the output combined file(s), specified in the 'outroot' parameter. If the remaining file input parameters are left at the default value of "None", the script will inspect the headers of the input source spectral files for associated ARF (and RMF) files to combine, as well as background PHA spectra and their associated instrument responses. Otherwise, the script will combine the files explicitly entered into the 'src_arfs', 'src_rmfs', 'bkg_spectra', 'bkg_arfs', and 'bkg_rmfs' fields. If the source response files and background spectral files recorded in the headers of the input source spectral files cannot be located - or the corresponding header keywords have null values - these files will not be combined by the script. At a minimum, the input source PHA spectra will be combined.

In order to combine background ARF files - either by explicitly listing filenames in the 'bkg_arfs' parameter or by having the code automatically locate the files when this parameter is left empty - a list of background spectra must be available for combining. This may be done by explicitly entering background spectra into the 'bkg_spectra' parameter, or by ensuring that the BACKFILE header keyword in all input source spectral files contains the appropriate filename. This also applies to the 'bkg_rmfs' parameter.

FORMAT OF INPUT FILES

The combine_spectra script should be compatible with any OGIP-compliant PHA spectral file containing at least one FITS HDU (CXC Data Model block) associated with an HDUCLAS1 header key value equal to 'SPECTRUM', which contains CHANNEL, PI, and COUNTS columns. It is also required that EXPOSURE and BACKSCAL header keywords exist in all input spectra, and that any ARF or RMF files to be combined contain an EXPOSURE header keyword.

WARNING: As this script is incompatible with ARF files which do not contain an EXPOSURE header keyword, those created with XMM SAS tools may cause the script to exit with an error. In this case, the appropriate exposure value should be added to the header of the ARF file.

HEADER KEYWORDS OF OUTPUT FILES

The sum of the exposure times of all input source spectra is recorded in the EXPOSURE header keyword of the combined source spectrum. The BACKSCAL value is set to 1.0.

The sum of the exposure times of all input or located backgroud spectra is recorded in the EXPOSURE header keyword of the combined background spectrum. The BACKSCAL header keyword value is computed according to the formula in the section "OUTPUT COMBINED BACKGROUND SPECTRUM".

The ANCRFILE and BACKFILE header keywords of the output combined source spectrum are updated with the names of the combined source ARF and combined background spectrum, if these files were created; will be "NONE" if not.

The ANCRFILE header keyword of the combined background spectrum is updated with the name of the combined background ARF, if this file was created; will be "NONE" if not.

The script copies the first source RMF file in an input (or located) list to the name of a 'combined' source RMF, which is recorded in the RESPFILE header keyword of the output combined source spectrum; will be "NONE" if no RMF files are available.

The script copies the first background RMF file in an input (or located) list to the name of a 'combined' background RMF, which is recorded in the RESPFILE header keyword of the output combined background spectrum; will be "NONE" if no RMF files are available.

Caveats

  • Combined ARF files produced by combine_spectra prior to the 12 October 2010 release of the updated CIAO contributed scripts package have incorrectly scaled SPECRESP column values. Users who combined an ARF with the earlier version of the script should remake it to get the correct output values. Othwerwise, a combined ARF file created with the older version of the script may be fixed by dividing the SPECRESP effective area values in the output combined ARF by the number of files which were combined, e.g., using the CIAO tool dmtcalc as follows:

dmtcalc infile=combined_src.arf outfile=combined_src.arf expression="SPECRESP=SPECRESP/3." clobber=True verbose=0

where, in this example, three files were combined.

  • While combine_spectra accepts lists of source and background RMF files, until a CIAO tool for combining RMF files becomes available, it will merely copy the first RMF in a user-input (or located) list to the filename of a 'combined' RMF file, i.e., '<outroot>_src.rmf' or '<outroot>_bkg.rmf'; RMF data will not actually be combined. This should not present a problem to the user, however, as separate spectral data sets and ARF responses should not be combined if the same RMF does not apply to each. If the RMFs are significantly different, it is advised that the individual spectra and ARF responses be analyzed separately.

  • Any grouping flags which may be present in input source or background PHA spectra will be ignored by the script.

OUTPUT COMBINED SOURCE SPECTRUM

A combined PHA source spectrum output by the script consists of CHANNEL, PI, COUNTS, and COUNT_RATE columns contained within a SPECTRUM block, with the COUNTS column containing the total, unweighted sum of the counts of the individual input source spectra. The COUNT_RATE column is computed as the total summed counts divided by the total summed exposure time of the individual spectra. The total exposure time is recorded in the EXPOSURE header keyword of the combined source spectrum, and the BACKSCAL value is set to 1.0.

SOURCE_COUNTS = src_counts1 + src_counts2 + ... + src_countsN

SOURCE_EXPOSURE = src_exp1 + src_exp2 + ... + src_expN

OUTPUT COMBINED BACKGROUND SPECTRUM

The counts in the individual background spectra to be combined are weighted before being summed in order to account for differences in exposure time and spectral extraction region area between each source and background spectrum. The scale factor, f_N, used for weighting each background spectrum prior to combining, is computed according to the following formula:

f_N = {(bkg_exp1 + ... + bkg_expN)/[src_exp1(src_backscal1/bkg_backscal1) + ... + src_expN(src_backscalN/bkg_backscalN)]}*{(src_expN/bkg_expN)*(src_backscalN/bkg_backscalN)}

BGD_COUNTS = f_1*bkg_counts1 + f_2*bkg_counts2 + ... f_N*bkg_countsN

The exposure times of the individual background spectra are summed and recorded in the EXPOSURE header keyword of the combined background spectrum. The BACKSCAL header keyword value is computed according to the following formula:

BGD_BACKSCAL = (src_exp1 + ... + src_expN)/[src_exp1(src_backscal1/bkg_backscal1) + ... + src_expN(src_backscalN/bkg_backscalN)]

The filename of the combined background spectrum is recorded in the BACKFILE header keyword of the combined source spectrum.

Combining multiple source and background spectra according to the steps outlined above allows for simultaneous fitting of the combined source and combined background spectrum - or background subtraction - in a spectral fitting program such as Sherpa or XSpec. Refer to the HEASARC documentation for a detailed explanation of how to derive the formulae shown above.

STATISTICAL ERRORS

Statistical errors on the combined counts in the combined source and background spectra are not computed by the script, as it is left to the X-ray fitting program chosen to fit the data to assign uncertainties to each spectral bin. When Sherpa, for example, fits a PHA data set, the statistical errors on the counts are automatically calculated using the chosen fit statistic. Users are warned against combining background spectra with wildly varying spectral extraction region areas, as this may yield misleading uncertainty estimates; i.e., some extractions will be over-represented while others will be under-represented.

OUTPUT COMBINED ARF FILE(S)

Source and background ARF files input to combine_spectra are combined using the CIAO tool dmarfadd, which computes the exposure-weighted sum of the individual ARF SPECRESP values to create a single output ARF file; see the dmarfadd ahelp file for details. While dmarfadd uses the *mean* exposure time in calculating an exposure-weighted combined ARF, combine_spectra overrides this step and uses instead the *sum* of the exposure times of the individual ARF files. This ensures that the combined ARF is compatible with a combined PHA spectrum containing the simple sum of the counts of its constituent PHA spectra, as well as the summed (and not a mean) exposure time. Refer to the ACIS Extract documentation for a detailed explanation.

The filename of a combined ARF is recorded in the ANCRFILE header keyword of the corresponding combined spectrum (source or background).

Example 1

combine_spectra src_spectra=obs1843.pi,obs1842.pi outroot="combined"

The counts columns in the PHA source imaging spectral files obs1843.pi and obs1842.pi are summed and written to the combined output spectrum 'combined_src.pi'. The source ARF files, RMF files, and associated background spectral files are read from the ANCRFILE, RESPFILE, BACKFILE keywords in the headers of the input source spectra and combined, producing the output files 'combined_src.arf', 'combined_src.rmf', and 'combined_bkg.pi'. If background spectra are located for each input source spectrum, their headers are searched for background ARF and RMF files to combine. If the ARF and RMF filenames recorded in the header of a background spectrum are identical to those recorded in the corresponding source spectrum header, the combined responses for the source are copied to the filenames 'combined_bkg.arf' and 'combined_bkg.rmf'.

Example 2

combine_spectra src_spectra=/data/pha1.pi,/data/pha2.pi
outroot="summed" bkg_spectra=pha1_bkg.pi,pha2_bkg.pi
bkg_arfs=@pha_arf.lis

The input source spectra 'pha1.pi' and 'pha2.pi' are combined to produce the summed counts spectrum 'summed_src.pi'. The script locates the associated source ARF and RMF files for these spectra and combines them, producing output files 'summed_src.arf' and 'summed_src.rmf'. Instead of having the script search for the associated background spectra and background ARFs to combine, they are explicitly entered into the 'bkg_spectra' and 'bkg_arfs' parameters.

Example 3

combine_spectra src_spectra=@pha3.lis outroot="combined" verbose=2

An ASCII file containing a list of Level=3 PHA spectral filenames is input to combine_spectra for combining. The script locates the associated source responses, background spectra, and background responses, and combines these files as well. The verbosity level is changed from the default value of 1 to 2, in order to print to the screen each step of the code as it is carried out.

Parameters

name type ftype def min max reqd
src_spectra file input       yes
outroot string input None     yes
src_arfs file   None     no
src_rmfs file   None     no
bkg_spectra file   None     no
bkg_arfs file   None     no
bkg_rmfs file   None     no
clobber boolean   no     no
verbose integer   1 0 5 no

Detailed Parameter Descriptions

Parameter=src_spectra (file required filetype=input)

The list of filenames of source PHA imaging spectra to combine.

A comma-separated list of OGIP-compliant PHA spectral files may be entered directly, or as a stack (see "ahelp stack" for more information). Enter non-Chandra PHA spectra with caution, as the expected header keywords, FITS HDUs, or FITS table column names may be incompatible with the CIAO tools invoked by the script.

Parameter=outroot (string required filetype=input default=None)

The root name to use for output files.

The string entered into this parameter is prepended to the filenames of combined spectra and responses output by the script. The combined spectrum file is named '<outroot>_src.pi', the combined source ARF is named '<outroot>_src.arf', and so on.

Parameter=src_arfs (file not required default=None)

The list of filenames of source ARF files to combine.

A comma-separated list of ARF response files may be entered directly, or as a stack (see "ahelp stack" for more information). All files should have the same energy range and binning. Enter non-Chandra ARF response files with caution, as the expected header keywords, FITS HDUs, or FITS table column names may be incompatible with the CIAO tools invoked by the script. If this parameter is left null, the script will try to locate source ARF files to combine by reading the ANCRFILE header keyword in each of the input source PHA spectra. If the ANCRFILE header keyword in each source spectrum is not null, and the referenced files exist, these files will be combined.

Parameter=src_rmfs (file not required default=None)

The list of filenames of source RMF files to combine.

A comma-separated list of RMF response files may be entered directly, or as a stack (see "ahelp stack" for more information). Enter non-Chandra RMF response files with caution, as the expected header keywords, FITS HDUs, or FITS table column names may be incompatible with the CIAO tools invoked by the script. If this parameter is left null, the script will try to locate source RMF files by reading the RESPFILE header keyword in each of the input source PHA spectra. If the RESPFILE header keyword in each source spectrum is not null, and the referenced files exist, the filename of the first source RMF in the input (or located) list will be copied to '<outroot>_src.rmf' (until a tool for combining RMF files becomes available).

Parameter=bkg_spectra (file not required default=None)

The list of filenames of background PHA spectra to combine.

A comma-separated list of background PHA spectral files may be entered directly, or as a stack (see "ahelp stack" for more information). Enter non-Chandra PHA files with caution, as the expected header keywords, FITS HDUs, or FITS table column names may be incompatible with the CIAO tools invoked by the script. If this parameter is left null, the script will try to locate background PHA files to combine by reading the BACKFILE header keyword in each of the input source PHA spectra. If the BACKFILE header keyword in each source spectrum is not null, and the referenced files exist, these files will be combined.

Parameter=bkg_arfs (file not required default=None)

The list of filenames of background ARF files to combine.

A comma-separated list of ARF response files may be entered directly, or as a stack (see "ahelp stack" for more information). All files should have the same energy range and binning. Enter non-Chandra ARF response files with caution, as the expected header keywords, FITS HDUs, or FITS table column names may be incompatible with the CIAO tools invoked by the script. If this parameter is left null, the script will try to locate background ARF files to combine by reading the ANCRFILE header keyword in each of the input (or located) background PHA spectra. If the ANCRFILE header keyword in each background spectrum is not null, and the referenced files exist, these files will be combined.

Parameter=bkg_rmfs (file not required default=None)

The list of filenames of background RMF files to combine.

A comma-separated list of RMF response files may be entered directly, or as a stack (see "ahelp stack" for more information). Enter non-Chandra RMF response files with caution, as the expected header keywords, FITS HDUs, or FITS table column names may be incompatible with the CIAO tools invoked by the script. If this parameter is left null, the script will try to locate background RMF files by reading the RESPFILE header keyword in each of the input (or located) background PHA spectra. If the RESPFILE header keyword in each background spectrum is not null, and the referenced files exist, the filename of the first background RMF in the input (or located) list will be copied to '<outroot>_bkg.rmf' (until a tool for combining RMF files becomes available).

Parameter=clobber (boolean not required default=no)

Overwrite existing output files?

Parameter=verbose (integer not required default=1 min=0 max=5)

Controls the amount of information printed to the screen.

Verbose can be from 0 to 5, generating different amounts of output. The default value is 1, which will print a minimal amount of information to the screen, e.g., which files have been located, which will be combined, and any warning or error messages which may occur from incompatibilities between the user input and the script. A verbosity level of 2 or higher will print each step of the script to the screen as it is being carried out, e.g., calls to the CIAO tools which are invoked by the script.

CHANGES IN THE OCTOBER 2010 RELEASE

  • Bug fix: the SPECRESP column values in the combined ARF were not scaled by the number of files that were combined. Users who combined an ARF with an earlier version of the script should remake it to get the correct output values.
  • The RESPFILE keyword value in the header of combined background spectrum is set to "NONE" if the background RMFs were not combined. Previously, the keyword value was left blank.

NOTES

This script is not an official part of the CIAO release but is made available as "contributed" software via the CIAO scripts page. Please see the installation instructions page for help on installing the package.

Bugs

See the bugs page on the CIAO website for an up-to-date listing of known bugs.

See Also

py.contrib
lc_clean, lc_sigma_clip, lightcurves
tools
deflare

Last modified: