/ ciao4.17 / ahelp / mkwarf.html

Jump to: Description · Examples · Parameters · Description of the Algorithm · Tool Caveats · Deprecation of 'pbkfile' parameter · Bugs · See Also

AHELP for CIAO 4.17

mkwarf

Context: Tools::Response

Synopsis

Generate a weighted ARF.

Syntax

mkwarf  infile outfile weightfile spectrumfile egridspec [pbkfile]
[threshold] [feffile] [mskfile] [asolfile] [mirror] [detsubsysmod]
[dafile] [ardlibpar] [geompar] [clobber] [verbose]

Description

`mkwarf' creates a weighted ARF file based on an input weight map (WMAP). Optionally, it also produces a set of weights that are used by `mkrmf' to create the associated weighted RMF. The use of a weighted response may allow one RMF and ARF to be used to analyze a spectrum extracted from a large area that covers multiple response regions (and hence a range of spectral responses).

Assumptions and Caveats

Please note that the algorithm used by mkwarf and mkrmf for calculating the weighted responses assumes that there is no spatial variation in the source spectrum across the extraction region(s) used. In addition, it should only be used to create "PI" responses - ie for the analysis of spectra binned over the PI column and not the PHA column.

ACIS: Creating the Weight Map (WMAP)

The WMAP should be created by running the tool sky2tdet. sky2tdet projects an image for a region of interest in sky(x,y) coordinates to TDET (tiled detector) coordinates. This WMAP properly accounts for chip edges and bad pixels in the response file. Users should no longer use the dmextract tool to create WMAPs for use with mkwarf.

HRC: Creating the Weight Map (WMAP)

The WMAP for HRC can be created either with dmextract using the "wmap" parameter or it can be created directly with dmcopy.

How to weight the WMAP

Ideally, the response function for a given region should be weighted by the fraction of the incident flux from the source that falls within that region. However, this is generally what we are looking to obtain from the observation! We instead know the distribution of counts (i.e. the incident flux after it has passed through the telescope mirror and been absorbed by the detector) across the regions. Below we discuss two possible approaches to the problem; the choice of which - if either - is better depends on the characteristics of the data and the science objectives of the analysis.

a. spectrumfile=none

The simplest approach is to use the raw counts to weight the response (in which the spectrumfile parameter should be set to none). This means that the weights will include the effect of the mirror response and detector QE (including bad pixels) - ie the effective area term - which may bias the spectral fits. One way to minimise this is to pick a restricted energy range over which to make the weight map, one over which the effective area (or, more correctly, the product of the source spectrum and the effective area) does not vary strongly.

b. using the spectrumfile parameter

Another approach is to use the spectrum of the source to correct the detected counts for the effective area of the telescope/instrument combination. Of course, we don't know the source spectrum to use to correct the model. One approach is therefore to use an iterative scheme, where the spectral model used to weight the WMAP is updated after each fit. The spectrumfile parameter is used to supply the an ASCII source model file to use in weighting the WMAP. Note that the energy range of the spectrum should be adjusted to match that used to create the WMAP.

Examples of creating weighted ARFs can be found in the CIAO science threads.

Examples

Example 1

mkwarf "spectrum1.fits[wmap]" spectrum1.warf spectrum1.wgt none
0.5:9.5:0.02

This will create a weighted ARF, called spectrum1.warf, over the energy grid of 0.5 to 9.5 keV in 20 eV bins. The weighting uses the pixel values in the WMAP extension of the input PHA file (spectrum1.fits), with no spectral weighting.

The weights file created by mkwarf would then be used in mkrmf by setting:

  unix% pset mkrmf weights=spectrum1.wgt
  unix% pset mkrmf infile=CALDB
  unix% pset mkrmf axis1="energy=0:1"

where the infile parameter is set to CALDB since we do not need to use acis_fef_lookup in this case, and the value for the energy grid is irrelevant (as long as something is supplied) since it's overridden by the energy grid in the weights file (here spectrum1.wgt).

Example 2

mkwarf "spectrum2.fits[wmap]" spectrum2.warf spectrum2.wgt none
0.5:9.5:0.02 threshold=0.01

As with example 1, but this time only those FEF regions which contain at least 1 percent of the total counts are included in the calculation of the weighted ARF. This option can be used to speed up mkrmf by removing the need to calculate RMFs for regions which do not significantly contribute to the response. The choice of thresehold depends on both the distribution of counts across the WMAP and the science objectives of your analysis.

Example 3

mkwarf "sources.pi[WMAP]" sources.warf sources.wgt
egridspec=0.01:11:0.01 dafile=CALDB

Create a weighted ARF over the energy grid of 0.01 to 11 keV in 0.01 eV bins. The dafile parameter is set to apply the dead area correction.

Example 4

mkwarf csmooth_out.fits spectrum3.warf spectrum3.wgt spec.txt
"grid(spectrum1.rmf[MATRIX][cols ENERG_LO,ENERG_HI])"

This will use the (supposedly) smoothed image in 'csmooth_out.fits' as the WMAP with the spectral weights in the ASCII file 'spec.txt' to create the ARF and RMF weights on an energy grid that matches that of the RMF stored in spectrum1.rmf.

Example 5

sky2tdet "acis_evt.fits[sky=region(ciao.reg),energy=500:2000]" asp.hist
"tdet[wmap]"
mkwarf tdet out.arf weightfile=none feffile=none spectrum=none
egrid=0.3:9.3:0.01
mkacisrmf infile=CALDB outfile=out.rmf wmap=tdet ...etc...

In this example the input to the mkwarf tool is the output from sky2tdet. The feffile and weightfile have been set to "none" since the output weightfile is not needed (it is only used by the mkrmf tool).

Example 6

dmcopy "hrc_evt2.fits[sky=region(ciao.reg)][bin tdet=16]" hrc_tdet.wmap
mkwarf infile=hrc_tdet.wmap outfile=hrc.warf egrid=0.3:9.3:0.01 mode=h
feffile=""

This creates a weighted ARF for an HRC datasets. The WMAP is created by filtering the event file in sky coordinates, and the binning the data in TDET coordinates. Note: The HRC TDET image size is very large (16k x 16k or 32k x 32k ) so using a small bin size is not practical.

Since there are no FEF files for HRC (there is no HRC RMF creation tool), the feffile is left blank (equivalent to "none"). Since the feffile is blank, the output weights file cannot be create so it is left at it default blank value.

Parameters

name	type	ftype	def	min	max	reqd	stacks
infile	file	input				yes
outfile	file	output				yes
weightfile	file	output				yes
spectrumfile	file	input				yes
egridspec	string					yes
pbkfile	string	input				no
threshold	float		0	0	1
feffile	file	ARD	CALDB
mskfile	file
asolfile	file						yes
mirror	string		HRMA
detsubsysmod	string
dafile	string	input	CALDB
ardlibpar	file		ardlib
geompar	file		geom
clobber	boolean		no
verbose	integer		0	0	5

Detailed Parameter Descriptions

Parameter=infile (file required filetype=input)

Input WMAP file name. Be sure to include the DM filter "[WMAP]" if you are using a WMAP attached to a PHA file.

For ACIS, the sky2tdet tool should be used to create a WMAP in TDET coordinates. For HRC, user can simply bin their spatially filtered event file in TDET or DET coordinates.

Parameter=outfile (file required filetype=output)

The output file name for the weighted ARF file.

The weighted ARF created by mkwarf is OGIP compliant.

Parameter=weightfile (file required filetype=output)

The output file name for the optional weights file.

This file is needed by mkrmf to create the weighted RMF (the weights parameter). It contains a list of FEF regions used to create the weighted ARF, along with the weight used for each region.

This parameter may be left blank or set to the value "none" (no quotes) if and only if the feffile parameter is also blank or "none". This is useful when working with HRC datasets which do not have FEF files, and when working with vast majority of ACIS datasets that require the mkacisrmf tool, rather than the mkrmf tool, to compute the response matrix.

Parameter=spectrumfile (file required filetype=input)

Input spectral weights file

If the raw counts in the input weight map are to be used to weight the spectral response then set this parameter to "none" (or left blank). Otherwise this parameter should be set to the name of an ASCII file containing columns for energy (in keV) and a weight value (such as the spectral model in photon/cm^2/s/keV).

The energy range of the spectrum should be adjusted to match that used to create the WMAP. Note that the weights column is normalized before being used (unlike the spectrumfile parameter for mkinstmap).

Parameter=egridspec (string required)

Output energy grid specification

This parameter defines the energy grid of the output ARF. It accepts the same syntax as the engrid parameter of mkarf.

The ACIS detector is calibrated over the range 0.224004 - 12 keV; choosing values outside this range may result in errors from the tool.

Parameter=pbkfile (string not required filetype=input)

Deprecated. See below.

Parameter=threshold (float default=0 min=0 max=1)

Fractional threshold cut for FEF regions

Any FEF region that has a weight (counts or spectrum weighted counts) below the fractional threshold of the total will be ignored from the calcuation of the ARF and will not go into the weights file. This can be used to exclude regions that only minially contribute to the over-all spectrum/flux and thus speed up the generation of the RMFs by mkrmf.

Note 1

The final fractional contribution (in the 'FRACTION' column in the weight file output) can be less than the threshold parameter since the thresholding is done and then the fractions of the remaining regions are re-normalized.

Note 2

The total weight/flux can be reduced by more than the threshold percent if, for example, there are many low counts regions and a single bright region. The low count regions individually could be less than the threshold of the total, but the sum of the flux/counts from these regions could be more (considerably more) than the threshold.

Parameter=feffile (file filetype=ARD default=CALDB)

The optional FEF file containing the calibration areas.

The FEF file is needed when users need to create the output weights file which is used by the mkrmf tool. Since most ACIS observations are now calibrated with the mkacisrmf tool, and HRC observations do not have FEF files, the feffile and weightfile can generally be set to the value "none" (no quotes).

In the event users need to use mkrmf, then the default value of CALDB should be used. The name of the FEF file selected by the tool is output if the verbose value is set to 2 or greater and is also included in the header of the output files as the FEFFILE keyword.

The verbose=2 output when running mkwarf looks like:

  Mapping response regions to FEF regions
  FEF File: /soft/ciao/CALDB/data/chandra/acis/fef_pha/acisD2000-01-29fef_pha_ctiN0001.fits
  Mapping response regions to FEF regions. Done

and the FEFFILE value can be found from the header using the dmkeypar tool:

  unix% dmkeypar spectrum.wgt feffile echo+
  /soft/ciao/CALDB/data/chandra/acis/fef_pha/acisD2000-01-29fef_pha_ctiN0002.fits

Parameter=mskfile (file default=)

The detector mask file

The mask file (msk1.fits) for the observation. The mask file is needed in particular when a window or subarray was used. A value of "NONE" indicates that no mask file will be applied.

More information on the mask file is available in the CIAO Dictionary.

Parameter=asolfile (file stacks=yes)

This parameter may be left blank when working with WMAPs in TDET coordinates. It is only necessary to provide the asol files for WMAPs in DET coordinates.

If the aspect solution file(s) - pcad*_asol1.fits - for the observation are provided, the average dy, dz, and dtheta values are computed and are used to adjust the SIM alignment. This reduces problems seen when the WMAP is at the edge of the chip, causing the mapping from detector to chip coordinates to fail.

Multiple aspect solution files may be provided as a comma-separated list or as a stack; see "ahelp stack" for more information.

Parameter=mirror (string default=HRMA)

The ARDLIB mirror parameter; can use additional qualifiers to change mirror response.

The ardlib help file contains information about the available qualifiers.

Parameter=detsubsysmod (string default=)

A modifier that is added to the detector name sent into ARDLIB.

Unlike some other ARDLIB enabled tools; mkwarf runs on multiple chips and as such does not have a detsubsys parameter. This parameter allows one to modify the internal detsubsys value to allow the response product to be modified. Things such as detector QE/U can be overridden (see "ahelp ardlib" for more information).

Parameter=dafile (string filetype=input default=CALDB)

ACIS "dead area" coefficients file, which may have the values "NONE" (no dead area computation), CALDB (for automatic lookup), or an explicit file reference to an ACIS "dead area" coefficients FITS table.

The ACIS dead area refers to a slight decrease in detector efficiency due to the background cosmic ray flux which temporarily renders some pixels useless. The calibration product is a coefficient (per CCD) which gives the fractional area lost (or "deadened") per second. Since the area lost increases with time, the magnitude of the effect depends upon the ACIS clocking parameters (number of rows, window location, frame-time) which determine how long a pixel was exposed to the background cosmic ray flux during the primary exposure and during electronic readout from the frame-store area. For full-frame timed-exposure, the dead area is about 4% at maximum CHIPY and about 2% at the readout. It is smaller for subarrays.

The dafile parameter is ignored for HRC data.

Parameter=ardlibpar (file default=ardlib)

The name of the ARDLIB parameter file.

It is very important that users have setup the appropriate bad pixels files in ardlib.par before running mkwarf.

Parameter=geompar (file default=geom)

The name of the Pixlib Geometry parameter file.

Parameter=clobber (boolean default=no)

Overwrite output files if they already exist?

Parameter=verbose (integer default=0 min=0 max=5)

Controls amount of debugging output is sent to screen.

Description of the Algorithm

The weighted ARF is constructed by computing the ARF for every non-zero pixel in the input WMAP. That is for each pixel in the WMAP, the ARF is computed based on the detector efficiency, and the pixel's location relative to the optical axis of the mirrors.

Bad pixels and columns must have non-zero weights in the WMAP so that mkwarf can correctly account for the fraction of the effective area lost.

If ACIS data, and if the feffile and weightfile are specified, then the the tool accumulates the WMAP fraction weighted sum of the ARFs of all the pixels in the FEF region accounting for the fraction of bad pixels/columns in the FEF region. The output weights file is the fractional ARFs output per FEF region.

The output weighted ARF is the sum of the individual per-pixel ARFs weighted by the relative WMAP values and with the optional spectral weights applied.

Tool Caveats

The algorithm assumes that the source spectrum is spatially uniform, so that only the normalisation of the spectrum varies with position.
The algorithm is designed to represent the weighted ARF over a large region. If the exposure varies strongly over most of the extraction region then the ARF may not fully account for this variation. This is mainly of concern for regions at the edge of an ACIS chip.
The weights file must only be used with mkrmf to create a PI RMF; PHA RMFs are not supported.

Deprecation of 'pbkfile' parameter

In CIAO 4.6, the ACIS parameter block file is no longer a required input to this tool. Therefore the 'pbkfile' parameter has been deprecated. The parameter has been retained for now but it is non functional.

This change is made possible by a series of header keyword updates that were made in Chandra standard data processing and are replicated in the chandra_repro script. Four new keywords: SUM_2X2, FEP_CCD, ORC_MODE, and OCLKPAIR have been added to the standard ACIS event file, that, in addition to existing keywords, provide the tool the information needed for the ACIS Dead Area calibration.

These keywords are required to use this tool in CIAO 4.6. Data from the archive with ASCDSVER newer than DS8.4.2 will already have these keywords. This will be the default for all but a few early observations. Users with data that was created prior to ASCDSVER=8.4.2 will need to follow one of these paths to use CIAO 4.6.

Retrieve most recent data from Chandra archive. All but a few datasets in the Chandra archive have been reprocessed to include these keywords (the earliest observations). The last bulk reprocessing included several updates including corrections for aspect "drift" within an observation as well as updated timing information. A summary of the updates can be found here: https://cxc.harvard.edu/cda/repro4.html
Reprocess the data through chandra_repro. This will update the headers with the new keywords and ensure that the other calibrations are consistent with the latest CALDB.
Just update the headers using the r4_header_update script. This will just add new keywords to the event file without any other calibration updates.

Bugs

Caveats

Warnings related to missing SUM_2X2, OCLKPAIR, FEP_CCD, or ORC_MODE keywords

Users trying to run this tool with old versions of data products, those created with ASCDSVER less than version 8.4.2, may see warnings like

# mkwarf (CIAO): WARNING: The pbkfile parameter is no longer used. The information required for the Dead Area calibration will be extracted from wmap.

# mkwarf (CIAO):  WARNING: Could not find keyword SUM_2X2 in file wmap. The dead area calibration will not be included in the response.

Users should reprocess their data or use the r4_header_update script.

ERROR: Max egridspec energy=10 above max FEF energy=9.886

mkwarf is required to compute and write a "weightfile" output file which contains FEF regions for use by mkrmf. If the energy range in the input RMF is greater than that in the FEF files, you get an error like:

ERROR: Max egridspec energy=10 above max FEF energy=9.886

Although the comparison to the FEF files is unnecessary in this case, there is currently no way to turn it off (e.g. set the weightfile to "NONE").

Workaround:

In order to avoid the error, it is necessary to define an energy range for mkacisrmf that falls within the boundaries of the FEF files, i.e. approximately 0.28 - 9.8 keV.

Issue with the WMAP in DET coordinates when the source is at a chip edge

The preferred WMAP will be in TDET coordinates so this will not be a problem.

If the observation has large dy & dz offsets in the aspect solution file and they are quite variable during the observation, the tool will fail with a CALDB error. The large (and varying) offsets cause the mapping from DET to CHIP coordinates to fail and the tool cannot determine which response calibration file to use in creating the RMF. The follow message may be printed at verbose >0.

# mkwarf (CIAO): ERROR: Could not map response region to FEF region

Users should be sure to supply an aspect solution file.

The mkwarf tool may run out of memory for large regions.

# mkwarf (CIAO): dsALLOCERR -- ERROR: Could not allocate memory.

If a weighted ARF is being created for a large region, on order of a full chip (or larger), mkwarf may hit the intrinsic memory limit of a 32-bit application.

Workaround:

There is a parameter in sky2tdet, bin - that specifies the binning of the WMAP, which is used as input to mkwarf.

If you encounter this bug, increase the bin value to create a coarser WMAP.

Couldn't determine chip position for pixel: (1013.000000,0.000000) with value=1.000000. Skipping pixel

The issue is that the conversion from detector coordinates (i.e. those used to create the WMAP) to chip coordinates needs the SIM_Z values. In reality, SIM_Z varies with time, but as we have an image (hence no time information) and the SIM_Z variations are usually small, a single value stored in the image header is used for the transformation.

This means that there's actually some ambiguity in the mapping from the WMAP to chip location. In reality this isn't a concern, since we already bin on larger scales than introduced by the SIM_Z variation (i.e. we are only concerned with 32x32 pixel regions on the chip). However, it does mean that those pixels close to the chip boundaries can end up as apparently not falling on a chip. In this case, we issue the warning shown and ignore the counts from this pixel in the WMAP.

If the ignored counts are small compared to the total signal in the WMAP, as in the vast majority of situations, then it's not a problem. It can be a problem if most of the counts in the WMAP end up being ignored. In reality, this situation is unlikely to occur; two possible situations would be where the source region is narrow and lies along the chip boundary (this is a truly extreme case) or if you are using mkwarf to calculate "point source" responses near chip boundaries.