Jump to: Description · Examples · Parameters · Changes in CIAO 4.17 · Bugs · See Also

AHELP for CIAO 4.17

dmcopy

Synopsis

Filter and bin tables and images.

Syntax

dmcopy  infile outfile [kernel] [option] [verbose] [clobber]

Description

dmcopy is a very general program which copies a `virtual file' (a table or image with a filter specification) to a physical disk file. For details of the virtual file syntax, see "ahelp dm". Some uses are:

Filtering tables. dmcopy selects rows and columns from a table, making an output file consisting of the filtered table.
Binning tables to an image. Select any n columns from a table, and make an n-dimensional binned image from them.
Filtering images, selecting subsets (spatial, temporal, energy, etc.) of an image.

When do you get a table as output and when do you get an image?

If the input to dmcopy is an image, the output will be an image. If the input is a table and you use the "bin" or "opt image" commands or the "option=image" dmcopy parameter setting, you'll get an image as output. Otherwise, you'll get a table.

Examples

Example 1

dmcopy "bas.fits[stdevt]" out.fits

This copies the table STDEVT in dataset "bas.fits" to "out.fits". Note that use of the Unix command line requires quotes when the string has characters such as "[]" or "()" in it. In the Data Model (DM) paradigm, any GTI (Good Time Interval) table in bas.fits is also copied, as it contains information that's considered to be "part of" the event table.

Example 2

dmcopy "bas.fits[stdevt][time=5000:5200]" out.fits

This copies the table STDEVT in dataset "bas.fits" to "out.fits", including only those rows with times between 5000 and 5200. This assumes that STDEVT contains a column called TIME. We also copy along the Good Time Intervals table, if present, and update it by intersecting with our time filter. If present, ONTIME, LIVETIME and EXPOSURE keywords are also updated.

Note that if the STDEVT table (or "block") was the "first interesting block" in the file - see "ahelp dm" for more information on what "interesting" means to the Data Model - then the example could have been written as:

dmcopy "bas.fits[time=5000:5200]" out.fits

Example 3

dmcopy "bas.fits[stdevt][time=5000:5200]" out.fits option=all

The use of the "option=all" command in dmcopy forces any other tables or images in the bas.fits dataset to be copied over along with the table you are filtering. For instance, a ROSAT FITS event file may have a STDQLM table, among others. The default behaviour of dmcopy is to drop such extra tables during a copy, but option=all ensures they are carried along.

Example 4

dmcopy input.fits "output.txt[opt kernel=text/simple]"

Copying a FITS file to a simple text output file that may be used by other code; see "ahelp dmascii" for more information on working with ASCII files.

Example 5

dmcopy input.txt output.fits

Create a FITS file from a text file; see "ahelp dmascii" for more information on working with ASCII files.

Example 6

dmcopy "data.txt[time=100:200][opt sep=@]" "filtered.out[opt
kernel=text,sep=&]"

Copy the filtered input data to the output file, changing the separator character from "@" to "&". The output could then be used to create a table in LaTeX. See "ahelp dmascii" for more information on working with ASCII files.

Example 7

dmcopy "bas.fits[stdevt][pha=20:30,grade>4]" tmp1.fits
dmcopy "bas.fits[stdevt][time=1522012:1522320,1522400:1522600]"
tmp2.fits
dmcopy "bas.fits[stdevt][(x,y)=circle(3001.2,4982.3,14.5)]" tmp3.fits
dmcopy "bas.fits[stdevt][sky=circle(3001.2,4982.3,14.5)]" tmp3.fits
dmcopy "evt.fits[events][#row=10:13]" evt_row10-13.fits
dmcopy "acis.fits[events][status=10x1x10]" tmp3.fits
dmcopy "region.fits[2][shape=point]" tmp3.fits

These examples filter the same table selecting only some events. (You can use the dmlist tool to find out what columns are available for filtering). Filtering by row number is done as shown in the fifth example. Filtering on bit type columns is also allowed (using a bit string of ones and zeros with 'x' used as a wild card) and on string type columns - see the last two examples above.

Example 8

dmcopy "input.fits[EVENTS][bin x=10:100:1,y=1:100:1]" image1.fits
option=image

This bins an event list in input.fits to create an image in "image1.fits". The qualifier [bin x=10:100:1, y=1:100:1] selects a subset of the image with no blocking. To "block" the image, use e.g. [bin x=10:100:2,y=1:100:2], giving an image each of whose pixels corresponds to 4 pixels in the original image.

Example 9

dmcopy "image.fits[200:512,128:500]" image3.fits

This copies only that part of the image for which the first axis pixel number lies between 200 and 512 and the second axis pixel number is between 128 and 500.

Parameters

name	type	ftype	def	min	max	reqd	stacks
infile	file	input				yes	yes
outfile	file	output				yes	yes
kernel	string		default
option	string
verbose	integer		0	0	5
clobber	boolean		no

Detailed Parameter Descriptions

Parameter=infile (file required filetype=input stacks=yes)

Input virtual file

The CXC virtual file syntax is filename[qualifier][qualifier]....; see "ahelp dm" for details.

FITS table and image files are supported. If the virtual file is not well defined, the program gives an error message and exits. Qualifiers that can be used to filter the input file are:

[block]: the block part of the syntax is the table/image name (for FITS, HDU name or 1-based HDU number). You can omit the [block], in which case the first `interesting' block is used: in the case of FITS, null primary headers and good time intervals are ignored.

filter - is a list specifying ranges of column values to accept, filtering out rows of a table or portions of an image. In the case of an image, where the axes may be unnamed, the 'name=' can be omitted and the ranges are assumed to be in order of the axes.

cols - specifies the names of columns to include, when selecting from a table.

bin - is a sequence defining table columns to bin to create an image file.

newblock - is a name for the resulting block; if omitted, the name of the input block is used.

If a stack of infile's is provided, a stack of outfile's must also be provided with the same number of file names.

Parameter=outfile (file required filetype=output stacks=yes)

The output filename to be created.

If a stack of infile's is provided, a stack of outfile's must also be provided with the same number of file names.

Parameter=kernel (string default=default)

The output file format type. Defaults to the same as input.

Parameter=option (string)

Low level options.

The "option" parameter can be set to any of the options described in ahel dmopt. Additionally, the following values are accepted by dmcopy.

option = all, Copies all blocks in the input file, not just the one you are filtering. The other blocks are copied without change. However, GTIs are treated specially.
option = image, Forces the virtual file to be interpreted as an image, even if the input is a table and there is no [bin] element in the specification.
option = bare, When working with FITS tables, this option will skip copying over the Primary Image HDU leaving it bare.

Parameter=verbose (integer default=0 min=0 max=5)

Verbose can be from 0 to 5, generating different amounts of output.

Parameter=clobber (boolean default=no)

If outfile already exists, clobber=yes will allow you to overwrite it.

Changes in CIAO 4.17

Fix bug causing opt=all to corrupt good time intervals. This was frequently the case when users have applied a time filter to a Chandra gratings dataset and then used dmcopy with opt=all to copy the EVENTS and REGION blocks; the GTIs would become corrupted along with the ONTIME, LIVETIME, and EXPOUSURE keywords. The dmappend work around is no longer required.

Note: for some historical datasets (eg ROSAT) that do not explicitly specify which block the GTI is associated with, the datamodel will now only associate the GTI with a single, first interesting, block rather than all blocks.

Bugs

Incorrect GTIs when time-filtering grating data with dmcopy opt=all

The CIAO team has identified a bug which affects all ACIS and HRC grating data analysis.

When a time filter is applied to a level=1.5 (evt1a.fits) or level=2 (evt2.fits) grating file and the dmcopy opt=all option is used, the GTI block in the output file is not updated to reflect the correct time range.

Time-related header keyword values - such as ontime, livetime, and exposure - will be incorrect because they are calculated from the time ranges in the GTI block.

Workaround:

Run dmcopy without "opt=all" when applying any time filters. Then run dmappend to copy the grating REGION block to the filtered output file, e.g.

unix% dmcopy "evt1a.fits[EVENTS][@flt1.fits]" evt2.fits
unix% dmappend "evt1a.fits[region][subspace -time]"  evt2.fits

The subspace filter is necessary so that the GTIs aren't reapplied to the output file, as explained in the dmappend caveat.

Rebinning an image with different values for the two axes causes the coordinate information to be lost

For example:

unix% dmcopy acis.img"[bin x=::5,y=::6]" acis5x6.img

Using the same value for both axes works correctly:

unix% dmcopy acis.img"[bin (x,y)=::5]" acis5.img