Replace pixel values in source regions of an image with values interpolated from surrounding background regions
dmfilth infile outfile method srclist bkglist [randseed] [clobber] [verbose]
This tool replaces pixel values in a source region or stack of source regions in an input image with values interpolated from surrounding background regions. All regions are specified using standard region library syntax.
There are two different types of method for calculating the replacement pixels:
- The DIST, POISSON, and GLOBAL methods randomly sample values from a probability distribution.
- The POLY and BILINT methods fit a function to the data in the background region and use this fit to calculate the new pixel values.
These methods are described in more detail below.
Pixel values in the source region(s) are sampled from the distribution of pixel values in the background region(s). A single background region or stack of background regions may be used, but they must not include pixels in the source region(s).
Pixel values in the source region(s) are sampled from the Poisson distribution whose mean is that of the pixel values in the background region. A single background region or stack of background regions may be used, but they must not include pixels in the source region(s).
The statistical distribution of total counts in regions filled with method=POISSON is broader than expected. In particular, tests with simulated blank fields containing hundreds of counts in each source region reveal that the quantity
[(counts in filled region)-(original counts in region)]/sqrt(original counts in region)
has a standard deviation approximately 70% broader than expected from simple Gaussian statistics. Users should proceed with caution in performing rigorous quantitative analysis with dmfilth.
Pixel values in the source region(s) are sampled from the Poisson distribution whose mean is that of all pixel values not within the source regions. The bkglist parameter is not used in this method.
A 2D bi-quadratic function of the form
is fit to the data within the background region(s), and the results are used to compute pixel values within the source region(s). Users may specify one background region that includes all source regions, or a stack of background regions with one for each source region; in either case the background region(s) must include the pixels in the source region(s).
The value for each source region pixel is a bilinear interpolation of the form
using the 4 values in the nearest background pixels in the perpendicular directions +/-45 degrees and +/-135 degrees to determine the 4 interpolation coefficients. Since only 4 pixels are used, this method may produce noticable "pattern noise" due to statistical fluctuations in regions of few counts. The bkglist parameter is not used in this method.
dmfilth in1.fits out1.fits DIST "circle(4200.0,3600.0,15.0)" "circle(3800.0,3900.0,7.0)"
Interpolate over the region
by sampling from the pixel distribution in the background region
dmfilth in1.fits out1.fits DIST "circle(4200.0,3600.0,15.0)" "circle(3800.0,3900.0,7.0)" randseed=123
This is the same as the first example except that the random-number generator is seeded with the value 123, rather than using the system time. Repeated runs of this example will create exactly the same output file, whereas they would be different if randseed=0.
dmfilth in1.fits out1.fits POLY "circle(4200.0,3600.0,15.0)" "circle(4200.0,3600.0,25.0)"
Here we have the same source image and source region as previously, but we use a two-dimensional polynomial fit over the region
to define the replace values. Note that the source region is entirely within the region used to fit the background.
dmfilth in1.fits out1.fits POISSON @src.lis @bkg.lis
Each separate region in the file src.lis is filled with values drawn from a Poisson distribution whose mean is the mean level in the corresponding background region in bkg.lis.
dmfilth in1.fits out1.fits GLOBAL "box(4200.0,3800.0,20.0,10.0)" bkglist=""
The box region centered on (4200.0,3800.0) with width 20 and height 10 is replaced with values dran from a Poisson distribution. The mean of this distribution is set to the mean level in all pixels in the image that do not lie within the source region. The bkglist parameter is not used here, and so is set to "".
dmfilth in1.fits out1.fits BILINT @src.lis bkglist=""
Interpolate over the source region stack using bilinear interpolation between the nearest non-source pixels in directions of +/-45 degrees and +/-135 degrees. If, for a particular pixel within the source region, the interpolation matrix is singular, the average of the 2 linear interpolations between the nearest background pixels in the x- and y-directions is used instead. The bkglist is not used in this example.
Detailed Parameter Descriptions
Parameter=infile (file required filetype=input)
The input image. If using an event file, add a binning specification to the filename, e.g. infile="acis_evt2.fits[bin sky=4]"; see "ahelp dmbinning" for more information.
Parameter=outfile (file required filetype=output)
The output dataset.
Parameter=method (string required default=GLOBAL)
The interpolation method.
The allowed values of this enumerated type are POLY, DIST, GLOBAL, POISSON, and BILINT.
Parameter=srclist (string required stacks=yes)
The srclist is a stack of region specifications to be interpolated over.
Parameter=bkglist (string required stacks=yes)
Region(s) from which to define the background (if the method is not GLOBAL or BILINT).
The bkglist is a stack of region specifications to be sampled from for the methods POISSON and DIST. For these methods, the background regions may not include any pixels contained in source regions. For the POLY method, a 2D polynomial is fit over the bkglist region(s), which must include all source region pixels.
For all three methods, if the number of background regions in bkglist is 1, that background region will be applied to all source regions. If the number of background regions in bkglist is equal to the number of source regions, each background region will be used with the corresponding source region. In all other cases, a fatal error will be reported.
The bkglist is not used for the GLOBAL and BILINT methods.
Parameter=randseed (integer default=1 min=0 max=65536)
The seed for the random number generator.
Specifies the seed for the random number generator used by the POISSON, DIST, and GLOBAL methods. A value of 0 uses the system time to set the seed.
Parameter=clobber (boolean default=no)
Specifies if an existing output file should be overwritten.
Parameter=verbose (integer default=0 min=0 max=5)
Specifies the level of verbosity in displaying diagnostic messages (0-5).
Changes in CIAO 4.13
Improved error checking with method=POLY when there are no valid source or background regions.
- When using POISSON method, the distribution varies from true Poisson distribution slightly.
This is only detectable with many (hundreds) of runs.
- The DIST method assumes positive, integer counts (15 Mar 2012)
The tool will fail if this method is used with exposure-corrected images or if there are negative pixels in the source or background regions.
unix% dmstat "events.fits[sky=region(source.reg)]" EVENTS_IMAGE(x, y) min: -2.0534541607 @: ( 4620.5 4868.5 ) max: 63.77072525 @: ( 4588.5 4876.5 ) cntrd[log] : ( 5.7060806841 5.973494067 ) cntrd[phys]: ( 4586.1486455 4876.2879525 ) sigma_cntrd: ( 6.3023091173 6.8273372496 ) good: 70 null: 40
- ERROR: Could not allocate memory for 'output_pixels'.
A common error with dmfilth is to use an event file as input instead of an image. If an event file is used event file, the tool bins it into a very large image (8192 x 8192 for ACIS). With an image that large all the system memory can easily be exhausted -- especially if running on a 32bit machine which is limited to ~2Gb of memory per-application (regardless of how much memory is installed).
Ensure that an image is being used (dmlist the file will show either 'Table' or 'Image'). If using a table (ie event file) then it should be binned into an image first.
- dmbinning, dmfiltering
- aconvolve, acrosscorr, apply_fov_limits, arestore, arfcorr, asp_offaxis_corr, dmappend, dmcontour, dmcoords, dmcopy, dmellipse, dmextract, dmgroup, dmimg2jpg, dmimgadapt, dmimgblob, dmimgcalc, dmimgdist, dmimgfilt, dmimghist, dmimghull, dmimglasso, dmimgpick, dmimgpm, dmimgproject, dmimgreproject, dmimgthresh, dmmaskbin, dmmaskfill, dmnautilus, dmregrid, dmregrid2, dmstat, evalpos, get_fov_limits, get_sky_limits, imgmoment, install_marx, make_psf_asymmetry_region, mean_energy_map, pileup_map, psf_project_ray, psfsize_srcs, simulate_psf, src_psffrac, tgextract, tgextract2