Skip to the navigation links
Last modified: 27 Jan 2010
Where are the PDFs?

Running wavdetect

CIAO 4.2 Science Threads



Overview

Last Update: 27 Jan 2010 - reviewed for CIAO 4.2: no changes

Synopsis:

wavdetect operates on the input in two stages. First it detects possible source pixels in a dataset by repeatedly correlating it with "Mexican Hat" wavelet functions with different scale sizes. Pixels with sufficiently large positive correlation values are removed from the image as assumed sources, and subsequent correlations are performed at the same scale. The second stage generates a source list from information from the first stage at each wavelet scale.

An alternative algorithm for specifying the allowed number of false sources detected is available, where extensive simulations were performed to characterize the dependence of the number of pixels per false source on source significance and background (expressed in the new sigcalfile parameter), the tool can now determine the significance threshold for source detection itself, if the user inputs directly the desired false source rate (e.g., desired number of false sources per field per scale). Both threshold algorithms are fully supported, and users need make no changes if they still wish to use the current algorithm.

Purpose:

To illustrate several ways to use the Detect tool wavdetect.

Read this thread if:

if you want to detect sources in an ACIS or HRC observation. This tool separates closely spaced point sources and finds extended sources so long as wavlet scales are chosen appropriately.

Related Links:




Contents



Getting Started

Sample ObsID used: 578 (ACIS-S, 3C 295)

File types needed: evt2

Due to the heavy computational load, it is recommended to run wavdetect on images no larger than 1024x1024; this limit is affected by the available memory as well. In the following examples, a full resolution 512x512 image centered on the cluster core is used:

unix% dmcopy \
      "acisf00578N002_evt2.fits[ccd_id=7][bin x=3848:4360:1.0,y=3860:4372:1.0]" \
      s3_img.fits

To view the file, load it into ds9:

unix% ds9 s3_img.fits &

which produces Figure 1.

[Thumbnail image: The filtered and binned event file is displayed in ds9.]

[Version: full-size]

[Print media version: The filtered and binned event file is displayed in ds9.]

Figure 1: Image of the cluster core

A full resolution image of the data filtered to 512 square pixels is used in this thread.


Running Wavdetect

A Simple Example

To run wavdetect with the basic parameters set, it is necessary to specify the input and output filenames only. Note that it is not required to create an ASCII version of the region file; if the regfile parameter is left blank, it will not be used.

unix% punlearn wavdetect
unix% pset wavdetect infile=s3_img.fits
unix% pset wavdetect outfile=s3_src.fits
unix% pset wavdetect scellfile=s3_scell.fits
unix% pset wavdetect imagefile=s3_imgfile.fits
unix% pset wavdetect defnbkgfile=s3_nbgd.fits
unix% pset wavdetect regfile=s3_src.reg
unix% wavdetect
Input file name (s3_img.fits): 
Output source list file name (s3_src.fits): 
Output source cell image file name (s3_scell.fits): 
Output reconstructed image file name (s3_imgfile.fits): 
Output normalized background file name (s3_nbgd.fits): 

The contents of the parameter file may be checked using plist wavdetect.

There are many outputs from the tool:

  • outfile: the FITS source list; contains same sources as regfile (see below)

  • scellfile: the source cell map shows the image pixels which were used to estimate source properties.

  • imagefile: the reconstructed source image is the combined correlation maxima regions, which correspond to probable sources. It is essentially what the data would look like if there were no background and noise fluctuations.

  • defnbkgfile: the normalized background derived from the data after the identified sources have been taken out.

  • regfile: the ASCII region file; contains same sources as outfile (see above).

Full descriptions of these files are given in the "Wavdetect Input Parameters & Data Products Reference" chapter of the Detect manual.

To display the image with the detections overlaid:

unix% ds9 s3_img.fits &

Load the source list from either file (Region → Load Regions... → s3_src.fits OR s3_src.reg). The detections are shown in Figure 2.

[Thumbnail image: The detections are overlaid on the event file as green ellipses.]

[Version: full-size]

[Print media version: The detections are overlaid on the event file as green ellipses.]

Figure 2: Simple chip S3 detections

The three core sources are detected as a single region when the default parameter values are used.

Key parameters that you may wish to vary are scales and sigthresh. The scales parameter is a list of radii (in image pixels) of Mexican Hat wavelet functions; a wavelet transform is performed for each scale in the list. Usually a list of 5 scales is chosen, and each succeeding value is a factor of 2 or SQRT(2) larger than the previous one. The primary concern is to match the first scale size to the PSF. The choice of how many scaled wavelets to use depends somewhat on the available computing resources.

The sigthresh parameter is the significance threshold for source detection. Specifying a sigthresh that is proportional to the inverse of the number of pixels in the image allows ~1 false source per field (i.e. the default value of 10-6 corresponds to one spurious source in a 1000x1000 pixel map).

The result of running wavdetect on the same image with scales="1.0 2.0 4.0 8.0 16.0" and sigthresh=4e-06 are shown in Figure 3 (zooming in reveals the core and both hotspots (Figure 4) of this radio galaxy at the center of the cluster). The contents of the parameter file for this run may be checked using plist wavdetect.

[Thumbnail image: Detections from both runs of wavdetect are shown on the data as different-colored ellipses.]

[Version: full-size]

[Print media version: Detections from both runs of wavdetect are shown on the data as different-colored ellipses.]

Figure 3: Chip S3 detections: different scales and sigthresh

The red ellipses are the sources detected in the simple wavdetect run (s3_src.reg); the green ellipses are the sources detected with the different scales and threshold.

[Thumbnail image: The center of the image is displayed at zoom=4 in ds9.]

[Version: full-size]

[Print media version: The center of the image is displayed at zoom=4 in ds9.]

Figure 4: Galaxy core and hotspots

The three core sources are identified as individual detections when different scales and significance threshold are used. The data is displayed at zoom=4.


With an Exposure Map

In regions with substantial exposure variations, wavdetect may inaccurately estimate detection significance. Using an exposure map can suppress false positives (erroneous detections) and can help refine source property estimates. On account of this, the number of sources detected with an exposure map may be different than the number detected without one.

The exposure map was created by following the ACIS Exposure Map (Single Chip) thread. For the purpose of illustration, we used a monochromatic instrument map of 0.71 keV. When creating your own exposure map, be sure that the exposure map and input image are congruent (i.e. same size, image center, and binning factor).

Run wavdetect:

unix% punlearn wavdetect
unix% pset wavdetect infile=s3_img.fits
unix% pset wavdetect outfile=s3_expmap_src.fits
unix% pset wavdetect scellfile=s3_expmap_scell.fits
unix% pset wavdetect imagefile=s3_expmap_imgfile.fits
unix% pset wavdetect defnbkgfile=s3_expmap_nbgd.fits
unix% pset wavdetect regfile=s3_expmap_src.reg
unix% pset wavdetect expfile=s3_0.71keV_expmap.fits
unix% wavdetect 
Input file name (s3_img.fits): 
Output source list file name (s3_expmap_src.fits): 
Output source cell image file name (s3_expmap_scell.fits): 
Output reconstructed image file name (s3_expmap_imgfile.fits): 
Output normalized background file name (s3_expmap_nbgd.fits): 

The contents of the parameter file may be checked using plist wavdetect.

To display the image with an overlay of the source detections:

unix% ds9 s3_img.fits &

Load the source list from either file (Region → Load Regions... → s3_expmap_src.fits OR s3_expmap_src.reg). The results are shown in Figure 5.

[Thumbnail image: The detections are overlaid on the event file as green ellipses.]

[Version: full-size]

[Print media version: The detections are overlaid on the event file as green ellipses.]

Figure 5: Chip S3 detections: with an exposure map

In this case, the detections with the exposure map are identical to those without an exposure map.

There isn't any significant exposure variation across the field in this example, so the source properties do not change in the exposure map run.

To examine the source properties, use either dmlist with the column names of interest:

unix% dmlist "s3_expmap_src.fits[cols RA,DEC,PSF_SIZE]" data
 
--------------------------------------------------------------------------------
Data for Table Block SRCLIST
--------------------------------------------------------------------------------
 
ROW    RA                   DEC                  PSF_SIZE    
 
     1       212.8558538789        52.1797387065     0.51768761873245
     2       212.8847083521        52.1878910087     0.61479145288467
     3       212.8640791885        52.1921762232     0.47023788094521
     4       212.8362796737         52.202181660     0.40190586447716
     5       212.8351087846        52.2029039026     0.40246158838272
     6       212.8475613372        52.2255903030     0.53206640481949
     7       212.8714240484        52.2259177730     0.62263220548630
     8       212.8842308305         52.227961680     0.75592726469040
     9       212.8067711419        52.2281781731     0.59245705604553
    10       212.8236615527        52.2284789414     0.55507236719131
    11       212.8310964079        52.2335976922     0.60252219438553
    12       212.8362660158        52.2366611370     0.65922313928604
    13       212.8626620537        52.2375255009     0.75592726469040

or prism:

unix% prism s3_expmap_src.fits &


Caveats

Source detection and cosmic-ray afterglows

CIAO users who are trying to detect sources of about 10 counts or less should be aware of potential confusion with cosmic-ray afterglows that have not been removed from the data.

The Cosmic-Ray Afterglows why topic explains the limitations of the current tools and describes analysis tips to help eliminate more of the afterglows.


A detection tool, not a flux tool

This tool is designed to be used as a source detection algorithm, and only secondarily as a source flux measurement tool. Flux estimation with wavdetect is generally quite reliable, but there is a small fraction of cases where the algorithm will fail to return useful source parameter estimates. This is inevitable, as there is no perfect source detection algorithm.

In section 3.2.2 of "A Wavelet-Based Algorithm for the Spatial Analysis of Poisson Data" (P. E. Freeman, et al. 2002, ApJS, 138, 185) it says:

"We note two situations where care must be exercised in interpreting
results. First, the source cell for an [apparently] extended source
may be too small if the smoothing scale is ~ [the PSF scale]."

The workaround for this is to rerun wavdetect with the parameter scales set to "4.0 5.0" to get a better accounting of the source in question.




Parameters for /home/username/cxcds_param/wavdetect.par


#
#   parameter file for wavdetect
#
#
#   input
#
        infile = s3_img.fits      Input file name
#
#   output
#
       outfile = s3_src.fits      Output source list file name
     scellfile = s3_scell.fits    Output source cell image file name
     imagefile = s3_imgfile.fits  Output reconstructed image file name
   defnbkgfile = s3_nbgd.fits     Output normalized background file name
#
#   scales
#
        scales = 2.0 4.0          wavelet scales (pixels)
      (regfile = s3_src.reg)      ASCII regions output file
#
#   output options
#
      (clobber = no)              Overwrite existing outputs?
     (ellsigma = 3.0)             Size of output source ellipses (in sigmas)
     (interdir = .)               Directory for intermediate outputs
#
#########################################################################
#
#   wtransform parameters
#
#
#   optional input
#
     (bkginput = )                Input background file name
  (bkgerrinput = no)              Use bkginput[2] for background error
     (falsesrc = -1.0)            Allowed number of false sources per image
   (sigcalfile = ${ASCDS_CALIB}/wtsimresult.fits -> /soft/ciao-4.0/data/wtsimresult.fits) Significance calibration file
#
#   output info
#
  (outputinfix = )                Output filename infix
#
#   output content options
#
    (sigthresh = 1e-06)           Threshold significance for output source pixel list
 (bkgsigthresh = 0.001)           Threshold significance when estimating bkgd only
#
#   exposure info
#
      (exptime = 0)               Exposure time (if zero, estimate from map itself
      (expfile = )                Exposure map file name (blank=none)
    (expthresh = 0.1)             Minimum relative exposure needed in pixel to analyze it
#
#   background
#
      (bkgtime = 0)               Exposure time for input background file
#
#   iteration info
#
      (maxiter = 2)               Maximum number of source-cleansing iterations
     (iterstop = 0.0001)          Min frac of pix that must be cleansed to continue
#
#   end of wtransform parameters
#
########################################################################
########################################################################
#
#   wrecon parameters
#
#
#   PSF size parameters
#
      (xoffset = INDEF)           Offset of x axis from optical axis
      (yoffset = INDEF)           Offset of y axis from optical axis
        (eband = 1.4967)          Energy band
      (eenergy = 0.393)           Encircled energy of PSF
     (psftable = ${ASCDS_CALIB}/psfsize20010416.fits -> /soft/ciao-4.0/data/psfsize20010416.fits) Table of PSF size data
#
#   end of wrecon parameters
#
########################################################################
#
#   run log verbosity and content
#
          (log = no)              Make a log file?
      (verbose = 0)               Log verbosity
#
#   mode  
#
         (mode = ql)              




Parameters for /home/username/cxcds_param/wavdetect.par


#
#   parameter file for wavdetect
#
#
#   input
#
        infile = s3_img.fits      Input file name
#
#   output
#
       outfile = s3_src_2.fits    Output source list file name
     scellfile = s3_scell_2.fits  Output source cell image file name
     imagefile = s3_imgfile_2.fits Output reconstructed image file name
   defnbkgfile = s3_nbgd_2.fits   Output normalized background file name
#
#   scales
#
        scales = 1.0 2.0 4.0 8.0 16.0 wavelet scales (pixels)
      (regfile = s3_src_2.reg)    ASCII regions output file
#
#   output options
#
      (clobber = no)              Overwrite existing outputs?
     (ellsigma = 3)               Size of output source ellipses (in sigmas)
     (interdir = .)               Directory for intermediate outputs
#
#########################################################################
#
#   wtransform parameters
#
#
#   optional input
#
     (bkginput = )                Input background file name
  (bkgerrinput = no)              Use bkginput[2] for background error
#
#   output info
#
  (outputinfix = )                Output filename infix
#
#   output content options
#
    (sigthresh = 4e-06)           Threshold significance for output source pixel list
 (bkgsigthresh = 0.001)           Threshold significance when estimating bkgd only
     (falsesrc = -1.0)            Allowed number of false sources per image
   (sigcalfile = ${ASCDS_CALIB}/wtsimresult.fits -> /soft/ciao-4.0/data/wtsimresult.fits) Significance calibration file
#
#   exposure info
#
      (exptime = 0)               Exposure time (if zero, estimate from map itself
      (expfile = )                Exposure map file name (blank=none)
    (expthresh = 0.1)             Minimum relative exposure needed in pixel to analyze it
#
#   background
#
      (bkgtime = 0)               Exposure time for input background file
#
#   iteration info
#
      (maxiter = 2)               Maximum number of source-cleansing iterations
     (iterstop = 0.0001)          Min frac of pix that must be cleansed to continue
#
#   end of wtransform parameters
#
########################################################################
########################################################################
#
#   wrecon parameters
#
#
#   PSF size parameters
#
      (xoffset = INDEF)           Offset of x axis from optical axis
      (yoffset = INDEF)           Offset of y axis from optical axis
        (eband = 1.4967)          Energy band
      (eenergy = 0.393)           Encircled energy of PSF
     (psftable = ${ASCDS_CALIB}/psfsize20010416.fits -> /soft/ciao-4.0/data/psfsize20010416.fits) Table of PSF size data
#
#   end of wrecon parameters
#
########################################################################
#
#   run log verbosity and content
#
          (log = no)              Make a log file?
      (verbose = 0)               Log verbosity
#
#   mode  
#
         (mode = ql)              




Parameters for /home/username/cxcds_param/wavdetect.par


#
#   parameter file for wavdetect
#
#
#   input
#
        infile = s3_img.fits      Input file name
#
#   output
#
       outfile = s3_expmap_src.fits Output source list file name
     scellfile = s3_expmap_scell.fits Output source cell image file name
     imagefile = s3_expmap_imgfile.fits Output reconstructed image file name
   defnbkgfile = s3_expmap_nbgd.fits Output normalized background file name
#
#   scales
#
        scales = 2.0 4.0          wavelet scales (pixels)
      (regfile = s3_expmap_src.reg) ASCII regions output file
#
#   output options
#
      (clobber = no)              Overwrite existing outputs?
     (ellsigma = 3.0)             Size of output source ellipses (in sigmas)
     (interdir = .)               Directory for intermediate outputs
#
#########################################################################
#
#   wtransform parameters
#
#
#   optional input
#
     (bkginput = )                Input background file name
  (bkgerrinput = no)              Use bkginput[2] for background error
#
#   output info
#
  (outputinfix = )                Output filename infix
#
#   output content options
#
    (sigthresh = 1e-06)           Threshold significance for output source pixel list
 (bkgsigthresh = 0.001)           Threshold significance when estimating bkgd only
     (falsesrc = -1.0)            Allowed number of false sources per image
   (sigcalfile = ${ASCDS_CALIB}/wtsimresult.fits -> /soft/ciao-4.0/data/wtsimresult.fits) Significance calibration file
#
#   exposure info
#
      (exptime = 0)               Exposure time (if zero, estimate from map itself
      (expfile = s3_0.71keV_expmap.fits) Exposure map file name (blank=none)
    (expthresh = 0.1)             Minimum relative exposure needed in pixel to analyze it
#
#   background
#
      (bkgtime = 0)               Exposure time for input background file
#
#   iteration info
#
      (maxiter = 2)               Maximum number of source-cleansing iterations
     (iterstop = 0.0001)          Min frac of pix that must be cleansed to continue
#
#   end of wtransform parameters
#
########################################################################
########################################################################
#
#   wrecon parameters
#
#
#   PSF size parameters
#
      (xoffset = INDEF)           Offset of x axis from optical axis
      (yoffset = INDEF)           Offset of y axis from optical axis
        (eband = 1.4967)          Energy band
      (eenergy = 0.393)           Encircled energy of PSF
     (psftable = ${ASCDS_CALIB}/psfsize20010416.fits -> /soft/ciao-4.0/data/psfsize20010416.fits) Table of PSF size data
#
#   end of wrecon parameters
#
########################################################################
#
#   run log verbosity and content
#
          (log = no)              Make a log file?
      (verbose = 0)               Log verbosity
#
#   mode  
#
         (mode = ql)              



History

03 Jan 2005 reviewed for CIAO 3.2: no changes
03 Jun 2005 updated links for CIAO 3.2 version of Detect manual
19 Dec 2005 updated for CIAO 3.3: updated files in detect data tarfile
01 Dec 2006 reviewed for CIAO 3.4: no changes
16 Jan 2008 updated for CIAO 4.0: added information about new wavdetect algorithm; updated parameter file listings to include falsesrc and sigcalfile, removed kernel; ds9 now automatically looks for the "[SRCLIST]" extension in the region file, so it doesn't have to be specified
16 Apr 2008 added Source Detection and Cosmic-Ray Afterglows caveat
12 Jan 2009 updated for CIAO 4.1: wavdetect algorithm was improved in this release; images are inline
27 Jan 2010 reviewed for CIAO 4.2: no changes

Return to Threads Page: Top | All | Imag

Where are the PDFs?
Last modified: 27 Jan 2010