Running celldetect
![[CXC Logo]](../../imgs/cxc-logo.gif){kind=logo}
CIAO 4.17 Science Threads
Overview
Synopsis:
celldetect uses a sliding square detect cell whose size is matched to the instrument PSF to search for statistically significant enhancements over the background. At each point where a cell is placed, a signal-to-noise ratio of source counts to background counts is computed. If this ratio is above the detection threshold, a candidate source is recorded.
Purpose:
To illustrate several ways to use the Detect tool celldetect. This tool works well in simple fields with well-separated point sources.
Last Update: 17 Feb 2022 - Review for CIAO 4.14. Replaced the first dataset with a different observation of the same field.
Contents
- Getting Started
- Running celldetect
- Caveat: source detection and cosmic-ray afterglows
- Parameter files:
- History
-
Images
- Figure 1: Filtered event file (ObsID 2254)
- Figure 2: Simple chip S3 detections (ObsID 578)
- Figure 3: ACIS-I (ObsID 2254): detections with an exposure map
- Figure 4: ACIS-I detections from recursive blocking (ObsID 1522)
- Figure 5: ACIS-I detections from recursive blocking with an exposure map (ObsID 1522)
- Figure 6: Comparison of recursive blocking results with and without an exposure map (ObsID 1522)
Getting Started
Download the sample data: 2254 (ACIS-S, 3C 295); 1522 (ACIS-I, Trapezium Cluster)
unix% download_chandra_obsid 2254,1522
To run this thread, we want a source image and a congruent exposure map (for the "Running wavdetect with an exposure map" section). We chose to create both by running the fluximage script. The image and exposure map may also be created by following the Multiple Chip ACIS Exposure Map and Exposure-corrected Image thread. We also require a PSF map: an image whose pixel values indicates the size of the PSF at that location.
For the first three thread examples, a full resolution image of the ACIS-I array is used:
unix% fluximage "acisf02254_repro_evt2.fits[ccd_id=0:3]" binsize=1 bands=broad outroot=acisf02254 clob+
unix% dmlist acisf02254_broad_thresh.img blocks
--------------------------------------------------------------------------------
Dataset: acisf02254_broad_thresh.img
--------------------------------------------------------------------------------
Block Name Type Dimensions
--------------------------------------------------------------------------------
Block 1: EVENTS_IMAGE Image Int4(2733x2735)
If the length of either axis exceeds the maxlogicalwindow parameter value, the tool will operate in "recursive blocking" mode.
To view the file, load it into ds9:
unix% ds9 acisf02254_broad_thresh.img &
which produces Figure 1.
Figure 1: Filtered event file (ObsID 2254)
![[Thumbnail image: The filtered and binned event file is displayed in ds9.]](events.png)
[Version: full-size]
Figure 1: Filtered event file (ObsID 2254)
We can now also create the PSF map using the mkpsfmap tool.
unix% mkpsfmap acisf02254_broad_thresh.img acisf02254_broad.psfmap energy=1.4967 ecf=0.8 clob+
We used energy=2.3 to match the broad-band energy used to create the exposure maps, and ecf=0.8 (ie 80%). celldetect converts the PSF size into a cellsize which is a multiple of 3 logical pixels (3, 6, 9, 12,...). It does not use the value to perform any photometric correction. This means ecf and energy values do not need to be very precise.
Running celldetect
A Simple Example
To run celldetect with the basic parameters set, it is necessary to specify the infile and output file names only. Note that it is not required to create an ASCII version of the region file; if the regfile parameter is left blank, it will not be used.
unix% punlearn celldetect unix% pset celldetect infile=acisf02254_broad_thresh.img unix% pset celldetect outfile=acisf02254_src.fits unix% pset celldetect regfile=acisf02254_src.reg unix% pset celldetect psffile=acisf02254_broad.psfmap unix% celldetect Input file (acisf02254_broad_thresh.img): Output source list (acisf02254_src.fits):
The contents of the parameter file may be checked using plist celldetect.
There are two source lists created, both containing the same sources: an ASCII file (acisf02254_src.reg) and a FITS file (acisf02254_src.fits). Both formats are fully described in the Detect manual.
To display the image with the source detections overlaid:
unix% ds9 acisf02254_broad_thresh.img &
Load the source list from either file (Region → Load Regions... → acisf02254_src.fits OR acisf02254_src.reg). The detections are shown in Figure 2.
Figure 2: Simple chip S3 detections (ObsID 578)
![[Thumbnail image: The detections are overlaid on the event file as green ellipses.]](detections.png)
[Version: full-size]
Figure 2: Simple chip S3 detections (ObsID 578)
These source detections were determined with the default parameter values.
A key parameter that you may wish to vary is thresh, the detect cell signal-to-noise (S/N) threshold; it was left at the default value of 3 in the above example. Cells whose S/N exceeds this value are identified as containing possible source candidates. Note that not all such cells will result in detected sources; if the findpeaks parameter is set to "yes" (the default), adjacent cells whose ratios exceed thresh will be combined into a single detection. Simulations indicate that values of thresh as low as ~2 may be appropriate for short, on-axis observations, but it is recommended to start with a value of 3 or 4.
With An Exposure Map
False sources may be detected in the vicinity of significant exposure variations, such as detector edges or chip gaps. To mitigate this effect, run celldetect with an exposure map.
The exposure map was created in the Get Started section by running the fluximage script with the broad energy band definition. The exposure map may also be created by following the Multiple Chip ACIS Exposure Map and Exposure-corrected Image thread.
When running the tool with an exposure map, the expratio parameter must be set. This is the ratio of the average exposure of the background frame around the detect cell to that in the cell. Detections for sources whose ratio falls below the specified value are omitted from the source regions file (regfile); all detections are present in the output source list (outfile) and the column EXPO_RATIO is created.
unix% punlearn celldetect unix% pset celldetect infile=acisf02254_broad_thresh.img unix% pset celldetect outfile=acisf02254_expmap_src.fits unix% pset celldetect regfile=acisf02254_expmap_src.reg unix% pset celldetect expstk=acisf02254_broad_thresh.expmap unix% pset celldetect psffile=acisf02254_broad.psfmap unix% pset celldetect expratio=0.9 unix% celldetect Input file (acisf02254_broad_thresh.img): Output source list (acisf02254_expmap_src.fits):
The contents of the parameter file may be checked using plist celldetect.
To display the image:
unix% ds9 acisf02254_broad_thresh.img &
Load the source list from the region file (Region → Load Regions... → acisf02254_expmap_src.reg). Figure 3 shows the results from both runs; the red ellipses are the results of using an exposure map and the green ellipses are the additional sources picked up when running without an exposure map.
Figure 3: ACIS-I (ObsID 2254): detections with an exposure map
![[Thumbnail image: Detections from run with exposure map on the data as different-colored ellipses.]](expmap.png)
[Version: full-size]
Figure 3: ACIS-I (ObsID 2254): detections with an exposure map
The detections shown when using an exposure map.
In this case, the exposure map does not make a discernable difference in the detections.
Using Recursive Blocking
As mentioned before, if the input is larger than maxlogicalwindow, celldetect automatically uses a "recursive blocking scheme": the inner maxlogicalwindow pixel region of the dataset is searched for sources, then the inner 2*maxlogicalwindow pixel region (excluding the part already analyzed) is blocked by 2 and searched for sources, then the inner 4*maxlogicalwindow is blocked by 4, etc., until TLMIN, TLMAX is reached.
For each consecutive search, the portion of the data that has already been searched for sources is excluded from the analysis. Therefore, only one blocking factor is used for each region on the dataset. Since a source could be missed if it straddles the border between regions of different blocking factors, each pass actually extends a bit into the previously-analyzed region. Occasionally this leads to two detections of the same source, but such cases are automatically identified (see the output source list columns DOUBLE and DOUBLE_ID, discussed in the "celldetect Input Parameters & Data Products Reference Chapter" of the Detect manual).
For this example, we use the level=2 event file of ObsID 1522. The DETNAM keyword shows that the four ACIS-I chips (0-3) and two ACIS-S chips (6-7) were on for this observation:
unix% dmkeypar acisf01522N005_evt2.fits DETNAM echo+ ACIS-012367
A Data Model filter is added to the input datafile so that only the ACIS-I chips are used in the analysis:
unix% punlearn celldetect unix% pset celldetect infile="acisf01522N005_evt2.fits[ccd_id=0:3][energy=500:7000]" unix% pset celldetect outfile=acisi_block_src.fits unix% pset celldetect regfile=acisi_block_src.reg unix% pset celldetect fixedcell=12 unix% celldetect maxlogicalwindow=2048 Input file (acisf01522N005_evt2.fits[ccd_id=0:3][energy=500:7000]): Output source list (acisi_block_src.fits):
The contents of the parameter file may be checked using plist celldetect.
To display the event data with the source detections overlaid:
unix% ds9 acisf01522N005_evt2.fits &
Load the source list from either file (Region → Load Regions... → acisi_block_src.fits OR acisi_block_src.reg). The detections are shown in Figure 4.
Figure 4: ACIS-I detections from recursive blocking (ObsID 1522)
![[Thumbnail image: The detections are overlaid on the event file as green ellipses.]](blocking.png)
[Version: full-size]
Figure 4: ACIS-I detections from recursive blocking (ObsID 1522)
Recursive blocking allows source detection over a larger area than the maximum image size (maxlogicalwindow).
To see what recursive blocking was used on the event file, examine the BLOCK column of the FITS output file. This dmlist command gives the position of each source and the the blocking factor used at that point:
unix% dmlist "acisi_block_src.fits[cols x,y,block]" data -------------------------------------------------------------------------------- Data for Table Block SRCLIST -------------------------------------------------------------------------------- ROW POS(X,Y) BLOCK 1 ( 3868.9213483146, 4056.2996254682) 1 2 ( 3871.16250, 4059.950) 1 3 ( 3871.3913043478, 4147.8260869565) 1 . . . 614 ( 5146.50, 3300.33750) 2 615 ( 5159.5655737705, 3298.9426229508) 2 616 ( 5269.0, 3215.8529411765) 2
Using Recursive Blocking with Exposure Maps and PSF Maps
Here we repeat the recursive blocking example with the addition of exposure maps and PSF maps. Note: users can elect to supply only PSF maps, only exposure maps, or as we show here, both exposure maps and PSF maps.
One exposure map and one PSF map is needed to match each blocking factor that celldetect will use. Create the exposure maps by running the fluximage script, as shown in the ACIS Exposure Map (Multiple Chips) thread.
Each exposure map and PSF map must be made with the binning factor used in the recursive blocking scheme (typically 1, 2, 4). The exposure map must also match the spatial region that celldetect will search at each step of recursive blocking: the exposure map binned by a factor of 1 should cover the inner maxlogicalwindow pixels of the image, bin factor 2 should cover the inner 2*maxlogicalwindow region, and so on.
One easy way to determine the correct number of bins and the grid to use is to first run celldetect with a fixed cell size and set verbose=2. The tool will then print the grid it uses to bin the input file, which is the same grid needed to create the exposure maps and PSF maps. celldetect is fast so this should not take very long to complete.
unix% punlearn celldetect; unix% celldetect "acisf01522N005_evt2.fits[ccd_id=0:3][energy=500:7000]" out=temporary.src verbose=2 fixedcell=12 input file: acisf01522N005_evt2.fits[ccd_id=0:3][energy=500:7000] output file: temporary.src acisf01522N005_evt2.fits[ccd_id=0:3][energy=500:7000][bin x=3072.5:5120.5:1,y=3072.5:5120.5:1][opt type=i4] reading data image creating candidate source list cell size 12 acisf01522N005_evt2.fits[ccd_id=0:3][energy=500:7000][bin x=2048.5:6144.5:2,y=2048.5:6144.5:2][opt type=i4] reading data image creating candidate source list cell size 12 acisf01522N005_evt2.fits[ccd_id=0:3][energy=500:7000][bin x=0.5:8192.5:4,y=0.5:8192.5:4][opt type=i4] reading data image creating candidate source list cell size 12
We can then use those grids with fluximage to create the individual exposure maps
unix% fluximage acisf01522N005_evt2.fits"[ccd_id=0:3]" \ out=bin1 xygrid=3072.5:5120.5:1,3072.5:5120.5:1 mode=h clob+ band=broad ... unix% fluximage acisf01522N005_evt2.fits"[ccd_id=0:3]" \ out=bin2 xygrid=2048.5:6144.5:2,2048.5:6144.5:2 mode=h clob+ band=broad ... unix% fluximage acisf01522N005_evt2.fits"[ccd_id=0:3]" \ out=bin4 xygrid=0.5:8192.5:4,0.5:8192.5:4 mode=h clob+ band=broad ... unix% /bin/ls bin?_thresh.expmap > expmap.lis
and use those exposure maps to create the PSF maps
unix% mkpsfmap bin1_broad_thresh.expmap bin1_broad.psfmap energy=1.4967 ecf=0.8 clob+ unix% mkpsfmap bin2_broad_thresh.expmap bin2_broad.psfmap energy=1.4967 ecf=0.8 clob+ unix% mkpsfmap bin4_broad_thresh.expmap bin4_broad.psfmap energy=1.4967 ecf=0.8 clob+ unix% /bin/ls bin?_broad.psfmap > psfmap.lis
The exposure maps and PSF maps are input to the expstk and psffiles parameters; note the stack syntax (@) used with the list file.
unix% punlearn celldetect unix% pset celldetect infile="acisf01522N005_evt2.fits[ccd_id=0:3][energy=500:7000]" unix% pset celldetect outfile=acisi_block_expmap_src.fits unix% pset celldetect regfile=acisi_block_expmap_src.reg unix% pset celldetect psffile=@psfmap.lis unix% pset celldetect expstk=@expmap.lis unix% pset celldetect expratio=0.99 unix% celldetect maxlogicalwindow=2048 Input file (acisf01522N005_evt2.fits[ccd_id=0:3][energy=500:7000]): Output source list (acisi_block_expmap_src.fits):
The contents of the parameter file may be checked using plist celldetect.
Display the event data with the source detections overlaid:
unix% ds9 acisf01522N005_evt2.fits &
Load the source list from either file (Region → Load Regions... → acisi_block_expmap_src.fits OR acisi_block_expmap_src.reg). The results are shown in Figure 5. Placing these results on top of those from the recursive blocking without an exposure map example (Figure 6) shows that using an exposure map helps to eliminate spurious detections along the chip edges.
Figure 5: ACIS-I detections from recursive blocking with an exposure map (ObsID 1522)
![[Thumbnail image: The detections are overlaid on the event file as green ellipses.]](blockexpmap.png)
[Version: full-size]
Figure 5: ACIS-I detections from recursive blocking with an exposure map (ObsID 1522)
Including an exposure map with the recursive blocking reduces the number of false sources in the vicinity of significant exposure variations, such as detector edges or chip gaps.
Figure 6: Comparison of recursive blocking results with and without an exposure map (ObsID 1522)
![[Thumbnail image: Detections from both recursive blocking runs are shown on the data as different-colored ellipses.]](compare.png)
[Version: full-size]
Figure 6: Comparison of recursive blocking results with and without an exposure map (ObsID 1522)
The red ellipses are the sources detected in the run with the exposure map (acisi_block_expmap_src.reg); the green ellipses are the additional sources detected when running without an exposure map (acisi_block_src.reg).
Caveat: source detection and cosmic-ray afterglows
CIAO users who are trying to detect sources of about 10 counts or less should be aware of potential confusion with cosmic-ray afterglows that have not been removed from the data.
The Cosmic-Ray Afterglows why topic explains the current processing options and the limitations of the available tools used to detect afterglows.
Parameters for /home/username/cxcds_param/celldetect.par
(comment lines have been omitted)
infile = acisf02254_broad_thresh.img Input file
outfile = acisf02254_src.fits Output source list
(psffile = acisf02254_broad.psfmap) single psf file or list of psf files
(expstk = ) list of exposure map files
(regfile = acisf02254_src.reg) ASCII regions file
(clobber = no) Overwrite exiting outputs?
(thresh = 3) Source threshold
(snr_diminution = 1.0) Diminution on SNR threshold - range (< 0 to 1) - Allows fine grained cell sliding
(findpeaks = yes) Find local peaks?
(centroid = yes) Compute source centroids?
(ellsigma = 3) Size of output source ellipses (in sigmas)
(expratio = 0) cutoff ratio for source cell exposure variation
(fixedcell = 0) Fixed cell size to use (0 for variable cell)
(xoffset = INDEF) Offset of x axis from data center
(yoffset = INDEF) Offset of y axis from data center
(cellfile = ) Output cell size image stack name
(maxlogicalwindow = 8192) Max logical window
(bkgfile = ) Background file name
(bkgvalue = 0) Background count/pixel
(bkgerrvalue = 0) Background error
(convolve = no) Use convolution?
(snrfile = ) SNR output file name (for convolution only)
(verbose = 0) Log verbosity level
(log = no) Make a celldetect.log file?
(mode = ql)
Parameters for /home/username/cxcds_param/celldetect.par
(comment lines have been omitted)
infile = acisf02254_broad_thresh.img Input file
outfile = acisf02254_expmap_src.fits Output source list
(psffile = acisf02254_broad.psfmap) single psf file or list of psf files
(expstk = acisf02254_broad_thresh.expmap) list of exposure map files
(regfile = acisf02254_expmap_src.reg) ASCII regions file
(clobber = no) Overwrite exiting outputs?
(thresh = 3) Source threshold
(snr_diminution = 1.0) Diminution on SNR threshold - range (< 0 to 1) - Allows fine grained cell sliding
(findpeaks = yes) Find local peaks?
(centroid = yes) Compute source centroids?
(ellsigma = 3) Size of output source ellipses (in sigmas)
(expratio = 0.9) cutoff ratio for source cell exposure variation
(fixedcell = 0) Fixed cell size to use (0 for variable cell)
(xoffset = INDEF) Offset of x axis from data center
(yoffset = INDEF) Offset of y axis from data center
(cellfile = ) Output cell size image stack name
(maxlogicalwindow = 8192) Max logical window
(bkgfile = ) Background file name
(bkgvalue = 0) Background count/pixel
(bkgerrvalue = 0) Background error
(convolve = no) Use convolution?
(snrfile = ) SNR output file name (for convolution only)
(verbose = 0) Log verbosity level
(log = no) Make a celldetect.log file?
(mode = ql)
Parameters for /home/username/cxcds_param/celldetect.par
(comment lines have been omitted)
infile = 1522/primary/acisf01522N004_evt2.fits[ccd_id=0:3,energy=500:7000] Input file
outfile = acisi_block_src.fits Output source list
(psffile = ) single psf file or list of psf files
(expstk = ) list of exposure map files
(regfile = acisi_block_src.reg) ASCII regions file
(clobber = no) Overwrite exiting outputs?
(thresh = 3) Source threshold
(snr_diminution = 1.0) Diminution on SNR threshold - range (< 0 to 1) - Allows fine grained cell sliding
(findpeaks = yes) Find local peaks?
(centroid = yes) Compute source centroids?
(ellsigma = 3) Size of output source ellipses (in sigmas)
(expratio = 0) cutoff ratio for source cell exposure variation
(fixedcell = 12) Fixed cell size to use (0 for variable cell)
(xoffset = INDEF) Offset of x axis from data center
(yoffset = INDEF) Offset of y axis from data center
(cellfile = ) Output cell size image stack name
(maxlogicalwindow = 2048) Max logical window
(bkgfile = ) Background file name
(bkgvalue = 0) Background count/pixel
(bkgerrvalue = 0) Background error
(convolve = no) Use convolution?
(snrfile = ) SNR output file name (for convolution only)
(verbose = 0) Log verbosity level
(log = no) Make a celldetect.log file?
(mode = ql)
Parameters for /home/username/cxcds_param/celldetect.par
(comment lines have been omitted)
infile = 1522/primary/acisf01522N004_evt2.fits[ccd_id=0:3,energy=500:7000] Input file
outfile = acisi_block_expmap_src.fits Output source list
(psffile = @psf.lis) single psf file or list of psf files
(expstk = @expmap.lis) list of exposure map files
(regfile = acisi_block_expmap_src.reg) ASCII regions file
(clobber = no) Overwrite exiting outputs?
(thresh = 3) Source threshold
(snr_diminution = 1.0) Diminution on SNR threshold - range (< 0 to 1) - Allows fine grained cell sliding
(findpeaks = yes) Find local peaks?
(centroid = yes) Compute source centroids?
(ellsigma = 3) Size of output source ellipses (in sigmas)
(expratio = 0.99) cutoff ratio for source cell exposure variation
(fixedcell = 0) Fixed cell size to use (0 for variable cell)
(xoffset = INDEF) Offset of x axis from data center
(yoffset = INDEF) Offset of y axis from data center
(cellfile = ) Output cell size image stack name
(maxlogicalwindow = 2048) Max logical window
(bkgfile = ) Background file name
(bkgvalue = 0) Background count/pixel
(bkgerrvalue = 0) Background error
(convolve = no) Use convolution?
(snrfile = ) SNR output file name (for convolution only)
(verbose = 0) Log verbosity level
(log = no) Make a celldetect.log file?
(mode = ql)
History
| 03 Jan 2005 | reviewed for CIAO 3.2: no changes |
| 03 Jun 2005 | updated links for CIAO 3.2 version of Detect manual |
| 19 Dec 2005 | updated for CIAO 3.3: the acis_expmap script has been updated to version 3.3 for the new asphist tool syntax; updated files in detect data tarfile |
| 05 Jan 2006 | update to acis_expmap usage message (script functionality is unchanged) |
| 01 Dec 2006 | reviewed for CIAO 3.4: no changes |
| 16 Jan 2008 | updated for CIAO 4.0: acis_expmap script removed from the CIAO scripts package (obsolete); ds9 now automatically looks for the "[SRCLIST]" extension in the region file, so it doesn't have to be specified; kernel parameter removed from celldetect; removed data tarfile |
| 16 Apr 2008 | added Source Detection and Cosmic-Ray Afterglows caveat |
| 13 Jan 2009 | updated for CIAO 4.1: images are inline |
| 29 Jan 2010 | reviewed for CIAO 4.2: images no changes |
| 12 Jan 2011 | reviewed for CIAO 4.3: no changes |
| 15 Dec 2011 | reviewed for CIAO 4.4: use fluximage to create the image and exposure map |
| 03 Dec 2012 | Review for CIAO 4.5 |
| 25 Feb 2013 | Added bpix to download_chadra_obsid list |
| 25 Nov 2013 | Review for CIAO 4.6. Added step to run r4_header_update for obsid 578. |
| 16 Dec 2014 | Reviewed for CIAO 4.7. Minor edits to file names. |
| 29 Dec 2016 | Reviewed for CIAO 4.9; no changes. |
| 28 Mar 2018 | Updated for CIAO 4.10; new maxlogicalwindow parameter now controls when recursive binning begins. |
| 15 Nov 2018 | Updated for CIAO 4.11. The psftable has been removed. In its place, users now must provide psffile: a PSF map, ie the output from the mkpsfmap tool. There have been additional changes to how the recursive blocking is being done which may reduce the number of duplicate (DOUBLE) detections. There is also a new algorithm to check for sources that are missed because they are split between detect cells such that their SNR is below the threshold. |
| 17 Feb 2022 | Review for CIAO 4.14. Replaced the first dataset with a different observation of the same field. |