An Image of Diffuse Emission

1. Create an Image of the Region

Throughout this thread, we will use the image created by fluximage, namely images/broad_thresh.img, and you may proceed to source detection with wavdetect; however, should you want to select the region and energies you are interested in, it may done using dmcopy and an events file - e.g.

unix% dmcopy "repro/acisf00315_repro_evt2.fits[energy=300:7000][bin x=4004.5:4404.5:1,y=3625.5:4025.5:1]" \
       original.fits

Note that the get_sky_limits tool can be used to find the sky range covered by an image if you want to match an existing image:

unix% get_sky_limits images/broad_thresh.img 
Running: get_sky_limits
  version: 12 September 2012
Checking binning of image: images/broad_thresh.img
  Image has 1289 x 1289 pixels
  Pixel size is 1.0 by 1.0
  Lower left (0.5,0.5) corner is x,y= 3536.5, 3171.5
  Upper right (1289.5,1289.5) corner is x,y= 4825.5, 4460.5
  DM filter is:
    x=3536.5:4825.5:#1289,y=3171.5:4460.5:#1289
  mkexpmap xygrid value is:
    3536.5:4825.5:#1289,3171.5:4460.5:#1289

We use ds9 to view the image, as shown in Figure 2:

unix% ds9 images/broad_thresh.img -scale log -zoom 0.5 -smooth -invert

Figure 2: Diffuse emission plus point sources

[Version: full-size]

Figure 2: Diffuse emission plus point sources

The ACIS-S3 chip is displayed.

2. Source detection with wavdetect

We can now run wavdetect to identify the point sources. Note that here we have chosen to increase the ellsigma parameter from 3 to 4, since we wish to make sure we excise as much of each source as possible from the image. This value should be modified to suit the requirements of your analysis. Similarly the scales setting should be adjusted to match the data.

unix% punlearn wavdetect
unix% pset wavdetect psffile=broad_thresh.psfmap
unix% pset wavdetect regfile=sources.reg 
unix% pset wavdetect ellsigma=4
unix% wavdetect broad_thresh.img sources.fits sources.scell sources.image sources.nbkg
wavelet scales (pixels) (2.0 4.0): 
Image of the size of the PSF (broad_thresh.psfmap): 

The contents of the parameter file may be checked using plist wavdetect.

From inspection, one can see that many of the sources detected by wavdetect are not point sources, but clumps of diffuse emission. Scientific judgment must be used to modify or delete regions before saving the final region file. The Using the Output of Detect Tools thread shows how to display and modify source lists.

Here we manually remove those detections which do not look like point sources. In this example, the modifed source list has been saved as sources_mod.reg:

unix% head sources_mod.reg
 Region file format: DS9 version 4.1
global color=blue dashlist=8 3 width=1 font="helvetica 10 normal roman" select=1 highlite=1 dash=0 fixed=0 edit=1 move=1 delete=1 include=1 source=1
physical
Ellipse(4399.06,3337.57,18.9642,13.2067,34.1611) #  
Ellipse(4304.93,3440.89,14.3973,9.63261,54.2142) #  
Ellipse(4409.26,3504.19,14.3911,9.85874,48.463) #  
Ellipse(4292.44,3614.14,7.73868,5.04972,57.2064) #  
Ellipse(4332.65,3659.78,5.62888,4.18144,64.6945) #  
Ellipse(4103.68,3663.62,9.55326,4.91821,43.1674) #  
Ellipse(4270.7,3663.2,3.15498,1.53822,68.7819) #  

Note

The source list can contain overlapping regions since this case is handled by the roi tool used in the next section.

Figure 3: Original and modified source regions

[Version: full-size]

Figure 3: Original and modified source regions

The green ellipses show those regions we have decided to use (sources_mod.reg; i.e. these will be removed from the image). The red ellipses are the sources from the original wavdetect sourcelist (sources.reg) that we have decided to ignore.

The image was displayed using the following command:

unix% ds9 broad_thresh.img -smooth radius 2 -smooth -scale log

We end the section by using dmmakereg to convert the modified source list into FITS format, as required by the roi tool used in the next section:

unix% punlearn dmmakereg
unix% dmmakereg "region(sources_mod.reg)" sources_mod.fits wcsfile=broad_thresh.img
unix% dmlist sources_mod.fits cols
 
--------------------------------------------------------------------------------
Columns for Table Block REGION
--------------------------------------------------------------------------------
 
ColNo  Name                 Unit        Type             Range
   1   POS(X,Y)             pixel        Real8          -Inf:+Inf            Position
   2   SHAPE                             String[16]                          Region shape type
   3   R[2]                 pixel        Real8(2)       -Inf:+Inf            Radius
   4   ROTANG[2]            pixel        Real8(2)       -Inf:+Inf            Angle
   5   COMPONENT                         Int2           -                    Component number

wcsfile

dmmakereg uses the wcsfile to convert the region from celestial coordinates (RA,Dec) to physical coordinate. It only does this if it needs to. If the input region is already in physical coordinates then the wcsfile is ignored. Simple numeric values without any annotation are treated as being in physial coordinates. A region in celestial coordinates will be recognized either by the use of colon separated sexagesimal coordinates, with the string fk5 preceeding the regions, or with the special "d" following the values to indiciate decimal degrees. More examples are shows in the DM filtering help file.

In this example, the region is already in physical coordinates so no conversion is performed.

3. Create Source and Background Regions

We now use the roi tool to create the source and background regions since it is much-more capable than the previous solution (mkBgReg.pl and mkSubBgReg.pl). This requires three stages:

1. Using roi to create source and background regions for each source;

2. combining the regions into a format that dmfilth can process;

3. and finishing with an optional validation and modification phase.

The roi tool takes care of combining overlapping source regions and excluding neighbouring sources from background regions.

First we run roi, where the output files (there will be one file for each source) are written to the sources/ sub-directory:

unix% mkdir sources
unix% punlearn roi
unix% pset roi infile=sources_mod.fits
unix% pset roi outsrcfile=sources/src%d.fits
unix% pset roi bkgfactor=0.5
unix% roi
Input src list (sources_mod.fits): 
Input field of view region (): 
Input streak region (): 
Output source list (sources/src%d.fits): 
Background radius computation method (add|mul|area) (mul): 
Background radius (0:) (3): 
unix% pget roi num_srcs
58
unix% dmlist sources_mod.fits counts
60      

The contents of the parameter file may be checked using plist roi. Note that we set the bkgfactor parameter to a small, positive value to ensure that the outer radius of the source region is smaller than the inner radius used in background region, since dmfilth can error out when these two boundaries are the same (Figure 4).

Figure 4: Source and background region created by roi

Figure 4: Source and background region created by roi

Since we set bkgfactor in the call to roi, we have ensured that there is some separation between the inner radius of the background (the green area) and the source region (the blue area). This is done to avoid potential problems in the dmfilth run below, since there may be pixels that appear in both source and background regions due to pixellation issues if the two radii are the same.

Also shown is the area excluded from the background region due the presence of a neighbouring source (the dashed ellipse).

The parameter file for roi is updated to contain the number of source created; here we have 58. This is smaller than the original number of sources (60) since roi has combined sources that overlap. This behavior can be controlled by changing the tool's group parameter.

The output of roi is a set of FITS region files, each containing a region representing the source (SRCREG) and background (BKGREG) for each source:

unix% ls -1 sources/ | head -5
sources/src1.fits
sources/src10.fits
sources/src11.fits
sources/src12.fits
sources/src13.fits
unix% dmlist sources/src\*.fits blocks
 
--------------------------------------------------------------------------------
Dataset: sources/src1.fits
--------------------------------------------------------------------------------
 
     Block Name                          Type         Dimensions
--------------------------------------------------------------------------------
Block    1: PRIMARY                        Null        
Block    2: SRCREG                         Table         5 cols x 1        rows
Block    3: BKGREG                         Table         5 cols x 2        rows

...

--------------------------------------------------------------------------------
Dataset: sources/src14.fits
--------------------------------------------------------------------------------
 
     Block Name                          Type         Dimensions
--------------------------------------------------------------------------------
Block    1: PRIMARY                        Null        
Block    2: SRCREG                         Table         5 cols x 2        rows
Block    3: BKGREG                         Table         5 cols x 8        rows
 
...

--------------------------------------------------------------------------------
Dataset: sources/src17.fits
--------------------------------------------------------------------------------
 
     Block Name                          Type         Dimensions
--------------------------------------------------------------------------------
Block    1: PRIMARY                        Null        
Block    2: SRCREG                         Table         5 cols x 1        rows
Block    3: BKGREG                         Table         5 cols x 3        rows

...

From the dmlist output, we see that a number of sources can have multiple rows in their SRCREG block - e.g. src14.fits - or more than two rows in the BKGREG block (src14.fits and src17.fits); you expect a minimum of two as we have an ellipse minus the original source ellipse. In Figure 5 we show the source and background regions in src14.fits:

Figure 5: Looking at the output of the roi tool

[Version: full-size]

Figure 5: Looking at the output of the roi tool

This image was created by saying

unix% ds9 -zoom 2 -smooth -scale log broad_thresh.img -region sources/src14.fits \
     broad_thresh.img -region "sources/src14.fits[bkgreg]"

and shows a source region on the left which has combined two sources, and on the right its background region, which has multiple sources removed from it; these overlapping regions can be seen in Figure 3.

The dmfilth tool used in the next section requires two files in ASCII region format: one for the sources, and one for the backgrounds. To do this we use the splitroi script, which is part of the CIAO scripts and modules package (although, as described at the end of this section, you can use sed, or similar tools, to create these files for you.

The script takes two arguments; the first is a list of files to use, using the shell's syntax for specifying regular expressions, and the name to use as the start of the output files (".src.reg" and ".bg.reg" are appended to this value):

unix% splitroi "sources/src*.fits" exclude

We now have two ASCII region files - exclude.src.reg and exclude.bg.reg - that can be used by dmfilth. To view them in ds9 you need to convert them to FITS format using dmmakereg; e.g.

unix% dmmakereg "region(exclude.bg.reg)" exclude.bg.fits
unix% ds9 broad_thresh.img -scale log -smooth -region exclude.bg.fits

creates Figure 6.

Figure 6: Viewing the background regions

[Version: full-size]

Figure 6: Viewing the background regions

Here we see the annuli used as the background for each source. For regions where the source density is high - making it hard to see what is going on - the individual region files created by roi can be used (e.g. Figure 5).

An alternative approach is to use the stack support in dmfilth and create files containing lines that say region(src1.fits) and region(src1.fits[bkgreg]) for each file; in teh following the region() syntax is needed to tell dmfilth that each line represents a region file rather than describing an actual region, as it is in the splitroi approach. For example:

unix% ls -1 sources/src*fits | sed 's/.*/region(&)/' > src.reg
unix% ls -1 sources/src*fits | sed 's/.*/region(&[bkgreg])/' > bg.reg
unix% head -3 src.reg 
region(sources/src1.fits)
region(sources/src10.fits)
region(sources/src11.fits)
unix% head -3 bg.reg
region(sources/src1.fits[bkgreg])
region(sources/src10.fits[bkgreg])
region(sources/src11.fits[bkgreg])

At this point you can inspect these region files to select which ones should be used and which ones removed. For this example we do not make any changes.

4. Fill in the Holes (dmfilth)

The tool dmfilth offers several options for extrapolating over regions (see the ahelp file for more information.) Here, we use the POISSON method, which assigns pixel values to the source region by sampling the Poisson distribution whose mean is that of the pixel values in the background region. This has been chosen for didactic purposes and may not be the best choice for your purposes (it assumes that the diffuse emission at the source has the same surface brightness as in the background regions which is questionable here for some sources, looking at Figure 6).

We run dmfilth using the region files created in the previous sections:

unix% punlearn dmfilth
unix% pset dmfilth infile=broad_thresh.img
unix% pset dmfilth outfile=diffuse.img
unix% pset dmfilth method=POISSON
unix% pset dmfilth srclist=@exclude.src.reg
unix% pset dmfilth bkglist=@exclude.bg.reg
unix% pset dmfilth randseed=0
unix% dmfilth 
Input image file (broad_thresh.img): 
Enter output file name (diffuse.img): 
Interpolation method (POLY|DIST|GLOBAL|POISSON|BILINT) (POISSON): 
List of sources to fill in (@exclude.src.reg): 
List of background regions (@exclude.bg.reg): 

The contents of the parameter file may be checked using plist dmfilth.

unix% ds9 diffuse.img -smooth -scale sqrt -invert

The output file is shown in Figure 7. As expected, the point sources are no longer visible.

Figure 7: Image of the diffuse emission

[Version: full-size]

Figure 7: Image of the diffuse emission

The locations where point sources were removed have been replaced by sampling the local background. This image can be compared to Figure 2.