Using the Output of Detect Tools

Edit the List of Sources

Once the source list is overlaid on the image, you may see some sources that you want to exclude from the analysis. Currently, ds9 does not number the sources in the field of view. However, double-clicking on a source region brings up the region info box, which lists the region "ID". In Figure 2, the source on the left side of the field of view was chosen - the ID is 12. This number corresponds to the row the region occupies in the FITS file.

NOTE: this technique only works with the first region file loaded into ds9. After that, the imager starts counting up. For example, if you load region_a.fits that has 10 sources, then region_b.fits that has 6, the regions of the second file will be counted from 11 onward. One may, of course, always examine the region file visually to determine which rows to delete.

Figure 2: Finding the ID of a source region

[Version: full-size]

Figure 2: Finding the ID of a source region

The selected source has an ID of 27.

To create a new region file that doesn't have row 27 in it, dmcopy is used:

unix% dmcopy "acisf02254_src.fits[#row=1:26,28:]" acisf02254_1gone_src.fits

unix% dmlist acisf02254_1gone_src.fits blocks
 
--------------------------------------------------------------------------------
Dataset: acisf02254_1gone_src.fits
--------------------------------------------------------------------------------
 
     Block Name                          Type         Dimensions
--------------------------------------------------------------------------------
Block    1: PRIMARY                        Null        
Block    2: SRCLIST                        Table        31 cols x 81       rows

The dmlist command shows that there are only 81 rows in the file now. This new file may be displayed in ds9 (Figure 3) to ensure that the correct source was removed.

Figure 3: Confirming a removed source

[Version: full-size]

Figure 3: Confirming a removed source

The unwanted source (blue cross) is no longer present in the region file (green ellipses).

This filter was fairly simple. It is possible to make more complicated filters with the DM syntax. If you wanted to exclude rows (sources) 8, 12, and 14 from the original file:

unix% dmcopy "acisf02254_src.fits[#row=1:7,9:11,13,15:]" acisf02254_3gone_src.fits

unix% dmlist acisf02254_3gone_src.fits blocks
 
--------------------------------------------------------------------------------
Dataset: acisf02254_3gone_src.fits
--------------------------------------------------------------------------------
 
     Block Name                          Type         Dimensions
--------------------------------------------------------------------------------
Block    1: PRIMARY                        Null        
Block    2: SRCLIST                        Table        31 cols x 79       rows

unix% dmlist acisf02254_3gone_src.fits"[cols ra,dec,net_counts]" data
 
--------------------------------------------------------------------------------
Data for Table Block SRCLIST
--------------------------------------------------------------------------------
 
ROW    RA                   DEC                  NET_COUNTS
 
     1       212.8476582345        52.2254818222              817.250
     2       212.8363474221        52.2021515227             76.18750
     3       212.8356429371        52.2027046910            383.18750
     4       212.8351854747        52.2031801937            149.43750
     5        212.835175130        52.2044274344             27.43750
     6       212.8068074149        52.2280851890             33.81250
     ...
 

Sort on a Particular Column

The tool dmsort makes it possible to sort on any column in the FITS file. To order the rows by NET_COUNTS:

unix% dmlist "acisf02254_src.fits[cols net_counts]" data
 
--------------------------------------------------------------------------------
Data for Table Block SRCLIST
--------------------------------------------------------------------------------
 
ROW    NET_COUNTS
 
     1              817.250
     2             76.18750
     3            383.18750
     4            149.43750
     5             27.43750
     6             33.81250
     7                 41.0
     8        23.4285717010
     9       150.2857208252
    10                 32.0
    11                 24.0
    ...

unix% dmsort acisf02254_src.fits acisf02254_sorted_src.fits keys=net_counts

unix% dmlist "acisf02254_sorted_src.fits[cols net_counts]" data
 
--------------------------------------------------------------------------------
Data for Table Block SRCLIST
--------------------------------------------------------------------------------
 
ROW    NET_COUNTS
 
     1        18.8974990845
     2                 19.0
     3        19.1590900421
     4                 20.0
     5                 22.0
     6                 22.0
     7        22.5900001526
     8        22.7142848969
     9        23.0795459747
    10        23.4285717010
    11                 24.0
    ...

The default behavior of dmsort is to organize the results in ascending order. To sort NET_COUNTS in descending order, add a dash ("-") before the column name:

unix% dmsort acisf02254_src.fits acisf02254_descend_src.fits keys=-net_counts

unix% dmlist "acisf02254_descend_src.fits[cols net_counts]" data
 
--------------------------------------------------------------------------------
Data for Table Block SRCLIST
--------------------------------------------------------------------------------
 
ROW    NET_COUNTS
 
     1              817.250
     2       771.3068237305
     3       712.4091186523
     4            383.18750
     5       301.8571472168
     6       238.1931762695
     7       216.7001495361
     8       214.9545440674
     9       174.9166717529
    10       170.7196502686
    11       150.2857208252
    ...

It is also possible to have several sort priorities (i.e. sort by NET_COUNTS, then NET_COUNTS_ERR); see ahelp dmsort for more information on how to sort files.

Filtering the Source List

If you are interested in only using a certain range of data in your analysis, use the DM filtering syntax to create a new source list with only those items.

To keep the sources that have 40 or more net counts:

unix% dmcopy "acisf02254_src.fits[net_counts=40:]" acisf02254_40cts_src.fits
unix% dmlist "acisf02254_40cts_src.fits[cols net_counts]" data
 
--------------------------------------------------------------------------------
Data for Table Block SRCLIST
--------------------------------------------------------------------------------
 
ROW    NET_COUNTS
 
     1              817.250
     2             76.18750
     3            383.18750
     4            149.43750
     5                 41.0
     6       150.2857208252
     7       301.8571472168
     8        68.4285736084
     9        90.3977279663
    10        45.0795440674
    11        62.1428565979
    12       102.2857131958
    13        73.5714263916
    14                 40.0
    ...

or those with NET_COUNTS between 25 and 80:

unix% dmcopy "acisf02254_src.fits[net_counts=25:80]" acisf02254_25-80cts_src.fits
unix% dmlist "acisf02254_25-80cts_src.fits[cols net_counts]" data
 
--------------------------------------------------------------------------------
Data for Table Block SRCLIST
--------------------------------------------------------------------------------
 
ROW    NET_COUNTS
 
     1             76.18750
     2             27.43750
     3             33.81250
     4                 41.0
     5                 32.0
     6        68.4285736084
     7                 36.0
     8        29.0795459747
     9        37.2386360168
    10        45.0795440674
    ...

See ahelp dmfiltering for other variations on the filtering syntax.

Converting Coordinates to Sexagesimal

Often it is desirable to have the list of sources in sexagesimal format, i.e. (RA,Dec) = (hms,dms). This can be done in ds9, with prop_precess, or using the regphystocel script.

• The regphystocel script can be used to convert regions in physical coordinates to celestial coordinates.

  % regphystocel acisf02254_src.fits acisf02254_src_ds9.reg clob+
  regphystocel
            infile = acisf02254_src.fits
           outfile = acisf02254_src_ds9.reg
           wcsfile = 
              text = 
               tag = 
           clobber = yes
           verbose = 1
              mode = ql
  % cat acisf02254_src_ds9.reg
  # Region file format: DS9 version 4.1
  global color=green dashlist=8 3 width=1 font="helvetica 10 normal roman" select=1 highlite=1 dash=0 fixed=0 edit=1 move=1 delete=1 include=1 source=1
  fk5
  ellipse(14:11:23.4379,52:13:31.7345,0.966892",0.926778",63.2685) #  
  ellipse(14:11:20.7233,52:12:7.7454,1.16711",1.04867",72.128) #  
  ellipse(14:11:20.5543,52:12:9.7368,1.04189",0.989279",49.0305) #  
  ellipse(14:11:20.4445,52:12:11.4486,1.17873",1.01639",138.111) #  
  ellipse(14:11:20.4420,52:12:15.9387,1.24158",1.20135",85.0671) #  
  ellipse(14:11:13.6337,52:13:41.1066,1.1083",0.851713",45.716) #  
  ellipse(14:11:33.0335,52:11:2.5639,1.67152",1.47723",54.7683) #  
  ellipse(14:11:30.8464,52:14:23.7191,1.62874",1.31701",95.4704) #  
  ellipse(14:11:27.4170,52:11:31.5065,1.40661",1.13749",29.8591) #  
  ...
  

• In ds9: load the event file and display the source list on top of it. To save the regions in another format:

  1. Region → Save Regions... → Save As "acisf02254_src_ds9.reg".
  2. After choosing "OK" in the region filename dialog, a format dialog is opened. Set the format to "CIAO" and the coordinate system to "WCS" and "Equatorial J2000".

  The output (an ASCII region file) still contains the shape definitions, but the coordinates are now given as sexagesimal:

  unix% cat acisf02254_src_ds9.reg
  # Region file format: CIAO version 1.0
  # Region file format: DS9 version 4.1
  global color=green dashlist=8 3 width=1 font="helvetica 10 normal roman" select=1 highlite=1 dash=0 fixed=0 edit=1 move=1 delete=1 include=1 source=1
  fk5
  ellipse(14:11:23.4379,52:13:31.7345,0.966892",0.926778",63.2685) #  
  ellipse(14:11:20.7233,52:12:7.7454,1.16711",1.04867",72.128) #  
  ellipse(14:11:20.5543,52:12:9.7368,1.04189",0.989279",49.0305) #  
  ellipse(14:11:20.4445,52:12:11.4486,1.17873",1.01639",138.111) #  
  ellipse(14:11:20.4420,52:12:15.9387,1.24158",1.20135",85.0671) #  
  ellipse(14:11:13.6337,52:13:41.1066,1.1083",0.851713",45.716) #  
  ellipse(14:11:33.0335,52:11:2.5639,1.67152",1.47723",54.7683) #  
  ellipse(14:11:30.8464,52:14:23.7191,1.62874",1.31701",95.4704) #  
  ellipse(14:11:27.4170,52:11:31.5065,1.40661",1.13749",29.8591) #  
  ...
  

  The file may be redisplayed in ds9, as it retains the CIAO region format.

• With prop_precess: prop_precess is a proposal tool that may be used to convert the (RA,Dec) in the FITS source list from decimal to sexagesimal format.

  unix% dmlist acisf02254_src.fits"[cols ra,dec]" data,clean > decimal.txt
  
  unix% prop_precess f j2000/deg t j2000/hms : decimal.txt : hms.txt
  Opened input file decimal.txt
  Opened output file hms.txt
  
  unix% cat hms.txt
  RA,Dec J2000.0                          RA,Dec J2000.0
  212.847658    52.225482                 14 11 23.44 +52 13 31.73
  212.836347    52.202152                 14 11 20.72 +52 12 07.75
  212.835643    52.202705                 14 11 20.55 +52 12 09.74
  212.835185    52.203180                 14 11 20.44 +52 12 11.45
  212.835175    52.204427                 14 11 20.44 +52 12 15.94
  212.806807    52.228085                 14 11 13.63 +52 13 41.11
  212.887640    52.184046                 14 11 33.03 +52 11 02.56
  212.878527    52.239922                 14 11 30.85 +52 14 23.72
  212.864238    52.192085                 14 11 27.42 +52 11 31.51
  212.862784    52.237534                 14 11 27.07 +52 14 15.12
  212.836261    52.236454                 14 11 20.70 +52 14 11.23
  ...
  

Creating Individual Spectra

It is possible to create a spectrum from any of the regions in the source list without having to divide up the file. To extract a PI spectrum of the second source in the file and bin it by a factor of 5:

unix% punlearn dmextract
unix% dmextract "acisf02254_repro_evt2[sky=region(acisf02254_src.fits[#row=2])][bin pi=::1]" \
      spectrum.fits

The contents of the parameter file may be checked using plist dmextract.

The resultant spectrum can be plotted in python using matplotlib.

$ python

from pycrates import read_file
import matplotlib.pyplot as plt

tab = read_file("spectrum.fits")
xcol = tab.get_column("channel")
ycol = tab.get_column("counts")

def mklabel(col):
    lab = col.name
    if hasattr(col, "unit") and len(col.unit)>0:
        lab = lab + ' [{}]'.format(col.unit)
    return(lab)

xlab=mklabel(xcol)
ylab=mklabel(ycol)

plt.plot(xcol.values, ycol.values)
plt.xlim(0, 100)
plt.xlabel(xlab)
plt.ylabel(ylab)
plt.title("File: spectrum.fits")
plt.savefig("plot1.png")

Figure 4 shows the plot. This data may also be fit in Sherpa, once the proper response files have been created; see the Step-by-Step Guide to Creating ACIS Spectra for Pointlike Sources thread for more information.

Figure 4: Plot of the spectrum

[Version: full-size]

Figure 4: Plot of the spectrum

This spectrum was created from a single source in the event file (id=2).

Creating Multiple Spectra

To create an individual spectrum for each of the sources, use the igrid(1:n:steps) stack syntax, where n is the number of sources in the file.

For acisf02254_src.fits, which has 15 sources in it:

unix% dmextract "acisf02254_repro_evt2[sky=region(acisf02254_src.fits[component=igrid(1:82:1)])][bin pi=::1]" \
      multi_spectra_phaII.fits op=pha2

The output file is a type II PHA file that has one spectrum per row:

unix% dmlist multi_spectra_phaII.fits blocks
 
--------------------------------------------------------------------------------
Dataset: multi_spectra_phaII.fits
--------------------------------------------------------------------------------
 
     Block Name                          Type         Dimensions
--------------------------------------------------------------------------------
Block    1: PRIMARY                        Null        
Block    2: SPECTRUM                       Table         7 cols x 82       rows
Block    3: GTI3                           Table         2 cols x 3        rows
Block    4: GTI6                           Table         2 cols x 5        rows
Block    5: GTI7                           Table         2 cols x 1        rows
Block    6: GTI2                           Table         2 cols x 6        rows
Block    7: GTI1                           Table         2 cols x 3        rows
Block    8: GTI0                           Table         2 cols x 3        rows

For more information on working with this type of file, see the Displaying the Spectrum section of the Examining Grating Spectra and Regions: PHA2 files thread (the information in that section applies to imaging files as well).

Creating a Composite Spectrum

It is also possible to extract a composite spectrum of all the (faint) sources in a field for a single spectral analysis. This may be useful in the case where the sources might be of similar characteristics, but there are not enough counts in any given source to do individual spectra.

Here we have deleted the sources associated with 3C295 (IDs 3-5):

unix% dmcopy "acisf02254_src.fits[#row=1:2,6:]" acisf02254_nocore_src.fits

Now we can use dmextract to accept all the regions together and create a spectrum:

unix% dmextract "acisf02254_repro_evt2[sky=region(acisf02254_nocore_src.fits)][bin pi=::1]" \
      spectrum_faint.fits

Again, the resultant spectrum can be plotted :

$ python

from pycrates import read_file
import matplotlib.pyplot as plt

tab = read_file("spectrum_faint.fits")
xcol = tab.get_column("channel")
ycol = tab.get_column("counts")

def mklabel(col):
    lab = col.name
    if hasattr(col, "unit") and len(col.unit)>0:
        lab = lab + ' [{}]'.format(col.unit)
    return(lab)

xlab=mklabel(xcol)
ylab=mklabel(ycol)

plt.plot(xcol.values, ycol.values)
plt.xlim(0,100)
plt.xlabel(xlab)
plt.ylabel(ylab)
plt.title("File: spectrum_faint.fits")
plt.savefig("plot2.png")

Figure 5 shows the plot.

Figure 5: Composite spectrum

[Version: full-size]

Figure 5: Composite spectrum

The composite spectrum was created by combining counts from 14 sources.

As mentioned before, this data may also be fit in Sherpa, once the proper response files have been created; see the Step-by-Step Guide to Creating ACIS Spectra for Pointlike Sources thread for more information.

Removing Unwanted Point Sources

Often it is desirable to remove point sources from an image for both cosmetic and scientific reasons. For example, if you are analyzing an extended source, an unfortunately located point source may affect the count rates in the object.

Suppose that we want to remove all the point sources from our field of view, except for the three in the core (sources 3-5). First, create a source list that excludes these three regions:

unix% dmcopy "acisf02254_src.fits[#row=1:2,6:15]" acisf02254_holes_src.fits

Now filter the event file with those regions:

unix% dmcopy "acisf02254_repro_evt2[exclude sky=region(acisf02254_holes_src.fits)]" acisf02254_holes_evt2.fits

unix% ds9 acisf02254_holes_evt2.fits &

The resulting image, Figure 6, has holes where the sources used to be. The Obtain and Fit a Radial Profile thread illustrates another case in which you might use this syntax (see the Removing Contaminating Point Sources section).

Figure 6: Event file with sources removed

[Version: full-size]

Figure 6: Event file with sources removed

The source regions are replaced by areas of zero counts.

A more sophisticated means of removing point sources is to use dmfilth. This tool removes the point source and then fills in the hole where the source was. See ahelp dmfilth for more information and the Identify and Remove Point Sources section of the Image of Diffuse Emission thread for an example.