Gallery: Filter

In this simple example where we filter on the sky column. The sky column is a virtual, vector column composed of the x and y columns. Since the units of the sky column are in physical pixels, the units of the region are also interpreted as physical pixels.

The region in this example is a single circle that we expressed directly as part of the file name's virtual file syntax.

The default behavior of the CXC datamodel is to set pixel values outside the region to 0, and to reduce the size of the image to the bounding box around the region. Both of these can be changed using the [opt ] directive.

This shows a gallery of most of the CIAO shapes that CIAO supports. The regions are stored in an ASCII file using the simple CIAO format.

The output image shows the input image filtered with the union of the individual shapes. This is the default logic. The individual shapes can also be intersected and/or negated.

The information about the filter is stored in the output file and is used downstream by CIAO in tools such as dmstat and dmextract. You can review the filters by looking at the file's subspace.

dmlist ngc1404_regfile.img subspace

You can get more complicated shapes by inverting regions and by using logical and and or.

In this example we see the same 3 shapes written in different orders produces different results. For simplicity we'll call the regions: A=circle(3683,3950,100), B=circle(3783,3950,100), and C=circle(3883,3950,100).

Left-most: A-B+C. In this example circle B is excluded from circle A. Writing it out with mathematical operators : A-B+C = A*!B+C = (A*!B)+C.

Left-of-Center: A+C-B. In this example shape B is moved to the end so that it is now only excluded from C: A+C-B = A+C*!B = A+(C*!B).

Right-of-Center: -B+A+C. First thing to note is the addition of the field() shape. Why? Remember that the minus operator is really two: multiplication and negation. So -B+A+C would literally be : *!B+A+C which is nonsensical and is actually illegal syntax. This command

dmcopy "ngc1404.img[sky=-circle(3783,3950,100)+circle(3683,3950,100)+circle(3883,3950,100)]" filter_logic3.img clob+

produces a error: "Bad syntax for region". The field() is added so that the shape has something to be excluded from. Now the expression is: field()-B+A+C = (field()*!B)+A+C. The field() includes everything and the circle is excluded from it; then the pixels inside circle A and circle C are then added back into the filter resulting in the image shown.

Right-most: A-B+C-B. Finally we show how to exclude circle B from both A and C it has to be explicitly expressed that way. A-B+C-B = (A*!B)+(C*!B).

This is the combination of 4 ellipses which have been constructed to create the exclusive OR (XOR) operation. The output retains the data that is in either ellipse, but not both of them. The ellipses are constructed with the same radii and center; the angle (measured from the +X axis) is different.

In CIAO, the minus operator, -, is actually short hand for two separate operations: and and not. So

Using this we can see how the above filter logic works. The data from the first ellipse is included, except for the region which overlaps the second ellipse. That is then union-ed (OR) with the data from the second ellipse, excluded for the region that overlaps the first ellipse.

This type of logic is not available in some other software systems. By removing the excluded regions, always from every overlapping shape, the result of this region file would be that all pixels would be excluded.

The figure above shows all four ellipses; however, since they overlap only two are visible. Also, since the ellipses are excluded, the red-hash line ds9 uses to indicate negation is seen. ds9 can only display the individual shapes, not the logic used to combine those shapes.

Note the use of [opt full]. The default behavior is to shrink the image to a bounding box around the filtered region, but using the full option the original image size is retained.

This is an example of using bounds() using the region file created in the above example. The individual shapes have not been used to construct the filter logic; rather the size/extent of each shape is used to create an overall bounding box around all the shapes in the region file.

The output image has been shrunk to the limits of the bounding box; unlike in the earlier example, the pixels "between" the shapes are included, since the bounds() syntax was used.

In this example we first take an ACIS raw image, acisf367640304N001_5E002_img0.fits, and compute the projections onto the X-axis. This is an image in ACIS chip coordinates: CHIPX goes along the X-axis and CHIPY along the Y-axis. Data are readout along the CHIPY direction to one of four amplifier nodes. Due to the high dynamic range of pixel values, it is difficult to display this image before background subtraction and therefore not shown here.

Due to the limited telemetry bandwidth, ACIS raw images are rarely telemetered to the ground. Usually, events are detected and the raw image buffer is flushed. However, for calibration and health-and-safety purposes, the CXC collects raw images from ACIS several times per year. Users must request these files from the archive operations group.

dmimgproject computes several statistics for each column (X-axis) or row (Y-axis) including the mean pixel value, median, standard deviation, etc. In this example, to replicate the on-orbit behavior we are going to use the mean value.

In the second command, dmimgreproject is used to replicate the per-column mean value back into each row of the image. The output image shown above. Each column is a constant value equal to the mean of the pixel values in that column. The four ACIS readout nodes are clearly visible as is the column-to-column variation within each node.

The small stripes of data on the far right-hand-side of the image is from the overclock pixels, which are not physically present on the CCD; a full discussion of the topic of overclock pixels is beyond the scope of this example.

First, we subtract the output bias map from the previous example from the original raw image. This removes all the node-to-node and column-to-column variations present in the raw exposure.

Then we apply a threshold where any pixels less than 0 are set to 0 using dmimgthresh. This is the image shown here.

What we see here is a ~10 second exposure on one of the front-side illuminated CCDs. The bright features are the result of charge liberated from cosmic rays hitting the CCD. The cosmic rays hit the CCD at various angles which leads to shapes of the charge trail. The point of entry is the narrow end of the charge trail. The charge tail blooms as the cosmic ray travels deeper into the Silicon substrate and further away from the influence of the charge-collecting gate structure.

There are also some low-amplitude, residual, column-by-column features which are acceptable given the other sources of noise in the image.

Here an upper threshold of 20 was applied. But rather than set the pixel values greater than 20 to 20, we instead set them to 0.

This is very similar to the on-board algorithm used to compute the ACIS bias images used for event detection.

The resulting image shown. The idea is that anything greater than this pre-determined threshold must be a cosmic ray. Since we expect the pixel values to be around zero now that we have removed the column-by-column variation, any pixel above a certain threshold is interpreted to be influenced by a cosmic ray charge bloom and the value is replaced with 0.

With the bright, central part of the charge clouds removed; only the edges remain giving the image a cratered look.

The previous steps were repeated with the other ACIS raw images. The result is a set of eight images which look similar to this example, each with its own unique pattern of cosmic ray crater edges.

To create the output file, the dmimgfilt tool takes the same point(0,0) from each of the images in the stack, computes the mean of those value and stores it in the output image. It then moves the point to the next pixel and repeats. This stack of input images is specified using the UNIX wildcard, *, syntax: acisf367640304N001_5E*crater.img.

The output then is simply the mean value from the same pixel taken from each file in the input stack. The output is scaled the same as the previous examples so we can see that the amplitude of the residual noise has been greatly decreased. There are still some artifacts.

Here we chose to use the mean to match what is done by the ACIS camera on-orbit. Given the nature of the noise and the limited number of data points, using a median or some type of clipped-mean may provide a flatter bias map.

This example shows a simple way to estimate the background by computing the median pixel value in an annulus around each pixel in the input image. This process works best when using a smoothed image, since the median of integer values is noisy.

The choice of the size of the annulus (ring) should be such that its inner radius is larger than the expected source size (ie PSF) and the outer radius is small enough to only sample the local background.

We can see in this example that the point sources have been effectively removed from the input image and the diffuse emission at larger spatial scales has been preserved.

This process imparts a low-amplitude, high-spatial frequency structure on the output. If that is undesirable, then users can simply smooth the output

aconvolve orion.bkg orion.sm_bkg "lib:gaus(2,5,5,5,5)"

This type of filter can also be considered a type of non-linear smoothing (taking the median rather than the mean or simple linear combination of values). We can see this in the output as we see the gaps between the chips have increased -- which can be very important when using this type of image for exposure corrections.

In this example we start by running dmimglasso to get a polygon region surrounding the diffuse emission in the background from the previous example. We actually run dmimglasso twice. Once to get an inner polygon, using a higher threshold of 0.5. This is the inner polygon; the pixel values inside this region will be filled in. Then we run dmimglasso again with a lower threshold of 0.4 to generate the outer polygon. The pixel values inside this region (but outside the interior polygon) will be used to compute the coefficients for the two-dimension second-order polygon.

Next we select only the first row of each of those region files using dmcopy. As we saw in another example the dmimglasso output may contain interior excluded regions. For this example we only want the outermost region which will be the first one in the output file.

Finally we then run dmfilth to fill in the hole. Using method=POLY it takes the pixel values in the background region (green polygon) that are not in the source region (white polygon) and uses them to do a 2D, second-order polygonal least-squares-fit over the source region.

In this example we take the regions from the previous example for the diffuse emission region (white curve) and replace the pixel values by sampling from a Poisson random variable using a count-rate determined from the events in the background (dashed green) region.

dmfilth requires that the background region must be completely disjoint from the source region when using method=POISSON. A co-located annulus is OK as long as there are no pixels at the edge which are included in both regions.

(Left) First we subtract the inner polygon from the outer polygon. Since they both started at the same location, the inner polygon must be entirely enclosed within the other polygon. Since the mask pixel values are either 1 (inside the polygon) or 0 (outside), if we subtract the inner mask from the outer mask, the polygon annulus that is left behind will be the region that is inside the outer polygon, but not inside the inner polygon.

(Right) Next we use the subtracted mask to filter the image using dmimgthresh. We set the expfile parameter to the mask file name. Then we set the threshold so that any pixels less than 1 are set to INDEF, which is the special value we use when we want the output pixel values to be NaN or BLANK (depending on the data-type).

The final image shows the filtered data using the ds9 heat color map. Pixel values that are outside the filtered region have been set to NaN and are colored blue.