Introduction to the Data Products

Archive versions

When a Chandra observation is taken, it is processed through the Standard Data Processing (SDP) pipeline, and the results stored in the Chandra archive. Occasionally the Chandra X-ray Center determines that there have been enough improvements to the software, or changes in the calibration products, that the archive - or a significant fraction of it - has to be reprocessed. This means that new version of the files will be created; so, if you downloaded an observation a long time ago, it is worth checking the archive to see if a new version is available. At the time of writing (February 22, 2022), the last reprocessing run was Repro V.

The current processing status, and history, of an observation can be found from the Chandra Observations Processing Status page. For ObsId 1843, the page looks like Figure 1 as of February 2022.

Figure 1: Archive processing status of ObsId 1843

[Version: full-size]

Figure 1: Archive processing status of ObsId 1843

The final table on the page shows that the data has been processed four times, the last time being in March 2021. This is why the "version number" for many of the files is N004 - e.g. acisf01843N004_evt2.fits.

The note on processing versions thread explains the version names used for the SDP pipeline, and how to check what version was used to create a particular dataset.

ObsID Directory

The ObsID directory is the top level in the directory tree created when the tarfile is unpacked. It contains two files and two data directories:

unix% pwd
/intro_data/1843

unix% ls
00README
axaff01843N003_VV001_vv2.pdf
oif.fits
primary/
secondary/

V&V report
axaff01843N003_VV001_vv2.pdf

All standard data products are checked by a CXC scientist before release to ensure data quality and to investigate the cause of any exposure losses or other anomalies. This process is known as "verification and validation", or V&V for short. The V&V report is distributed in PDF format with the data files.

This file contains a summary report of the V&V report. The full report is in a second PDF file (axaff01843N003_VV001_vvref2.pdf), which is packaged in the secondary data directory.

Users should review this information before beginning the analysis to ensure that there aren't any caveats from the V&V scientist.

Observation index file
oif.fits

The observation index file contains a summary of the data products associated with an observation. It is equivalent to a structured directory listing with descriptive information about the observation.

README
00README

Each observation includes an ASCII README file that provides information on how to cite Chandra data in publications, how to acknowledge usage of CIAO and Sherpa, and how to use the Chandra X-ray Center as a long term repository for aggregated high-level science data products.

In particular, the CXC encourages users to include Dataset Identifiers in their publications.

In the following sections, we describe the contents of the primary and secondary data directories.

Primary Directory

The data products are arranged such that all products necessary for most analysis (e.g. the CIAO threads) are in the primary directory.

unix% cd primary/
unix% pwd
/intro_data/1843/primary

unix% ls
acisf01843N004_cntr_img2.fits
acisf01843N004_cntr_img2.jpg
acisf01843N004_evt2.fits
acisf01843N004_full_img2.fits
acisf01843N004_full_img2.jpg
acisf01843_000N004_bpix1.fits
acisf01843_000N004_fov1.fits
orbitf084197100N001_eph1.fits
pcadf01843_000N001_asol1.fits
[hrc_dtf1.fits]
[acis_plt2.jpg]
[acis_pha2.fits]

Note

For a small number of observations the output will look different to above, because the observation was taken in interleaved mode - in which case there are two versions of certain files, labeled with e1 and e2, - or it is a multiple observation interval observation - when the level 1 and 1.5 data products can have multiple versions.

The majority of these files can be grouped into three categories:

Images

Two images are produced for every dataset: a full-field sky image (full_img2.jpg) and a high-resolution image of the center of the field (cntr_img2.jpg). FITS images are only created for imaging observation (no gratings).

Images are not created for ACIS operating in continuous clocking mode.

Imaging observations also have a full field image with the source candidates overlaid (src_img2.jpg). [The source images are no longer created as of SDP version DS 8.4 (June 2011).]

Grating observations include an image of each spectral component plotted as wavlength vs counts (heg_plt2.jpg, meg_plt2.jpg, and/or leg_plt2.jpg). [The grating images are no longer created as of SDP version DS 8.0 (July 2009).]

Remove those three sets of files from the directory listing and you have the primary data products necessary for most types of analysis:

acisf01843N004_evt2.fits
acisf01843_000N004_bpix1.fits
acisf01843_000N004_fov1.fits
orbitf084197100N001_eph1.fits
pcadf01843_000N001_asol1.fits
[hrc_dtf1.fits]
[acis_pha2.fits]

Level 2 event file
evt2.fits

The level 2 event file is the most important data product you receive. This file is created from the level 1 event list by filtering on the good time intervals (GTI) and status bits. The result is a list of events that is suitable for use in data analysis.

When new calibration is released, it is often necessary to remake the level 2 event file in order to apply the changes. This process is explained in the Create a New Level 2 Event File thread.

Source list
src2.fits

A source candidate list. These detections are obtained from a single run of celldetect with S/N threshold equal to 3 and should not be used directly in your analysis. Users interested in spatial analysis should perform their own detection runs with one of the three Chandra Detect tools: celldetect, wavdetect, and vtpdetect. The Detect Manual describes which tool is suited to a particular analysis.

For instructions on displaying the source list over the data, refer to the Using the Output of Detect Tools thread.

This file is only created for imaging observations.

[These files are no longer created as of SDP version DS 8.4 (June 2011).]

Bad pixels
bpix1.fits

A list of pixels identified as "bad"; criteria for flagging a pixel are listed in the Bad Pixels dictionary entry. Any tool that reads this file will exclude the bad pixels from its calculations.

The Reprocessing Data to Create a New Level=2 Event File thread contains more information, including instructions on how to make a new ACIS bad pixel file, if necessary. The New Observation-Specific HRC Bad Pixel File thread explains when and how to make a new HRC bad pixel file.

Sky field of view
fov1.fits

The field-of-view file contains regions which describe the edges of each instrument chip (one polygon per chip). This region file can be used to filter event files or images to contain specific chips.

This file has been created since SDP version 6.9.0. The skyfov tool may be used to create an fov1.fits file for observations with an earlier processing version.

These files are used in the Chandra FootPrint service for searching the archive of publicly-available observations.

Orbit ephemeris
orbit_eph1.fits

The definitive orbit ephemeris information for the observation. This file is required if you wish to barycenter-correct the observation times in order to account for the difference in photon arrival times as the Earth and Chandra move around the Sun.

The orbit ephemeris is also needed if you are analyzing an object which was moving with respect to the observer when observed, e.g. a planet or comet.

Aspect solution
pcad_asol1.fits

The aspect solution describes the orientation of the telescope as a function of time. The detected position of an event and the corresponding telescope aspect are combined for an accurate determination of the celestial position of that event.

Since an aspect solution file is created for each stable aspect interval, there may be more than one pcad_asol1.fits file for an observation. All the files must be used whenever a tool requires the aspect solution as input.

Detailed information on the aspect solution files is available from the the Aspect Solution why topic.

Dead time factor
dtf1.fits

The dead time correction factors as a function of time. For ACIS observations, the dead time correction is recorded in the DTCOR header keyword. Since the HRC dead time correction factor varies during the observation, however, the value of DTCOR represents an average and the individual values are stored in the dtf1.fits file.

The Computing Average HRC Dead Time Corrections thread explains when you might need to recompute the deadtime statistics for your observation.

This file is only created for HRC observations.

Type II PHA / Spectra
pha2.fits

The PHA file contains a spectrum for one grating part and order in each row of the file. The number of rows varies depending on the detector and grating combination: e.g. ACIS/HETG data has twelve rows while HRC/LETG only has two.

Detailed information on the pha2.fits file is available from the Examining Grating Spectra and Regions thread.

This file is only created for grating observations.

Secondary Directory

If you are interested in reprocessing your data, you will also need the level 1 files, which are in the secondary directory.

unix% cd ../secondary/
unix% pwd
/intro_data/1843/secondary

unix% ls
acisf01843_000N004_evt1.fits
acisf01843_000N004_flt1.fits
acisf01843_000N004_msk1.fits
acisf01843_000N004_mtl1.fits
acisf01843_000N004_stat1.fits
acisf084271178N004_0_bias0.fits
acisf084271178N004_1_bias0.fits
acisf084271178N004_2_bias0.fits
acisf084271178N004_3_bias0.fits
acisf084271178N004_4_bias0.fits
acisf084271178N004_5_bias0.fits
acisf084272477N004_pbk0.fits
axaff01843N003_VV001_vvref2.pdf
[acis_evt1a.fits  -or-  hrc_evt1a.fits]
[hrc_std_flt1.fits]
[hrc_std_dtfstat1.fits]
aspect/
ephem/

The V&V report - axaff01843N003_VV001_vvref2.pdf - is explained in the ObsID Directory section.

Aspect offsets
aoff1.fits

A set of offsets versus time (e.g. ra_off=(ra-ra_nom)) . The one piece of remaining useful info is information about gaps in the aspect records. These are derived from the aspect solution files (pcad_asol1.fits, primary directory) for the observation.

Most users will never need this file in their analysis, instead using the aspect solution files directly.

[This file will no longer created as of SDP version DS 8.4.1 (October 2011).]

Level 1 event file
evt1.fits

The level 1 event file contains all the events recorded for the observation. While many of these events have a status bit set to flags them as "bad", none of the information has been removed. This file is filtered on GTIs and status bits to create the level 2 event file.

The evt1.fits file is also the starting point for reprocessing your data, as explained in the Create a New Level 2 Event File thread.

Level 1.5 event file (optional)
evt1a.fits

The ACIS or HRC Level 1.5 event file is created for observations that used the gratings (LETG or HETG). The file contains additional columns related to the gratings including the grating order , which arm (eg HEG vs MEG), and the resolved grating wavelength. All these values are based on the zeroth order location used in the processing.

The zeroth order location the regions assigned grating coordinates is contained in the REGION extension which is appended to this file.

Good time intervals
flt1.fits / std_flt1.fits

The GTI information for the observation, e.g. the start and stop times of all accepted time intervals over the observation. The major contributor to creating GTIs is information about when there is aspect data and when that aspect data is good. When the event file is filtered, the GTIs are stored as extensions of the data file, creating a dynamic record of the time filters applied to the data.

Note that this file is named slightly differently for ACIS (flt1.fits) and HRC (std_flt1.fits) observations.

Mask
msk1.fits

The mask file records the valid part of the detector element - ACIS CCD or HRC plate - used for the observation (i.e. the portion for which events can be telemetered). The active portion of an element may be smaller than the default regions if an observation was performed using subarrays or custom windows. This information is used when creating response files, such as ARFs.

Mission timeline
mtl1.fits

The mission timeline (MTL) consists of all time series data that affects the quality of the data, such as aspect and ephemeris data. This information is used to construct the GTI limits in the flt1.fits file. The MTL ahelp file has more information on the mission timeline.

This file is required by acis_process_events to apply the temperature-dependent CTI calibration to ACIS data.

Alignment offsets
soff1.fits

The science instrument module offsets versus time. These are derived from the aspect solution files (pcad_asol1.fits, primary directory) for the observation. The soff1.fits file is binned, so there's not a one-to-one match to the rows in the pcad_asol1.fits files.

Most users will never need this file in their analysis, instead using the aspect solution files directly.

[These files are no longer created as of SDP version DS 8.4 (June 2011).]

Exposure statistics
stat1.fits

The exposure statistics file contains up to seven extensions, each pertaining to the sequence of ACIS CCD exposures processed and recorded into telemetry.

Most users will never need this file in their analysis.

This file is only created for ACIS observations.

Dead time factor statistics
std_dtfstat1.fits

The dead time factor statistics information is derived from the HRC deadtime factor file (dtf1.fits, primary directory) for the observation.

Most users will never need this file in their analysis.

This file is only created for HRC observations.

Bias map(s)
bias0.fits

When the bad pixel list (bpix1.fits) is created, each bias map is searched for pixels whose bias values are either too low or too high. There is one bias map for each ACIS chip that was used for the observation.

These files are only created for ACIS observations.

Parameter block
pbk0.fits

The parameter block file is needed in conjunction with the bias maps when creating a new bad pixel list. It is used to determine observational parameters, such as which CCDs are active, the READMODE and DATAMODE, etc.

This file is only created for ACIS observations.

unix% cd aspect/
unix% pwd
/intro_data/1843/secondary/aspect

unix% ls -1
pcadf084265646N004_osol1.fits
pcadf084271087N004_aqual1.fits
pcadf084272206N004_osol1.fits
pcadf084278602N004_osol1.fits

unix% cd ../ephem/
unix% pwd
/intro_data/1843/secondary/ephem

unix% ls -1
anglesf01843_000N004_eph1.fits
lunarf084197100N001_eph1.fits
solarf084197100N001_eph1.fits

What now?

Once you have checked the V&V report for the observation, you should reprocess the data. This ensures that the latest calibration products have been used, and is often one of the first things we ask users of the Chandra HelpDesk.