Chandra Data Reprocessing

The CXC is currently reprocessing the data from all observations obtained since the start of the mission. This reprocessing is part of an ongoing program to continuously provide to observers and archival researchers access to data that have been processed using the most up-to-date algorithms and software, and the best available calibrations.

The first phase of reprocessing includes all data from the start of routine science observations (1999 September 18) up until the baseline reprocessing release the CXC data system (2000 August 15). We expect this phase of reprocessing to be completed by early March 2001. Reprocessed proprietary data are distributed automatically to the observers as soon as the datasets complete validation and verification.

All science data are processed automatically through the standard data processing pipelines, which produce scientific data products suitable for further analysis and publication. The pipeline software is enhanced as problems are detected, or as new and improved processing algorithms are developed. The latter often result from improvements in our understanding of the on-orbit behavior of the Chandra spacecraft and instruments. New and better calibration data are also developed as our knowledge of the instrumental performance and characteristics improves. Because of this, data processed with more recent versions of the data system have enhanced data quality and are better calibrated than data processed early in the mission.

In addition, reprocessed data have better formatting and improved ease of use with the CXC data analysis package CIAO. The CIAO 2 science analysis threads (see http://cxc.harvard.edu/ciao/documents_threads.html)

are designed specifically to be used with reprocessed data. Users may determine whether their data have been through reprocessing by checking the value of the ``ASCDSVER" keyword in the FITS header of the data products. If ``ASCDSVER" is ``R4CU5UPD8" or higher, then the data were processed through the baseline reprocessing release or a later release of the data system.

Listed below is a brief summary of the major changes that are present in reprocessed data, compared to data processed through the version of the data system that was available at the start of the mission.

$\bullet$ More precise data time stamping, exclusion of corrupted data, and filling of data gaps (when possible).

$\bullet$Improved handling of ACIS continuous clocking and interleaved mode data.

$\bullet$Detection and removal of ACIS flaring pixels (due to cosmic-ray events).

$\bullet$New HRC event validity tests reject background events and eliminates ghosts.

$\bullet$Numerous fixes to good time interval determination for ACIS multi-chip observations.

$\bullet$Revisions to event file columns and formats, particularly for level 2.

$\bullet$The detector background lightcurve is added to the mission time line data product.

$\bullet$An enhanced, robust image centroider improves the Aspect solution; better, time-dependent calibrations improve absolute astrometry.

$\bullet$Observation summary pages are now created and distributed with the data.

$\bullet$Numerous improved calibrations are used, and are accessed via the Chandra calibration database (CALDB).

$\bullet$Numerous bug fixes are included.

Although all users will benefit significantly from using data that have been through phase 1 reprocessing, there are some data which will be further improved by new calibrations or enhanced algorithms that are not yet available or that were not available when phase 1 reprocessing commenced. These data will be re-reprocessed in phase 2 reprocessing, which will begin shortly after the completion of phase 1, and include:

$\bullet$ACIS continuous clocking (CC) mode observations (improved coordinate handling);

$\bullet$ACIS interleaved mode observations (improved dead time calibrations, and better handling of grating spectral extraction);

$\bullet$HRC-I observations (improved degap calibration);

$\bullet$HRC-S high precision timing observations (precise time-stamping of events).

In addition, it is possible that all HRC observations will be reprocessed in phase 2 due to changes in the degap calibrations.

Additional information about reprocessing may be obtained from the CXC Reprocessing Information web page:

http://cxc.harvard.edu/cdo/reprocessing.html

and also from the CIAO Reprocessing Summary web page:

http://cxc.harvard.edu/ciao/documents_reprocessing.html

Information about changes to the standard data processing pipelines that affect user data products can be found on the Standard Data Processing Release Notes web page:

http://cxc.harvard.edu/proc_stat/rel_notes/release_notes.html

- Ian Evans