Skip to the navigation links
Last modified: 9 Nov 2010
Where are the PDFs?

How to Download Chandra Data from the Archive

CIAO 4.2 Science Threads



Overview

Last Update: 9 Nov 2010 - the download_chandra_obsid script is a quick and easy method of download data files.

Synopsis:

ChaSeR is a graphical user interface for browsing and/or obtaining data from the Chandra Data Archive. The Archive contains over 800 public-domain datasets, which consist of all observations for which the proprietary period has expired, as well as calibration data. Note that proprietary data may also be accessed via ChaSeR, but it is password-protected from the general community.

[New] A script for downloading Chandra data from the Archive: download_chandra_obsid. To use the script, install the contrib tarfile, then run:

unix% download_chandra_obsid 1843

Purpose:

To download ObsID 1843 (ACIS-I, G21.5-0.9), which is used in all of the Introductory CIAO Threads.




Contents



Launching ChaSeR

Before beginning, download and install the Java application.

Once installed, launch ChaSeR from the command line:

unix% chaser &

Figure 1 shows the GUI that will appear.

[Thumbnail image: ChaSer, the GUI used download data from the Chandra Archive]

[Version: full-size]

[Print media version: ChaSer, the GUI used download data from the Chandra Archive]

Figure 1: The ChaSer GUI

The default settings of the ChaSer GUI when it is launched are to search all instrument configurations for all types and categories of observations.

WebChaSeR

If you do not wish to install ChaSeR, data may still be obtained through WebChaSeR, a web version of ChaSeR. While it has the same search capabilities as ChaSeR, it is not possible to select individual files for download. Using this interface, you can only select complete packages of primary and secondary products; see the "Select" step for information on the different packages.

Due to the slightly different functionality, portions of this thread - mainly the Three Steps to Using ChaSeR section - do not apply to users of WebChaSeR. The two applications are similar enough, however, that the thread may serve as a helpful guide.



The Three Steps to Using ChaSeR

Using ChaSeR to obtain data is a simple 3-step process:

1. Browse

Chandra Archive data may be browsed using any of the fields in the ChaSeR GUI, such as Instrument, Grating, ObsID, and Category. Since we are interested in an ACIS-I observation of G21.5-0.9, leave only the appropriate Instrument (ACIS-I) and Grating (NONE) options checked and enter the Target Name.

Figure 2 shows the completed form.


2. Select

The Search Results window brings up all the observations that match our search criteria, as shown in Figure 3.

The "Order By..." menu can be used to sort the results; here we have ordered them by "Observation ID."

Click on ObsID 1843 to highlight the row. After doing so one may:

Now the desired data products may be selected for download using the "Select primary products", "Select secondary products", and "Select supporting products" buttons.

The data products are arranged such that the products necessary for most analyses (e.g. the CIAO threads) are in the primary directory. If you are interested in reprocessing your data, you will also need the level 1 files, which are in the secondary directory. Each of the threads also includes a list of the files needed to complete it.

By default, all primary and secondary products are pre-selected. If you are only interested in downloading the specific files need for a thread, but need some help determing what those are, check the Standard Data Distribution Contents webpage. These tables list the file extensions (aka "Type") and the "Content"; the content keyword is how the products are labeled in the ChaSeR interface.

The selections are submitted using the "Add to retrieval list" button. You can then highlight a different observation and add the same (or different) data products to the cart for it. Once you have selected all the data you are interested in, proceed to the next step.


3. Retrieve

Click on "View retrieval list" to view the observations and data products submitted for selection (Figure 6).

[Thumbnail image: The retrieval list shows which files will be downloaded.]

[Version: full-size]

[Print media version: The retrieval list shows which files will be downloaded.]

Figure 6: The retrieval list

Since all the primary and secondary products were selected for retrieval, only the categories are listed. If specific files are chosen, they are listed individually.

One may see the list of data filenames and sizes (Figure 7) by clicking on the "Browse selected products" button.

[Thumbnail image: The Browse Results window lists the individual files which will be downloaded.]

[Version: full-size]

[Print media version: The Browse Results window lists the individual files which will be downloaded.]

Figure 7: Browse Results window

The total file size of the package is listed at the top of the screen. Use this information to make sure there is sufficient disk space to download and unpack the data.

The "Retrieve selected products" button is used to start the retrieval process. The files are compiled into a tarred package which is placed onto the anonymous FTP server (cda.harvard.edu) by default; the tarfile can also be saved directly to a local disk. When the download is complete, the ChaSeR window will look as shown in Figure 8. Note that the location of the logfile (chaser.log) and the FTP directory in which the tarfile was placed are given at the bottom of the GUI.

[Thumbnail image: The Completed download screen looks the same as the retrieval list, except the location of the tarfile on the FTP is added.]

[Version: full-size]

[Print media version: The Completed download screen looks the same as the retrieval list, except the location of the tarfile on the FTP is added.]

Figure 8: Completed download screen

When the download has completed, the location of the logfile (chaser.log) and the FTP directory in which the tarfile was placed are displayed at the bottom of the GUI.

The name and location of the tarfile is recorded in chaser.log, along with the ObsID that was downloaded:

unix% cat /export/chaser.log
[09/01/2008 14:55:46:310 EST]  Request retrieval of 1843
[09/01/2008 14:55:46:366 EST]  File(s) package_146_080109145545.tar.* will be available at cda.harvard.edu:/pub/srftp/I6bjzJSZ

When you are finished with ChaSeR, use the "Exit" button to close the GUI.



FTP and Unpack the Data

Get the data from the anonymous FTP server; note that the location of the tarfile will be different for your observation:

unix% ftp cda.harvard.edu
Connected to cda.harvard.edu (131.142.185.171).
220 cda2 FTP server (Version wu-2.6.2(1) Mon Oct 21 09:41:35 EDT 2002) ready.
Name (cda.harvard.edu:username): anonymous
331 Guest login ok, send your complete e-mail address as password.
Password:
230-
230-    Welcome to the FTP server at the CXC Science Center 
230-
230-            This server is cda.harvard.edu 
230-
230-If your FTP client crashes or hangs shortly after login please try
230-using a dash (-) as the first character of your password.  This will
230-turn off the informational messages that may be confusing your FTP
230-client.
230-
230-Publicly available files are in /pub
230-
230-Problems with this ftp site?  Contact cxcftp@head-cfa.harvard.edu
230-
230-    +---------------------------------------------------------+
230-    |                                                         |
230-    | PLEASE NOTE:         (ChaSeR users only)                |
230-    |                                                         |
230-    |    If the directory provided to you by ChaSeR appears   |
230-    |    to be empty, this is a sign that your tar file is    |
230-    |    not yet ready.  Please try again in a little while.  |
230-    |                                                         |
230-    |    If your directory seems to contain several tar       |
230-    |    files, e.g., file.tar.0, file.tar.1, ...             |
230-    |    retrieve all files, then untar as follows:           |
230-    |        cat file.tar.* | tar xvf -                       |
230-    |    We are limiting the size of individual files to      |
230-    |    less than 2 GB, but these files need to be           |
230-    |    concatenated before submitting them to tar.          |
230-    |                                                         |
230-    +---------------------------------------------------------+
230-
230 Guest login ok, access restrictions apply.
Remote system type is UNIX.
Using binary mode to transfer files.

ftp> cd /pub/srftp/I6bjzJSZ
250 CWD command successful.

ftp> get package_146_080109145545.tar
local: package_146_080109145545.tar remote: package_146_080109145545.tar
227 Entering Passive Mode (131,142,185,171,95,112)
150 Opening BINARY mode data connection for package_146_080109145545.tar (68024320 bytes).
226 Transfer complete.
68024320 bytes received in 9.55 secs (7e+03 Kbytes/sec)

ftp> bye
221-You have transferred 68024320 bytes in 1 files.
221-Total traffic for this session was 68026426 bytes in 2 transfers.
221-Thank you for using the FTP service on cda2.
221 Goodbye.

Unpack the tarfile:

unix% tar xvf package_0_030616095338.tar
1843/primary/acisf01843N002_src2.fits.gz
1843/primary/acisf01843N002_evt2.fits.gz
1843/primary/acisf01843N002_1_sum2.html.gz
1843/primary/acisf01843N002_2_sum2.html.gz
1843/primary/acisf01843N002_3_sum2.html.gz
.
. (output omited)
.
1843/secondary/acisf084271178N002_5_bias0.fits.gz
1843/secondary/axaff01843N001_VV001_vvref2.pdf.gz
1843/axaff01843N001_VV001_vv2.pdf.gz
1843/oif.fits

unix% ls
1843/
package_146_080109145545.tar

unix% rm package_146_080109145545.tar

If you downloaded data from multiple ObsIds, a directory will be created for each of them.

After gunzipping all the files, the directory looks like:

unix% cd 1843

unix% ls -1 *
axaff01843N001_VV001_vv2.pdf
oif.fits

primary:
acisf01843_000N002_bpix1.fits
acisf01843_000N002_fov1.fits
acisf01843N002_1_sum2.html
acisf01843N002_2_sum2.html
acisf01843N002_3_sum2.html
acisf01843N002_cntr_img2.fits
acisf01843N002_cntr_img2.jpg
acisf01843N002_evt2.fits
acisf01843N002_full_img2.fits
acisf01843N002_full_img2.jpg
acisf01843N002_src2.fits
acisf01843N002_src_img2.jpg
orbitf084197100N001_eph1.fits
pcadf084271087N002_asol1.fits

secondary:
acisf01843_000N002_aoff1.fits
acisf01843_000N002_evt1.fits
acisf01843_000N002_flt1.fits
acisf01843_000N002_msk1.fits
acisf01843_000N002_mtl1.fits
acisf01843_000N002_soff1.fits
acisf01843_000N002_stat1.fits
acisf084271178N002_0_bias0.fits
acisf084271178N002_1_bias0.fits
acisf084271178N002_2_bias0.fits
acisf084271178N002_3_bias0.fits
acisf084271178N002_4_bias0.fits
acisf084271178N002_5_bias0.fits
acisf084272477N002_pbk0.fits
aspect/
axaff01843N001_VV001_vvref2.pdf
ephem/

The data files are now ready to be used in CIAO. For a description of the files, read the Introduction to the Data Products thread.



History

03 Jan 2005 reviewed for CIAO 3.2: no changes
01 Dec 2005 reviewed for CIAO 3.3: no changes
01 Dec 2006 reviewed for CIAO 3.4: no changes
09 Jan 2007 updated for CIAO 4.0: filenames updated since ObsID 1843 went through Repro III
09 Jun 2008 updated image display to place figures inline with text
29 Sep 2008 corrected link for downloading Chaser
30 Dec 2008 reviewed for CIAO 4.1: no changes
25 Jan 2010 reviewed for CIAO 4.2: no changes
09 Nov 2010 the download_chandra_obsid script is a quick and easy method of download data files.

Return to Threads Page: Top | All | Intro

Where are the PDFs?
Last modified: 9 Nov 2010