Using CIAO on SciServer
What is SciServer?
SciServer is a fully integrated cyberinfrastructure system encompassing related tools and services to enable researchers to cope with scientific big data. SciServer enables a new approach that will allow researchers to work with Terabytes or Petabytes of scientific data, without needing to download any large datasets.
More specifically, SciServer is an online analysis platform that allows users to remotely perform data analysis. SciServer uses the Jupyter Labs framework to allow users to create and run Jupyter notebooks as well as Terminal/Shell commands.
The High Energy Astrophysics Science Archive Research Center (HEASARC) has made CIAO and all public Chandra data available on the SciServer Science Platform.
Getting Started
HEASARC provides a comprehensive Getting Started guide . Users should select the latest HEASARC container image and be sure to include the HEASARC data to have access to the Chandra CALDB and all public Chandra datasets.
Using CIAO
Once logged into SciServer, go to the "Compute" app and then click on the name of the container you created.
The First tab that will open will be the "HEASARC Sciserver Tutorials" tab. Click on the "+", next to "Introduction.md" to add a new tab.
From there you can choose to start a new Python Jupyter notebook tab: Notebook -> (ciao).
Alternatively you can create a new terminal window to run CIAO from the bash command line (Other -> Terminal). Since most of the CIAO documentation is written using Terminal/Shell examples then for beginners learning CIAO it might be easier to use the Terminal window. To setup for ciao type
conda activate ciao
Using SAOImageDS9
If you are using a Jupyter notebook to perform your analysis then it may be easier to use JS9 rather than DS9 to display images, create regions, etc. The HEASARC instructions provide information on how to start JS9 and interact with it in Python.
The CIAO examples all use DS9 for image display and analysis. Remarkably, HEASARC also has DS9 running in a Jupyter notebook. You can start DS9 by adding a new tab by clicking on the "+" along the top of the Jupyter lab window and selecting "DS9". Note: it has been observed that sometimes the window fails to start the first time the tab is launched. If you encounter a problem, try closing the tab and then adding it back again.
The version of DS9 running on sciserver does not have access to CIAO's Data Analysis eXtensions. To add DAX, you need to install CIAO's version of DS9. From the Terminal tab, with after activating the CIAO environment, you can need to run
conda install -c https://cxc.cfa.harvard.edu/conda/ciao ds9 --freeze-installed --yes
After this, dax will be available when you run 'ds9' from the Terminal with CIAO activated.
Accessing Chandra Data
Public chandra data can be retrieved using the download_chandra_obsid script as per usual or in Python using the download_chandra_obsids routine.
HEASEARC also maintains an up-to-date copy of the public Chandra archive on SciServer. The easiest way to locate a specific observation is to go to
cd /FTP/chandra/data/byobsid/#/####
where "#" is the last digit of the OBS_ID, and "####" is the OBS_ID without any leading zeros. So for example to get data for OBS_ID=635, go to
cd /FTP/chandra/data/byobsid/5/635
Proprietary data must be copied onto the SciServer platform by the individual with access to the data.