Instructions for ACA Dark Current Analysis

December 2015 - JMC

For a new dark current calibration occurring on YYYYDOY (where YYYYDOY will be something like 2005247):

Confirm that kadi dark current intervals are complete

> ska
> python
> from kadi.events import dark_cal_replicas
# For the replicas in the last 7 days
> dark_cal_replicas.filter(-7)

If the expected number of replicas are printed, continue with these processing instructions, else wait a day.

Set up a new data directory

Move to the data area and create a new directory:

> cd /proj/sot/ska/data/aca_dark_cal
> mkdir YYYYDOY

Retrieve the data

Data products should be available on the OPS LAN shortly after the measurement is completed.
Fetch the data using something like:

>: ssh c3po-v
> rsync /dsops/GOT/input/YYYY_DOY_VC2*.gz" /proj/sot/ska/data/aca_dark_cal/YYYYDOY/

Prepare Data

Unzip all telemetry files

> gunzip YYYYDOY/*_VC2*.gz

Validate data quality

Look at the filesizes of the VC2 telemetry files, they are typically about 30 MB. If one (or a few) of the files is particularly small then the set of minor frame segments is probably incomplete.

Create info file

Copy the info.csv file from the last dark current directory to the new directory. Edit this text file by hand to include the time, temperature, ra, and dec of the first dark current star catalog, retrieved from the starcheck page for the dark current week ( with temperature set at the expected commanded CCD temp ~ -19C ). Older directories use an info.fits format; either can be read with plot_dark.pro (which is called by dark_curr.pl below).

Process data and generate plots and html

> dark_curr.pl YYYYDOY

Update mica archive with this calibration data

First, check to see if the most recent dark current calibration has been archived. It is fast to just look to see if there is a directory for it:

> ls /data/aca/archive/aca_dark

You should see a directory in there with the YYYYDOY name. If not, update the mica archive by running the task schedule

> touch /proj/sot/ska/data/mica/task_sched_heart_attack
# Wait a minute, then remove the heart attack
> rm /proj/sot/ska/data/mica/task_sched_heart_attack
# Remove a heart beat file if it wasn't cleaned up > rm /proj/sot/ska/data/mica/task_sched_heartbeat # Run the update > /proj/sot/ska/bin/task_schedule.pl -config /proj/sot/ska/data/mica/task_schedule.cfg

This may take ~15 minutes and runs as a background task; the return of the prompt does not mean the job has completed. Check the log to see if ingest was successful:
tail -n 50 /proj/sot/ska/data/mica/logs/daily.1/mica_archive.log
(In testing, run asynchronously, I found the log for the most recent processing in daily.1, it may also be in daily.0 -JC) Look for the status of update_aca_dark.py and text like

Making output dark cal directory /data/aca/archive/aca_dark/2015355

Also check the dark cal archive directory to see if there is a new directory as expected

> ls -d /data/aca/archive/aca_dark/YYYYDOY

Make warm pixel plots

> /proj/sot/ska/share/aca_dark_cal/warm_pix_corr.py --out YYYYDOY/Result/

Setup the new dark current calibration report.

Make and goto new directory:

> mkdir /proj/sot/ska/www/ASPECT/dark_cal/<YYYY>_<MON>
> cd !$

Copy gif, png, and html files from the previous section

> cp /proj/sot/ska/data/aca_dark_cal/YYYYDOY/Result/*[.gif,.html,.png]; .

Edit the main Aspect page index in /proj/sot/ska/www/ASPECT/index.html to point to the new report by adding a link to dark_cal/YYYY_MON in the dark current analysis section.
Add a link from the dark_cal list /proj/sot/ska/www/ASPECT/dark_cal/index.html to this new calibration as well.
Update the TWiki with the links to this analysis.