`mica.archive.aca_dark`¶

`mica.archive.aca_dark.dark_cal`¶

mica.archive.aca_dark.dark_cal.dark_id_to_date(dark_id)[source]¶

Convert dark_id (YYYYDOY) to the corresponding DateTime ‘date’ format.

Parameters:	date – dark id (YYYYDOY)
Returns:	str in DateTime ‘date’ format

mica.archive.aca_dark.dark_cal.date_to_dark_id(date)[source]¶

Convert date to the corresponding YYYYDOY format for a dark cal identifiers.

Parameters:	date – any DateTime compatible format
Returns:	dark id (YYYYDOY)

mica.archive.aca_dark.dark_cal.get_dark_cal_dirs(*args, **kwds)[source]¶

Get an ordered dict of directory paths containing dark current calibration files, where the key is the dark cal identifier (YYYYDOY) and the value is the path.

Parameters:	source – source of dark cal directories (‘mica’\|’ska’)
Returns:	ordered dict of absolute directory paths

mica.archive.aca_dark.dark_cal.get_dark_cal_id(date, select='before')[source]¶

Return the dark calibration id corresponding to date.

If select is 'before' (default) then use the first calibration which occurs before date. Other valid options are 'after' and 'nearest'.

Parameters:	date – date in any DateTime format select – method to select dark cal (before\|nearest\|after)
Returns:	dark cal id string (YYYYDOY)

mica.archive.aca_dark.dark_cal.get_dark_cal_image(date, select='before', t_ccd_ref=None, aca_image=False)[source]¶

Return the dark calibration image (e-/s) nearest to date.

If select is 'before' (default) then use the first calibration which occurs before date. Other valid options are 'after' and 'nearest'.

Parameters:	date – date in any DateTime format select – method to select dark cal (before\|nearest\|after) t_ccd_ref – rescale dark map to temperature (degC, default=no scaling) aca_image – return an ACAImage instance instead of ndarray
Returns:	1024 x 1024 ndarray with dark cal image in e-/s

mica.archive.aca_dark.dark_cal.get_dark_cal_props(date, select='before', include_image=False, t_ccd_ref=None, aca_image=False)[source]¶

Return a dark calibration properties structure for date

If select is 'before' (default) then use the first calibration which occurs before date. Other valid options are 'after' and 'nearest'.

If include_image is True then an additional column or key image is defined which contains the corresponding 1024x1024 dark cal image.

Parameters:	date – date in any DateTime format select – method to select dark cal (before\|nearest\|after) include_image – include the dark cal images in output (default=False) t_ccd_ref – rescale dark map to temperature (degC, default=no scaling) aca_image – return an ACAImage instance instead of ndarray
Returns:	dict of dark calibration properties

mica.archive.aca_dark.dark_cal.get_dark_cal_props_table(start=None, stop=None, include_image=False, as_table=True)[source]¶

Return a table of dark calibration properties between start and stop.

If include_image is True then an additional column or key image is defined which contains the corresponding 1024x1024 dark cal image.

If as_table is True (default) then the result is an astropy Table object. If False then a list of dicts is returned. In this case the full contents of the properties file including replica properties is available.

Parameters:	start – start time (default=beginning of mission) stop – stop time (default=now) include_image – include the dark cal images in output (default=False) as_table – return a Table instead of a list (default=True)
Returns:	astropy Table or list of dark calibration properties

`mica.archive.aca_dark.dark_model`¶

`mica.archive.aca_dark.update_aca_dark`¶

mica.archive.aca_dark.update_aca_dark.get_ccd_temp(tstart, tstop)[source]¶

Get the best estimate of CCD temperature between tstart and tstop

Parameters:	tstart – start time in CXC seconds tstop – stop time in CXC seconds

mica.archive.aca_dark.update_aca_dark.get_dark_cal_id(date)[source]¶

Get the dark cal ID in the Ska dark current files corresponding to date. It is assumed the dark cal dirs are labeled by YYYYDOY within root.

This routine allows for plus/minus one day of slop.

Parameters:	date – Date of dark cal
Returns:	str in yeardoy format

mica.archive.aca_dark.update_aca_dark.get_zodi_props(dark_id)[source]¶

Get zodiacal light information for given dark_id.

Parameters are: (‘date’, ‘ra’, ‘dec’, ‘el’, ‘eb’, ‘sun_el’, ‘l_l0’, ‘zodib’).

Parameters:	dark_id – dark cal ID as YYYYDOY
Returns:	table Row object

mica.archive.aca_dark.update_aca_dark.main()[source]¶: Update all the Mica dark cal directories.

`mica.archive.aca_hdr3`¶

Experimental/alpha code to work with ACA L0 Header 3 data

class mica.archive.aca_hdr3.MSID(msid, start, stop, msid_data=None, filter_bad=False)[source]¶

ACA header 3 data object to work with header 3 data from available 8x8 ACA L0 telemetry:

>>> from mica.archive import aca_hdr3
>>> ccd_temp = aca_hdr3.MSID('ccd_temp', '2012:001', '2012:020')
>>> type(ccd_temp.vals)
'numpy.ma.core.MaskedArray'

When given an msid and start and stop range, the object will query the ACA L0 archive to populate the object, which includes the MSID values (vals) at the given times (times).

The parameter msid_data is used to create an MSID object from the data of another MSID object.

When filter_bad is supplied then only valid data values are stored and the vals and times attributes are np.ndarray instead of ma.MaskedArray.

Parameters:	msid – MSID name start – Chandra.Time compatible start time stop – Chandra.Time compatible stop time msid_data – data dictionary or object from another MSID object filter_bad – remove missing values

filter_bad(copy=False)[source]¶

Filter out any missing values.

After applying this method the vals attributes will be a plain np.ndarray object instead of a masked array.

Parameters:	copy – return a copy of MSID object with bad values filtered

class mica.archive.aca_hdr3.MSIDset(msids, start, stop)[source]¶

ACA header 3 data object to work with header 3 data from available 8x8 ACA L0 telemetry. An MSIDset works with multiple MSIDs simultaneously.

>>> from mica.archive import aca_hdr3
>>> perigee_data = aca_hdr3.MSIDset(['ccd_temp', 'aca_temp', 'dac'],
...                                 '2012:001', '2012:030')

Parameters:	msids – list of MSIDs start – Chandra.Time compatible start time stop – Chandra.Time compatible stop time

`mica.archive.aca_l0`¶

Functions¶

mica.archive.aca_l0.get_slot_data(start, stop, slot, imgsize=None, db=None, data_root=None, columns=None)[source]¶

For a the given parameters, retrieve telemetry and construct a masked array of the MSIDs available in that telemetry.

>>> from mica.archive import aca_l0
>>> slot_data = aca_l0.get_slot_data('2012:001', '2012:002', slot=7)
>>> temp_ccd_8x8 = aca_l0.get_slot_data('2005:001', '2005:010',
...                                     slot=6, imgsize=[8],
...                                     columns=['TIME', 'TEMPCCD'])

Parameters:	start – start time of requested interval stop – stop time of requested interval slot – slot number integer (in the range 0 -> 7) imgsize – list of integers of desired image sizes (defaults to all -> [4, 6, 8]) db – handle to archive lookup table data_root – parent directory that contains archfiles.db3 (for use when db handle not available) columns – list of desired columns in the ACA0 telemetry (defaults to all in 8x8 telemetry)
Returns:	data structure for slot
Return type:	numpy masked recarray

mica.archive.aca_l0.get_files(obsid=None, start=None, stop=None, slots=None, imgsize=None, db=None, data_root=None)[source]¶

Retrieve list of files from ACA0 archive lookup table that match arguments. The database query returns files with

tstart < stop and tstop > start

which returns all files that contain any part of the interval between start and stop. If the obsid argument is provided, the archived obspar tstart/tstop (sybase aca.obspar table) are used.

>>> from mica.archive import aca_l0
>>> obsid_files = aca_l0.get_files(obsid=5438)
>>> time_files = aca_l0.get_files(start='2012:001', stop='2012:002')
>>> time_8x8 = aca_l0.get_files(start='2011:001', stop='2011:010',
...                             imgsize=[8])

Parameters:	obsid – obsid start – start time of requested interval stop – stop time of requested interval slots – list of integers of desired image slots to retrieve (defaults to all -> [0, 1, 2, 3, 4, 5, 6, 7, 8]) imgsize – list of integers of desired image sizes (defaults to all -> [4, 6, 8]) db – handle to archive lookup table data_root – parent directory of Ska aca l0 archive
Returns:	interval files
Return type:	list

mica.archive.aca_l0.get_l0_images(start, stop, slot, imgsize=None, columns=None)[source]¶

Get ACA L0 images for the given start and stop times and the given slot. Optionally filter on image size via imgsize or change the default image metadata via columns.

>>> from mica.archive import aca_l0
>>> imgs = aca_l0.get_l0_images('2012:001', '2012:002', slot=7)
>>> imgs = aca_l0.get_l0_images('2005:001', '2005:002', slot=6, imgsize=[8])

The default columns are: [‘TIME’, ‘IMGROW0’, ‘IMGCOL0’, ‘BGDAVG’, ‘IMGSTAT’, ‘IMGFUNC1’, ‘IMGSIZE’, ‘INTEG’]

The image pixel values are given in units of DN. One can convert to e-/sec by multiplying by (5 / INTEG).

Parameters:	start – start time of requested interval stop – stop time of requested interval slot – slot number integer (in the range 0 -> 7) imgsize – list of integers of desired image sizes (default=[4, 6, 8]) columns – image meta-data columns
Returns:	list of ACAImage objects

`mica.archive.asp_l1`¶

Script to update Ska file archive aspect L1 products. Module also provides methods to retrieve the directory (or directories) for an obsid.

This uses the obsid_archive module with a configuration specific to the aspect L1 products.

Functions¶

mica.archive.asp_l1.get_files(obsid=None, start=None, stop=None, revision=None, content=None)[source]¶

List asp_l1 files for an obsid or a time range.

>>> from mica.archive import asp_l1
>>> obs_files = asp_l1.get_files(6000)
>>> obs_gspr = asp_l1.get_files(6000, content=['GSPROPS'])
>>> range_fidpr = asp_l1.get_files(start='2012:001',
...                                stop='2012:030',
...                                content=['FIDPROPS'])

Parameters:	obsid – obsid start – time range start (Chandra.Time compatible) stop – time range stop (Chandra.Time compatible) revision – revision integer or ‘last’ defaults to current released version content – archive CONTENT type defaults to all available ASP1 types
Returns:	full path of files matching query

mica.archive.asp_l1.get_dir(obsid)[source]¶

Get ASP L1 directory for default/released products for an obsid.

>>> from mica.archive import asp_l1
>>> asp_l1.get_dir(2121)
'/data/aca/archive/asp1/02/02121'

Parameters:	obsid – obsid
Returns:	directory
Return type:	string

mica.archive.asp_l1.get_obs_dirs(obsid)[source]¶

Get all ASP L1 directories for an obsid in the Ska file archive.

>>> from mica.archive import asp_l1
>>> obsdirs = asp_l1.get_obs_dirs(6000)

obsdirs will look something like:

{'default': '/data/aca/archive/asp1/06/06000',
2: '/data/aca/archive/asp1/06/06000_v02',
3: '/data/aca/archive/asp1/06/06000_v03',
'last': '/data/aca/archive/asp1/06/06000',
'revisions': [2, 3]}

Parameters:	obsid – obsid
Returns:	map of obsid version to directories
Return type:	dictionary

`mica.archive.obsid_archive`¶

Base Class used in mica.archive.asp_l1 and mica.archive.obspar

Generalized module for fetching and archiving obsid-organized telemetry such as asp_l1 and obspar products.

class mica.archive.obsid_archive.ObsArchive(config)[source]¶

Object to store configuration, logging, and processing tasks to fetch obsid telemetry from the CXC archive and store in a Ska file archive, while logging the archive files to a file lookup database.

The configuration dictionary config may have these key/values:

data_root: directory for products

(example /data/aca/archive/asp1)
temp_root: directory for temporary storage of fetched

telemetry
bad_obsids: file containing list of obsids that should be

ignored when in regular update mode
cols: headers that will be included in file lookup table
sql_def: sql file to build file lookup archfiles table
apstat_table: axafapstat database table from which to find

new processing (by id)
apstat_id: field in apstat_table to use as unique CXCDS

processing id
label: label of product type for log messages
small: arc5gl keyword/filetype for small file from products

(example asp1{fidprops}). This will be retrieved with “get %s” % config[‘small’] and the retrieved files will be used to determine product version.
small_glob: glob to match files retrieved by

“get %s” % config[small] (example ‘fidpr‘)
small_ver_regex: regular expression to search for version from

retrieved files (example ‘pacdfd+N(d{3})_’)
full: arc5gl keyword for products (example ‘asp1’)
rebuild: If True/set, allow update mode to rebuild the database

from obsid 1.

Parameters:	config – configuration dictionary
Returns:	ObsArchive instance

get_all_obspar_info(i, f, archfiles)[source]¶: Read obspar and add ‘obsid’ and ‘filename’ keys to the dictionary i and archfiles are just passed to make the logging prettier.

get_arch(obsid, version='last')[source]¶

Retrieve telemetry for an observation from the CXC archive and store in the Ska file archive.

Parameters:	obsid – obsid version – ‘default’, ‘last’, or revision/version number
Returns:	obsid directory in Ska file archive
Return type:	directory string

get_arch_info(i, f, archfiles)[source]¶: Get information for a file for the file lookup table/database. For obspars, call the get_obspar_info() method. For FITS files, call get_fits_info() method.

get_dir(obsid)[source]¶: Return the latest released directory for an obsid Return None if there are no ‘default’ / released products.

static get_file_ver(tempdir, fileglob, ver_regex)[source]¶

Determine the version/revision of a set of archived files from their file names.

Parameters:	tempdir – directory containing files fileglob – glob to match files in question ver_regex – regular expression to pull out version from the set of files
Returns:	version number
Return type:	integer

get_fits_info(i, f, archfiles)[source]¶

Read FITS file f with index i (position within list of filenames archfiles) and get dictionary of values to store in file lookup database. This values include all header key/value pairs with keys in config[cols] plus the header checksum, the filename, the year, and day-of-year.

Parameters:	i – index of file f within list of files archfiles f – filename archfiles – list of filenames for this batch
Returns:	info for a file
Return type:	dictionary

get_obs_dirs(obsid)[source]¶: Return a dictionary of the directories available for an obsid. This is just done with a glob in the data directories.

get_obspar_info(i, f, archfiles)[source]¶: Wrap get_all_obspar_info() and just include columns in config[‘cols’]

get_todo_from_links(archive_dir)[source]¶: Return a list of all of the *_last directories in the file archive (and specify revision=default to attempt to get new released products for them).

get_ver_num(obsid, version='default')[source]¶

Determine the version number associated with the current released products or with the products referenced by “version=last”.

Parameters:	obsid – obsid version – version string (‘default’\|’last’)
Returns:	version
Return type:	integer

set_env()[source]¶: Set environment included an arc5gl handle and and a handle to the axafapstat database

set_read_env()[source]¶: Set environment included an arc5gl handle and and a handle to the axafapstat database

update()[source]¶: Run the update process using the config already passed to the object.

update_link(obsid)[source]¶

Create links in the obsid data directories to make it easy to find the current ‘default’/released data, all versions that have been archived, and the ‘last’/unreleased/provisional data if available.

This is designed so that if obsid 5 has released data in version 1 and provisional data in version 2, that the directories and links will look like:

directory 00005_v01 directory 00005_v02 link 00005 -> 00005_v01 link 00005_last -> 00005_v02

`mica.archive.obspar`¶

Script to update Ska file archive obspars. Module also provides methods to retrieve the directory (or directories) for an obsid.

This uses the obsid_archive module with a configuration specific to the obspar products.

Functions¶

mica.archive.obspar.get_dir(obsid)[source]¶

Get obspar directory for default/released products for an obsid.

>>> from mica.archive import obspar
>>> obspar.get_dir(2121)
'/data/aca/archive/obspar/02/02121'

Parameters:	obsid – obsid
Returns:	directory
Return type:	string

mica.archive.obspar.get_obs_dirs(obsid)[source]¶

Get all obspar directories for an obsid in the Ska file archive.

>>> from mica.archive import obspar
>>> obsdirs = obspar.get_obs_dirs(6000)

obsdirs will look something like:

{'default': '/data/aca/archive/obspar/06/06000',
2: '/data/aca/archive/obspar/06/06000_v02',
3: '/data/aca/archive/obspar/06/06000_v03',
'last': '/data/aca/archive/obspar/06/06000',
'revisions': [2, 3]}

Parameters:	obsid – obsid
Returns:	map of obsid version to directories
Return type:	dictionary

mica.archive.obspar.get_obspar(obsid, version='default')[source]¶

Get the obspar for obsid. Return as a dict.

>>> from mica.archive import obspar
>>> obspar.get_obspar(7001)['detector']
'ACIS-I'

Parameters:	obsid – obsid version – processing version/revision
Returns:	dictionary of obspar

`mica.vv`¶

Functions¶

mica.vv.get_vv(obsid, version='default')¶

Retrieve V&V data for an obsid/version. This reads the saved JSON and returns the previously- calculated V&V data.

Parameters:	obsid – obsid version – ‘last’, ‘default’, or version number
Returns:	dict of V&V data

mica.vv.get_vv_dir(obsid, version='default')¶

Get directory containing V&V products for a requested obsid/version, including plots and json.

Parameters:	obsid – obsid version – ‘last’, ‘default’ or version number
Returns:	directory name for obsid/version

mica.vv.get_vv_files(obsid, version='default')¶

Get list of V&V files available for a requested obsid/version.

Parameters:	obsid – obsid version – ‘default’, ‘last’ or version number
Returns:	list of files

mica.vv.get_rms_data()¶

Retrieve/return all data from RMS trending H5 archive

Returns:	numpy array of RMS data for each star/obsid/version

mica.vv.get_arch_vv(obsid, version='last')¶

Given obsid and version, find archived ASP1 and obspar products and run V&V. Effort is made to find the obspar that was actually used during creation of the ASP1 products.

Parameters:	obsid – obsid version – ‘last’, ‘default’, or revision number of ASP1 products
Returns:	mica.vv.Obi V&V object

`mica.utils`¶

Functions¶

mica.utils.get_timeline_at_date(date, timelines_db=None)[source]¶

Return timeline that contains a given date. The ‘timeline_loads’ query is used to give the mp_dir as well as datestart and datestop for the timeline.

Parameters:	date – Chandra.Time compatible date timelines_db – optional already-open handle to a cmd_states/timelines database.
Returns:	dictionary of appropriate record from timeline_loads table

`mica.starcheck`¶

Functions¶

mica.starcheck.get_starcheck_catalog(obsid, mp_dir=None, starcheck_db=None, timelines_db=None)¶

For a given obsid, return a dictionary describing the starcheck catalog that should apply. The content of that dictionary is from the database tables of that parsed the starcheck report and has keys:

cat: catalog rows as astropy.table
manvr: list of maneuvers to this attitude
pred_temp: predicted ACA CCD temperature
warnings: list of warnings below catalog in starcheck output
obs: dictionary of observation target and pointing information
mp_dir: directory with products that are the source of this catalog data
status: string describing status of that observation, described below.

Status:

ran: observation was observed
planned: observation in a not-approved future schedule
approved: observation in an approved future schedule (ingested in timelines/cmd_states)
ran_pretimelines: ran, but before timelines database starts
timelines_gap: after timelines database start but missing data

Parameters:	obsid – obsid mp_dir – mission planning directory (in the form ‘/2017/FEB1317/oflsa/’) to which to limit searches for the obsid. If ‘None’, get_mp_dir() will be used to select appropriate directory. starcheck_db – optional handle to already-open starcheck database timelines_db – optional handle to already-open timelines database
Returns:	dictionary with starcheck content described above

mica.starcheck.get_starcheck_catalog_at_date(date, starcheck_db=None, timelines_db=None)¶

For a given date, return a dictionary describing the starcheck catalog that should apply. The content of that dictionary is from the database tables that parsed the starcheck report. A catalog is defined as applying, in this function, to any time from the end of the previous dwell through the end of the dwell in which the catalog was used.

Star catalog dictionary with keys:

cat: catalog rows as astropy.table
manvr: list of maneuvers to this attitude
pred_temp: predicted ACA CCD temperature
warnings: list of warnings below catalog in starcheck output
obs: dictionary of observation target and pointing information
mp_dir: directory with products that are the source of this catalog data
status: string describing status of that observation, described below.

Status:

ran: observation was observed
planned: observation in a not-approved future schedule
approved: observation in an approved future schedule (ingested in timelines/cmd_states)
ran_pretimelines: ran, but before timelines database starts
timelines_gap: after timelines database start but missing data

Parameters:	date – Chandra.Time compatible date starcheck_db – optional handle to already-open starcheck database timelines_db – optional handle to already-open timelines database
Returns:	dictionary with starcheck content described above

mica.starcheck.get_mp_dir(obsid, starcheck_db=None, timelines_db=None)¶

Get the mission planning directory for an obsid and some status information. If the obsid catalog was used more than once (multi-obi or rescheduled after being used in a vehicle-only interval), return the directory and details of the last one used on the spacecraft.

The returned directory describes the directory that was used for the products with this star catalog. The returned status has possible values:

ran: observation was observed
planned: observation in a not-approved future schedule
approved: observation in an approved future schedule (ingested in timelines/cmd_states)
ran_pretimelines: ran, but before timelines database starts
timelines_gap: after timelines database start but missing data

The return ‘date’ is the date/time of the MP_STARCAT time.

Parameters:	obsid – obsid starcheck_db – optional handle to already-open starcheck database timelines_db – optional handle to already-open timelines database
Returns:	directory, status, date . Described above.

mica.starcheck.get_monitor_windows(start=None, stop=None, min_obsid=40000, config=None)¶

Use the database of starcheck products to get a list of monitor windows This list is filtered by timelines content to only include catalogs that should have or will run.

Parameters:

start – Optional start time for filtering windows as fetched from the database
stop – Optional stop time for filtering windows as fetched from the database
min_obsid – Minimum obsid value for filtering. Default of 40000 is intended to fetch only ERs
config – config dictionary. If supplied must include ‘starcheck_db’ and ‘timelines_db’ keys with dictionaries of the required arguments to Ska.DBI to connect to those databases.

Returns:

astropy Table of monitor windows. See get_starcheck_catalog_at_date for description of the values of the ‘status’ column. The ‘catalog’ column contains the get_starcheck_catalog_at_date returned dictionary.

`mica.stats.acq_stats`¶

Functions¶

mica.stats.acq_stats.get_stats(filter=True)[source]¶

Retrieve numpy array of acq stats

Returns acq_stats:
Parameters:	filter – True filters out ‘known_bad’ rows from the table
	numpy.ndarray

`mica.stats.guide_stats`¶

Functions¶

mica.stats.guide_stats.get_stats(filter=True)[source]¶

Retrieve numpy array of acq stats

Returns gui_stats:
Parameters:	filter – True filters out ‘known_bad’ rows from the table
	numpy.ndarray

`mica.archive.aca_dark`¶

`mica.archive.aca_dark.dark_cal`¶

`mica.archive.aca_dark.dark_model`¶

`mica.archive.aca_dark.update_aca_dark`¶

`mica.archive.aca_hdr3`¶

`mica.archive.aca_l0`¶

Functions¶

`mica.archive.asp_l1`¶

Functions¶

`mica.archive.obsid_archive`¶

`mica.archive.obspar`¶

Functions¶

`mica.vv`¶

Functions¶

`mica.utils`¶

Functions¶

`mica.starcheck`¶

Functions¶

`mica.stats.acq_stats`¶

Functions¶

`mica.stats.guide_stats`¶

Functions¶

Table Of Contents

Previous topic

This Page

Navigation

mica.archive.aca_dark¶

Functions¶

Functions¶

Functions¶

Functions¶

Functions¶

Functions¶

Functions¶

Functions¶

Previous topic

This Page

Quick search

Navigation

`mica.archive.aca_dark`¶