Introduction.
Monitoring and alerts.
ACIS Ops Response to SCS107.
When to call a telecon.
How to call a telecon.
More links

Introduction

The ACIS Ops team learned in the first weeks of theChandra mission that the camera's front illuminated (FI) chips are highly vulnerable to damage from low energy protons at roughly 100 keV. Such protons cause displacement damage in the Si lattice producing charge traps which significantly increase the CTI. A single belt passage increased the FWHM by 20 eV in FI chips at -120C. As detailed in MIT's report, the expected damage from belt passages varies markedly as Chandra's orbit changes. 1999-2001, and 2010-2015 are the worst periods.

Since then, the ACIS Ops team has continuously monitored space weather in an attempt to gauge Chandra's present and near future radiation environment. Radiation levels of concern can arise from two sources:
  1. Trapped radiation belts. The ACIS instrument must be protected through each perigee passage through the radiation belts:
    1. The SIM must be translated to the HRC-S position while the Chandra Radiation Model (CRM) predicts proton fluxes above threshold. The CRM determines the radiation belt entry and exit times, which often but not always align with the EEF1000 and XEF1000 orbital events, as determined by NSDDC's AE-8 electron model. Translations to and from HRC-S occur at pad times before entry and after exit. CTI measurements are taken during these pad times.
    2. The "backstop pad time" runs from Radmon disable to EEF 1000, and from XEF 1000 to Radmon enable. The "CRM pad time" runs from the time of soft proton entry of the belts according to the CRM model (lengthened up to a maximum of 10 ks if ACIS Ops consents) to the AE8 predicted time of electron radiation belt entry.
    3. At all points in the orbit, the radiation monitor must be enabled whenever ACIS is in the focal plane. During radiation belt crossings, the radiation monitor should be disabled once the SIM translation to HRC-S is complete.
    4. Video boards must be powered down between CRM pad times and perigee. There should be a minimum of about four hours between powerdown and perigee.

    This requirement is enforced by diligent review of perigee passages in weekly command loads.

  2. High solar background. In order to limit increase in the full width half max at the top of an FI chip to an annual 0.1%, the ACIS operations team has established a radiation exposure budget for for ACIS: an annual accumulated fluence of 2e^10 protons/cm2-s-ster-MeV.

    As a rule of thumb, expecting 8 to 10 major events a year, we allow ourselves to accumulate a fluence of about 2e+9 for any one radiation event.

Monitoring and Alerts

Warning of a high radiation environment may come from several sources, most of them handily gathered at cxc's Radiation Central. The following summary moves from data with the longest to the shortest predictive range.

Sunspot tracking

Our best long look-ahead comes from tracking sunspots on the near and far faces of the Sun. Spots take about 26 days to make an (apparent, from the Earth's viewpoint) rotation across the Sun; a couple days less at the equator and more near the poles.

Solar X-rays and CMEs

In the event of an X class solar flare, NOAA's Space Weather Prediction Center will send an email alert to acisdude with the subject line "SUMMARY: X-Ray Event exceeded X1". That's our cue to jump to Today's Space Weather, which displays the last three days of solar X-ray flux in a logarithmic graph. Each horizontal line represents a factor of 10 increase in energy. Flares below M-7 or so are unlikely to give ACIS any grief. If they're above that level, we need to start keeping an eye on ACE and GOES rates. The most energetic particles may arrive in 0.5 to 3 days, depending in part on where the flare was directed.

In the past, Shanil received email notices of Coronal Mass Ejections (CMEs) from stenborg@kreutz.nascom.nasa.gov, which he forwarded to the rest of us. (TBD: what is the current mechanism for learning as rapidly as possible whether there has been a CME and what its angle was with Earth's line of sight?) The velocity of CMEs is variable, but if one is earth directed, it typically takes two to five days for the ejected mass to arrive, and several hours for it to sweep past earth.

ACE

The warnings most indicative of current Chandra conditions are the alerts from MTA based on ACE fluence monitoring. The ACE instrument orbits at Earth-Sun L1, so that it experiences solar winds and storms about an hour in advance of the Earth. The ACE page provides tables and charts in realtime of electron and proton fluxes at a variety of energies. We are interested in the P3 proton channel (115 - 195 keV). So as not to rely on a single detector, the two derived P3 numbers, conservatively scaled from the P5 and P6 channels can also trigger messages. Alerts are sent when:
  1. Three sot_red_alerts, spaced half an hour apart, when fluence for the orbit exceeds 1.0e^9. Additional alerts are sent on each tripling of the fluence.
  2. The fluence integrated over two-hours exceeds 3.6e^8. This more urgent condition will send pager alerts in addition to the sot_red_alert email.

GOES

The GOES satellites are in geosynchronous orbits about 38K kilometers up, with GOES-15 over the eastern and GOES-14 over the western hemisphere. They indicate near-earth proton levels, bearing in mind that Chandra's inclination exposes us to a different cross section of the magnetosphere.

GOES-14 replaced GOES-10 on December 1, 2009. Unlike its predecessor, GOES-14 will have no proton detection capability.

MTA sends a yellow alert when GOES-15 P2 > 30 and P5 > 0.25; and red alerts for P2 > 90.9 and P5 > 0.70. The red limits are designed to indicate a probable trip of the old P4GM (P2) and P41GM (P5) EPHIN channels. (NB: GOES definitions of P2 and P5 channels differ from the ACE definitions of P3, etc.)

Kp index

MTA will send notification through sot_yellow_alert when Earth's near-term Kp index as projected by models is high.

Kp is a measure of the magnetic field at the surface of the Earth, averaged from data at various ground stations around the globe. Increases in the Kp indicates that the particle density in the radiation belts is changing.

EPHIN

In 2008, EPHIN's detector A, because of a high proportion of dead times during which the detector was saturated, the instrument's failure mode A was set to "ON". Since then, EPHIN no longer has a channel devoted exclusively to protons. (It was always unable to detect protons at the low energies which are most problematic.) EPHIN now reports just two channels: E150 and E1300, and the limits are scaled to account for the fact that proton counts are also included. 174,216.03 E150 counts/s/cm2/sr will trigger an SCS-107. The E1300 limit, still specific to electrons, stands at at 19.296079 counts/s/cm2/sr. (These limits account for the update to the geometric factor in December 2009.) Note that EPHIN was decommissioned on day 2018:250.

HRC

As EPHIN heats up and grows less reliable, we will probably begin to rely on the High Resolution Camera's anticoincidence shield as our on-board radiation detector. This has severe drawbacks: the HRC can detect only high energy protons, and it saturates before it reaches levels corresponding to current EPHIN red limits. The prospects were discussed in the 2005 Flight Note 443 (pdf), with a breakdown of results of adopting the highest possible threshold value in a 2006 memo from Mike Juda.

ACIS TXINGS

ACIS threshold crossings ('txings' for short) can also be used to trigger autonomous radiation shutdowns, as detailed in Peter Ford's memo Using ACIS to detect and report high radiation conditions .

SCS 107 notification

When an SCS-107 is commanded, or EPHIN executes one spontaneously and the fact becomes apparent on the next contact, MTA sends an email to sot_red_alert indicating the time at which the SCS107 DISA status was noted in realtime telemetry. (This will not ordinarily be the same as the time at which the SCS-107 occurred. The OC will usually promulgate that actual time before the contact finishes.) As a cross-check that the SCS-107 is real, our Realtime page will show DPA-A and DPA-B currents of about 0.43 and 0.28 respectively, and the SIM at HRC-S.

Events that had been scheduled in the daily load before the SCS-107 will continue to show up on the Replan Central page, but greyed out.

Response to SCS 107

Confirm that ACIS is safe via our realtime page: If any of these checks fail, be diligent and make sure that SOP_UNSAFE_ACIS PHASE 1 is executed.

An exception to this is that there's a class of anomalies in which the 1STAT1ST bit fails to set. These are documented under ACIS Science Run Termination Failure Anomaly. This is less urgent than the other failures listed above.

Check the Fluence monitors to determine how much ACIS has detected already. These are based on the ACE Flux.
Run history-files.pl -s107 with the SCS 107 time. (The memos webpage instructions for radiation replan reviews are less detailed but more up-to-date.)
Be ready for the replan. Review the CTI RTS and be ready to write one up if there is enough time.

When to Call a Radiation Telecon

In the event of an SCS-107, ACIS is safe, and the Engineering Manager or the Flight Director will call a telecon.

An ACE Alert has come through and P3 channel rates have been rising. The default is to call a telecon if there's been an ACE alert. In case the alert is at the 3.6e^8 threshold, do the following calculations:

  1. Determine exposureTimeToRadzone (i.e., to Radmon Disable). (If we're already in the radzone, this is the RadmonDisable for the next orbit.) Subtract from these times any period spent at HRC-S or HRC-I, and divide by 2 or 5 respectively when the LETG or HETG is in place.
  2. Determine exposureTimeToContact, and exposureTimeTo2ndContact (i.e., the next commanding opportunity, and the one following that.) Make the same adjustments for SIM position and gratings as in (1); namely, subtract from these times any period spent at HRC-S or HRC-I, and divide by 2 or 5 respectively when the LETG or HETG is in place.
  3. Is the ACE flux clearly falling? If so, set "CalcFlux" to the current reading. Use the real P3 channel.
    Otherwise, extrapolate the flux to the next UTimeToRadzone or UTimeToContact (whichever is first) and set "CalcFlux" to that value.
  4. Calculate
    RadZFlu = current orbital fluence + CalcFlux * timeToRadzone
    ContFlu = current orbital fluence + CalcFlux * timeToContact
    NxtContFlu = current orbital fluence + CalcFlux * timeTo2ndContact.

[We really should have a script to pop out all these numbers. I'll volunteer to write it, once we've agreed on policy and scaling parameters.]

If there's been an alert, the default is to call a telecon via sot_red_alert before the next contact. However, if the alert comes outside of reasonable ( 7pm to 7am) hours and
  1. ContFlu < 1.0e^9 and the first contact occurs after 7 am or
  2. NxtContFlu < 1.0e^9 and the following contact occurs after 7 am
then, via email to sot_yellow_alert, call for the telecon the next morning, between 7 am and the first contact opportunity.

Finally, if ACE flux is clearly falling, and both RadZFlu and NxtContFlu are below 1.0e^9, you may defer a decision on calling a telecon and continue to monitor.

How to Call a Radiation Telecon

Send an email to sot_red_alert. Radiation telecons will ordinarily be on the contingency number 877-467-6272 (111165#)

Required personnel to make a decision on whether to shut down are a flight director, and representatives from FOT engineering and FOT MP. Desirable are people from PCAD (either Eric or Tom), SOT MP, SOT lead, and Steve O'Dell from NASA.

SCS-107 EXCUTION AND NON-LOAD TRACKING

If an SCS-107 occured - either manually or automatically - you want to record that Non-Load Event. When execution of the SCS-107 begins, be sure to note the time.

Once the SCS-107 execution is complete, you need to add this event to the "Non-Load Tracking File". This is a file that is used by various programs to incorporate the effects of of the shutdown into the Thermal Models.

The instructions for running the Non-Load Event Tracker can be found here.

Linkfest

Instructions for writing CAP for CTI using an RTS
Joe's Summary on radiation response doc, pdf
Rad response Ex. 1 doc, pdf
Rad response Ex. 2 doc, pdf
Rad response Ex. 3 txt, pdf,

There are more examples of radiation response, and memos about the decision process to be found off the ACIS memos page, under "Radiation Shutdown Decisions," which leads you to this page.

Here is a link to the MTA archive of old radiation shutdown plots and information. It's also off the SOT Homepage under Radiation, and Radiation Event Archive. The archive of Radiation Events.

There's a tutorial on space weather on the ACIS twiki, here.

Document history:
Royce Buehler, last update Sep 2017
Richard Edgar, last update Apr 2019