12-13 June 2003
Chandra Users' Committee
Members Attending: M. Arnaud, Y.H. Chu, J.P. Hughes (chair), W. Latter,
J. Lee, K. Long, J. Krolik, and S. Snowden
By Telecon: A. Cool
Members absent: J. P. Henry, K. Mitsuda, G. Taylor, C. Reynolds
Others: A. Tennant, H. Tananbaum, B. Wilkes, F.Seward, P. Green,
D. Kniffen (by telecon)
(1) The committee was concerned that the fraction of time to be
awarded the large and very large projects (LP, VLP) was fixed with
respect to the normal GO programs without regard to the relative
science between these classes of programs. The plans as presented to
the committee only allow for trading observing time between LP and VLP
projects. At the same time, the amount of time reserved to the LP and
VLP has doubled this round, while the amount of time going to normal
GO programs remains at the same level as previously. The committee
suggests that additional flexibility be incorporated into the review
process. The Big project panel should be instructed that it can
allocate less than 6 Ms of time to LP+VLPs. Observing time that is not
allocated by the Big project panel should then revert to the individual
panels roughly in proportion to their original allocations. The
panels should be clearly instructed that gray area normal proposals in
their panel may be promoted and therefore the proposals below the
cut-off should be ranked carefully.
(2) The Big project panel should be queried concerning whether the
process that separates LPs and VLPs from normal programs is likely to
result (in the opinion of the Big project panel) in the highest
science efficiency for Chandra. This might be a part of a somewhat
larger ensemble of questions regarding the peer review. In particular,
after the review, Big project panel members should be asked to respond
in writing to the following (or a similar set of) questions:
The CUC would like to receive a copy of the responses, even in just a
"summary" format, as one way of evaluating whether the VLP proposals
- In terms of science return per unit observing time, was the time
allocated between normal, LP and VLP proposals appropriate?
- In your opinion, was the quality of the lower ranking, but
approved LP or VLP proposals higher or, at least as high, as the
highest ranking, but unapproved normal proposals in your panel?
Please explain why or why not you feel the way you do.
- In your opinion, was there a significant difference (other than
observing time) in the nature of the LP and VLP programs? For example,
did they address topics that would not have been addressed properly
otherwise or was the nature of the approved programs different in some
way, such as being more of a "Legacy nature" than large proposals.
- What practical recommendations do you have regarding the peer
review process for consideration in planning for future cycles?
(3) The committee heard about the potential gains and risks associated
with a bake-out of ACIS intended to release the contamination that has
reduced the low energy quantum efficiency. Based on the risks (e.g.,
slightly increased CTI, change in calibration) and gains (increased
low energy QE), as presented, the committee strongly supports
(4) The calibration plan as presented for the pre and post bakeout of ACIS
needs to be worked more. In particular it was not clear how the spatial
distribution of the contamination was going to be characterized. For example,
perhaps the ACIS should be docked for some period of time both before and after
to accumulate enough photons to characterize the spatial variation using
the calibration sources. This is potentially a very important moment in the
life of Chandra, since there will be a step function change in the performance
of ACIS. All aspects of calibration should be looked at to ensure that
sufficient calibration data are collected in order that all pre-bakeout
observations can be analyzed accurately to maximize the science return of
(5) Calibration Document
The committee appreciates the progress that has been made in terms of
documenting the calibration plan. In particular the table describing
the state of calibration will be quite useful. We note that the draft
calibration document appears to be based heavily on MSFC-RQMT-2229,
which was written long before launch and does not reflect current
on-going in-flight calibration plans or changes in instrument
performance during the mission. The calibration document needs to
address these aspects of calibration as well.
Here are some specific comments/suggestions on the draft document:
(6) Software issues: Four themes for beyond CIAO 3
- Section 2.0. Since this came from MSFC-RQMT-2229 it has an ossified
feel to it. Some of the items are useful (e.g., Source Identification),
some are hopelessly naive (e.g., Sunyaev-Zel'dovich Effect), while some
don't make much sense at all (e.g., Kinetic Energy of Supernova Remnants).
It is not clear how much value this section has for the document. Perhaps
MSFC-RQMT-2229 should be put on-line and then one could just refer to it
here. What really matters are the numerical requirements which are already
in the summary table and should remain there.
- Section 3.0. This needs to be expanded in a number of ways:
(a) to include all relevant calibration quantities, e.g., relative
efficiency as a function of energy, PSF wings, off-axis PSF, spectral
response of ACIS, low energy contamination, etc. It would be extremely
useful if all calibration observations, items in the caldb, and any
other calibration issue relevant to science analysis could be traceable
back to this summary table.
(b) to include requirements/present/projection values for more
cases. For example, values should be given for individual ACIS chips
and even individual ACIS nodes if the differences are relevant.
Do we know the energy resolution of ACIS to 12% everywhere regardless
of chip or node? By the way, FWHM is *not* all we need to know about
Energy Resolution for ACIS - this should be broken down into the various
components of the spectral response of ACIS as a function of energy.
- Section 4.0. This section also needs to be extended. In-flight
calibration is, in fact, only one portion of what should be an
integrated calibration plan. As we learned from Dick Edgar at the June
2003 meeting, improved analyses of XRCF data can result in improved
calibration of the in-flight detectors. Then there are calibration
quantities that are time dependent (the ACIS gain and CTE for example)
and are monitored on a regular basis. This section should call out
the various different types of calibrations and for each quantity
identify what data were used to generate the calibration result (e.g.,
XRCF, in flight observation of a star cluster, etc.) and indicate
the frequency of any monitoring observations.
- Section 5.0 needs to be fleshed out in considerably more detail.
Clear priorities need to be set and a realistic schedule
established. Items listed here need to be traced back clearly to
sections 3 and 4. We note that much of what Dick Edgar spoke about
should be included here.
CIAO has been rather successful overall to date and certainly stands
up well compared to other astronomical software packages. The CUC
agrees that it is time, at mid-mission, to assess where CIAO is headed.
The most important concern of the CUC for "beyond CIAO 3" software
development is ensuring adequate support for core software for Chandra
users. New software and updated threads to support new calibration
requirements, to account for degradation in instrument performance,
and to continue to be able to perform basic fundamental analyses must
be the highest priority. This should be the most important theme for
"Beyond CIAO 3".
As the CIAO software survey showed, users are also interested in
modest improvements to CIAO like the ability to write scripts,
improved visualization/graphics tools, enhanced speed, better
syntax/interface, more documentation, and so on. To the committee, a
significant critique is that CIAO is not very integrated both in terms
of overall concept and in terms of look and feel. Improving each and
every CIAO task/thread/script would be a large effort, but would also
yield a great benefit for users. The CUC considers this to be an
important theme for "Beyond CIAO 3".
The practical benefits of the proposed "Beyond CIAO 3" themes were not
generally apparent to the committee (n.b., the presentations were
rather heavily focussed on technical details). The themes do not
appear to reflect an integrated plan oriented toward Chandra users.
The CUC recommends that the software group take a serious look at what
themes would be of greatest benefit to Chandra Users and propose a
realistic vision for "Beyond CIAO 3" especially in the light of
anticipated reduced funding during the extended phase of Chandra.
We also note specific issues related to the proposed themes.
The level of effort going into CIAO and other efforts is quite large,
perhaps as large as any NASA effort. It is also clear that the CXC has
plans that extend beyond Chandra. There needs to be a decision of the
overall vision for CIAO and analysis efforts. Furthermore the
committee urges that any long term efforts be coordinated with other
large astrophysics data analysis efforts.
- VO-era CXC: The CUC is concerned that this could be a drain on CXC
software resources and may not have a direct benefit for the average
Chandra user. It is also not clear whether Chandra funds should be
used to support software developments for VO. Perhaps funding for
this should come from a coordinated VO funding program? In any event
this looks like a longer timescale issue to the CUC and should be
accorded a lower priority than other themes until an overall vision
is endorsed by the committee.
- GUIs: Use of GUIs to enable users to bind together the various
tools contained in CIAO and to create new tools, e.g., extracting
spectra directly from images, would be desirable. Thoughtful use
of GUIs could greatly improve the CIAO user interface.
- CIAO High Spatial/Spectral Resolution Imaging: Many of the proposed
activities are incremental (e.g., ChaRT upgrades, aspect improvement,
ACIS sub-pixel resolution, etc.) and therefore consistent with the
"improve CIAO incrementally" theme in the second paragraph
above. There is no doubt that the multi-scale deconvolution technique
described here is great work, but it appears to be more of a
scientific research project, than software development. It is the kind
of thing the CUC would like to see submitted to the Chandra
Contributed Software page. However, what the committee was looking
for here was an indication that the software group had taken a broad
look at a large number of high level analysis tools for Chandra and
had identified one or more as possible themes. This should be done.
Other specific recommendations:
(7) Monitoring and Trends Analysis
- The CXC should develop and report an overall vision of the future
development of CIAO, incorporating our inputs from above, which it
reports to the committee.
- The CXC should coordinate its developments with other major
centers and should report back to the committee on its effort and
justify areas in which it is "going it alone" rather than
coordinate. Two major factors in defining advisability of a
particular package should be the stability of long term support
and the size of the user/developer community.
- The committee remains concerned about the choice of the S-Lang
scripting language for CIAO. We note the following specific concerns:
the cost in effort to those in the community who need to learn yet
another system; the questionable long-term support for S-Lang; and the
small size of its user community. The significance of the last
concern is that, unlike other, more widely-used scripting languages,
only a small number of people contribute user-developed routines.
It is the time today to have a major review over the advisability
of increasing the dependence of CIAO on S-Lang versus adoption of a
scripting language that has a wider audience.
The CUC recommends that the CXC provide a higher level of oversight to
the SI performance monitoring and trending analysis program to ensure
that appropriate performance quantities are being tracked.