Cycle 24 Peer Review

Rodolfo Montez Jr. and the Chandra Director's Office

The Cycle 24 peer review was held remotely from June 21-30 2022 with 87 reviewers, chairs, and pundits coming together in 11 topical panels (see Table 1 for the topical panel breakdown) and one Big Project Panel. The topical panels met for the first week (June 21-24) and the Big Project Panel the following week (June 27-30). The 11 topical panels were recruited by a team of Panel Organizers and supported at the review by 11 Panel Facilitators, the Panel Organizers, and the Chandra Director’s Office. This was the third time the CXC held its review remotely and the second time as a Dual Anonymous Peer Review.

Dual Anonymous Peer Review

As in Cycle 23, the Cycle 24 Peer Review was a Dual Anonymous Peer Review (DAPR). In a DAPR, the proposers (as usual) do not know the identity of the reviewers, but, in addition, the reviewers do not know the identity of the proposers. Proposers are asked to “anonymize” their text to avoid identifying the proposing team.

In Cycle 24 we required that proposers use numbered references (described below) to further improve the anonymity of a proposing team. As expected for a new requirement, several proposals were flagged as being in violation of this policy. In Cycle 24 such violations were not penalized unless the proposing team was directly identified. In Cycle 25, all violations will result in a stronger action.

As with other proposal requirements, the Chandra Director’s Office carefully reviews potentially non-compliant proposals on a case-by-case basis. A few proposals were found to be non-compliant with the DAPR requirements and removed from the Peer Review. We realize this is a serious action and the Chandra Director’s Office does not take such action lightly. Non-compliant proposals were further evaluated by the CXC Director and NASA Headquarters before a non-compliant proposal was removed.

Avoiding Non-Compliant Dual Anonymous Proposals in Cycle 25 and Beyond

To better help proposers in Cycle 25, we have compiled a non-exhaustive list of examples of non-compliant text in Scientific Justifications that will result in removal from the Peer Review:
Displaying the PI Name: including below the proposal title or within the science justification.
Revealing the Identities of Any Members of the Proposing Team: such as, “Co-I Montez has studied these apples for years.”
Using Non-Anonymized Language: such as, “We showed in [2]…”, “Our work [2,3,9]…”, etc. Note that, even though the authors are using numbered references, using phrases like these are non-compliant with the DAPR anonymizing criteria. These phrases should be anonymized to remove the direct association of these references with the proposing team, such as, “As shown in [2]…” and “The work in [2,3,9]…”.

In addition to the examples above, in Cycle 25 the following flavor of non-compliant text in Scientific Justifications will likely result in removal from the Peer Review:

• Using Non-Numbered References:

Non-CompliantCompliant
“The grass is green [Montez et al. 2021].” “The grass is green [3].”

• Directly Naming Proposing Teams’ Institution (especially when referring to the resources available at a given institution or in budget justifications of Archive and Theory proposals; you may reveal such information in the Team Expertise document, but not in the Science Justification):

Non-CompliantCompliant
“Based on the SAO overhead rate of…”.“Based on the institutional overhead rate of…”.

Cycle 24 Proposal Statistics

Cycle 24 Peer Review considered 423 proposals asking for a trigger probability-weighted 69.4 Ms of time (an oversubscription of 4.3 by time). 16.0 Ms were awarded to 154 proposals. Cycle 24 proposal statistics are available on the CXC website and are graphically represented in Figures 1-4. The distribution of science panels is given in Table 1 and Joint Program awards can be found in Table 2.

Cycle 24 included a call for Very Large Proposals (VLP), a category requiring > 1 Ms of observing time. The 16.0 Ms of total time allocated in Cycle 24 included 2.9 Ms to 6 approved LPs and 1 Ms to an approved VLP.

The Cycle 25 Call for Proposals (CfP) will be released on 15 December 2022 and the proposal deadline is 15 March 2023.

Cost Proposals

PIs of proposals with US collaborators were invited to submit Cost Proposals to SAO (due Oct 2022). Each project was allocated a budget based on the details of the observing program (as described in Section 10 of the Call for Proposals). Awards were made at the allocated or requested budget levels, whichever was lower.

A line plot showing seven data trends. The X-Axis shows Chandra Proposal Cycle, from 1 to 24. TheY-Axis shows Number of proposals and has a break; the bottom of the axis is linear from 0 to just over 80, while the top of the axis starts at 300 and proceeds linearly to 900. The seven data sets and their colors are GO/TOO in blue, Theory in purple, Archive in red, LP in orange, XVP in brown, VLP in green, and Total in a thicker black line. GO/TOO and Total are the only data points in the top part of the axis; they start at around 800 in Cycle 1, slowly declining to 400 (Total) and 300 (GO/TOO) in Cycle 24. On the bottom, Archive quickly rises from Cycle 2, reaching a steady value of around 80 by Cycle 5. LP, starting in Cycle 2, and Theory, starting in Cycle 4, remain relatively consistent throughout, at roughly 60 and roughly 40 proposals per cycle, respectively. VLPs decline from their introduction in Cycle 5 to only a few in Cycle 12; in Cycle 13, there are no VLPs, only XVPs, which remain at roughly 20, with a gradual decline over the four cycles they are present. VLPs reappear in Cycle 19, staying constant at around 10 per year.


Figure 1: The number of proposals submitted in each proposal category (GO, LP, Archive, etc.) as well as overall as a function of cycle. The y-axis is broken to include the entire range.

A line plot showing seven data trends. The X-Axis shows Chandra Proposal Cycle, from 1 to 24. TheY-Axis shows Oversubscription Ratio and stretches from 0 to just over 16. The seven data sets and their colors are GO/TOO in blue, Theory in purple, Archive in red, LP in orange, XVP in brown, VLP in green, and Total in a thicker black line. All are relatively constant, with LP and VLP showing the most extreme variability. The most recent VLP oversubscription is over 16, a significant outlier from the other points, which are almost all between 3 and 8.


Figure 2: The effective oversubscription ratio in terms of observing time for each proposal category as a function of cycle. Total oversubscription ratio is shown as the black solid line. Note that some of the fluctuations are due to small number statistics (e.g., Theory proposals). In Cycle 20, none of the submitted VLP proposals were selected by the peer review.

A line plot showing two data trends. The X-Axis shows Chandra Proposal Cycle, from 1 to 24. TheY-Axis is labeled Time, in units of Megaseconds, and stretches from 0 to just over 140. There are two colored data sets; these are labeled “Requested” in blue and “Approved” in orange. Aside from a spike to 140 Ms in Cycles 13 and 14, the requested line stays roughly between 80 and 100 Ms over the entire span. The approved line is fairly consistent over the entire Figure, other than a brief rise in the first few cycles and a slight bump in Cycles 13 and 14, hovering around 20 Ms throughout.


Figure 3: The requested and approved time as a function of cycle in Ms including allowance for the probability of triggering each TOO. The available time increased over the first three cycles, and in Cycle 5 with the introduction of Very Large Projects (VLPs). The subsequent increase in time to be awarded due to the increasing observing efficiency and the corresponding increase in requested time in response to the calls for X-ray Visionary Projects (XVPs) in Cycles 13-16 is clear.

A line plot showing three data trends. The X-Axis shows Chandra Proposal Cycle, from 1 to 24. The left Y-Axis is labeled Proposal Success Rate, and stretches from 0.0 to 0.5, and the right Y-Axis is Fraction of All Submitted Proposals, stretching from 0.0 to 1.0. There are two colored data sets, having vertical error bars of size roughly 0.1 on the left axis; these are labeled “Success rate for Male PIs” in blue and “Success Rate for Female PIs” in orange. These two data sets stay at roughly 0.3 to 0.4 and are statistically indistinguishable. The third data set is plotted in gray and is labeled “Submitted by Female PIs.” It starts at just below 0.2 in Cycle 1 and steadily rises to over 0.3 in Cycle 24.


Figure 4: The success rate of male (blue) and female (orange) PIs as a function of cycle, and the overall fraction of female PIs (gray). Since Cycle 10, the success rate for female and male PIs is typically statistically indistinguishable.

Tables showing: (1) Panel Organization for Cycle 24, (2) Summary of Joint Programs

Table 1

Table 1: Panel Organization for Cycle 24
Topical Panels
PanelSubject
1, 2 Normal Stars, WD, Planetary Systems and Misc
3, 4SN, SNR + Isolated NS, GW Events
5, 6WD Binaries + CVs, BH and NS Binaries, Galaxies: Populations Extragalactic
7, 8Clusters of Galaxies, Galaxies: Diffuse Emission
9, 10, 11AGN, Extragalactic Surveys
Big Project Panel
PanelSubject
Big Project PanelLarge and Very Large Proposals

Table 2

Table 2: Joint Proposal Summary for Cycle 24
Joint PartnerProposalsTotal Resources Awarded
HST1385.0 orbits
NOAO66.0 nights
NRAO17317.5 hours
NuSTAR5470.0 ks
Swift8212.0 ks
XMM3282.0 ks