Last modified: December 2013

Jump to: Description · Examples · Bugs · See Also

AHELP for CIAO 4.11 Sherpa v1


Context: confidence


Estimate confidence intervals for selected thawed parameters based on projection method


proj([id], [otherids], [parameters])


CIAO 4.2 has a new confidence function which supersedes projection; see "ahelp conf". The projection function has been deprecated and may be removed in the future; users should replace it with the confidence function in their analysis.

The projection command computes confidence interval bounds for the specified model parameters in the dataset. A given parameter's value is varied along a grid of values while the values of all the other thawed parameters are allowed to float to new best-fit values. The function may be called as "projection" or simply "proj".

The projection command differs from covariance ("ahelp covariance") in that all other thawed parameters are allowed to float to new best-fit values, instead of being fixed to the initial best-fit values. While projection is more general (e.g. allowing the user to examine the parameter space away from the best-fit point), it is in the strictest sense no more accurate than covariance for determining confidence intervals.

The computationally intensive projection function is parallelized to make use of multi-core systems (i.e., laptops or desktops with 2 or 4 cores) to provide significant improvements in efficiency compared to previous releases of Sherpa; the 'numcores' option of set_proj_opt may be used to specify how the cores should be used when projection is run.

When running on multiple ids and parameters, the arguments may be given in any order (see the examples); any argument that is not defined as a model parameter is assumed to be a data id.

To control the number of sigma used for the limits (i.e. the change in statistic), set the value of get_proj.sigma ("ahelp get_proj"). The default setting is to calculate +/- 1 sigma limits.

Because projection estimates confidence intervals for each parameter independently, the relationship between sigma and the change in statistic value delta_S can be particularly simple: sigma = the square root of delta_S for statistics sampled from the chi-square distribution and for the Cash statistic, and is approximately equal to the square root of (2 * delta_S) for fits based on the general log-likelihood.

When the search for confidence limits for a parameter has been completed, it is reported to the screen, e.g.

Confidence limits search for g2.fwhm finished (1 of 3)
Confidence limits search for g2.pos finished (2 of 3)
Confidence limits search for g2.ampl finished (3 of 3) 	

Accuracy of confidence intervals

An estimated confidence interval is accurate if and only if:

One may determine if these conditions hold, for example, by plotting the fit statistic as a function of each parameter's values (the curve should approximate a parabola) and by examining contour plots of the fit statistics made by varying the values of two parameters at a time (the contours should be elliptical, and parameter space boundaries should be no closer than approximately 3 sigma from the best-fit point). The int_proj ("ahelp int_proj"). and reg_proj ("ahelp reg_proj"). can be used to create the contours.

If either of the conditions given above does not hold, then the output from projection may be meaningless except to give an idea of the scale of the confidence intervals. To accurately determine the confidence intervals, one would have to reparameterize the model, or use Monte Carlo simulations or Bayesian methods.


Example 1

sherpa> proj()

Run projection on the default dataset(s), calculating limits for all thawed parameters of all data sets for which the user has defined a model to be fit. The output for two data sets looks like:

projection 1-sigma bounds:
   Param            Best-Fit  Lower Bound  Upper Bound
   -----            --------  -----------  -----------
   p1.gamma          2.15852   -0.0827856    0.0827856
   p1.ampl        0.00022484 -1.48256e-05  1.48256e-05
   g.gamma           2.15147   -0.0800326    0.0800326
   g.ampl         0.00176499 -2.14396e-05  2.14396e-05

Example 2

sherpa> set_proj_opt('numcores', 2)
sherpa> proj(2, g.gamma)

Use the set_proj_opt command to specify that only two of the available cores on the system should be utilized when the proj command is run. Compute parameter limits for the model parameter g.gamma, which is assigned to dataset 2. The output is:

projection 1-sigma bounds:
   Param            Best-Fit  Lower Bound  Upper Bound
   -----            --------  -----------  -----------
   g.gamma          2.15147   -0.0800326    0.0800326

Example 3

sherpa> proj(1, p1.ampl, 2, g.fwhm, g.ampl)
sherpa> proj(1, 2, p1.ampl, g.fwhm, g.ampl)

These commands are two equivalent ways of running projection on one parameter from dataset 1 (p1.ampl) and two parameters from dataset 2 (g.fwhm and g.ampl). Note that the order in which the ids and parameters names are supplied does not matter. In both cases, the output is:

projection 1-sigma bounds:
   Param            Best-Fit  Lower Bound  Upper Bound
   -----            --------  -----------  -----------
   p1.ampl       0.000224842 -1.48255e-05  1.48255e-05
   g.fwhm            2.76124   -0.0335314    0.0335314
   g.ampl         0.00176499 -2.14396e-05  2.14396e-05


See the bugs pages on the Sherpa website for an up-to-date listing of known bugs.

See Also

conf, covariance, get_conf, get_covar, get_int_proj, get_int_unc, get_proj, get_reg_proj, get_reg_unc, int_proj, int_unc, reg_proj, reg_unc, set_conf_opt, set_covar_opt, set_proj_opt