What do the WARNING messages returned by the confidence methods mean?
WARNING: hard minimum hit for parameter <parameter name>
When the confidence, projection, and covariance methods are used to estimate confidence intervals for thawed model parameters after a fit, sometimes a hard upper or lower limit will be reached for one or more parameters. This produces the message "WARNING: hard minimum hit for parameter <parameter name>", along with a row of dashes in the appropriate place in the projection or covariance output. (The covariance method can also return a null value for an upper/lower limit when the parameter space at the minimum is non-quadratic for a given parameter. The covariance matrix calculations assume that the parameters follow the Normal distribution. If the parameter space is non-smooth, then the covariance calculations fail and Sherpa returns "-----".)
Example confidence output:
sherpa> conf()
...
WARNING: hard minimum hit for parameter bpow1.gamma2
WARNING: hard maximum hit for parameter bpow1.gamma2
WARNING: hard minimum hit for parameter bpow1.eb
WARNING: hard maximum hit for parameter bpow1.eb
...
Dataset = 1
Confidence Method = confidence
Iterative Fit Method = None
Fitting Method = neldermead
Statistic = cstat
confidence 1-sigma (68.2689%) bounds:
Param Best-Fit Lower Bound Upper Bound
----- -------- ----------- -----------
bpow1.gamma1 1.54147 -0.0292891 0.0292709
bpow1.gamma2 8.10056 ----- -----
bpow1.eb 9.49083 ----- -----
bpow1.ampl 0.022806 -0.000378395 0.000383854
This occurs when the parameter bound found by one of the confidence methods lies outside the hard limit boundary for a model parameter - this could result from an issue with the signal-to-noise of the data, the applicability of the model to the data, systematic errors in the data, among others things.
A parameter hard limit represents either a hard physical limit (e.g., temperature is not allowed to go below zero), a mathematical limit (e.g., prevent a number from going to zero or below, when the log of that number will be taken), or the limit of what a float or double can hold (the fit should not be driven above or below the maximum or minimum values a variable can hold). For this reason, model parameter hard limits should not be changed by the user.
WARNING: The confidence level lies within <interval>
Another warning message which may be returned by confidence is that a model parameter lies within the stated range:
sherpa> conf(g15.Sigma)
...
g15.Sigma -: WARNING: The confidence level lies within (8.706380e-05,
9.252185e-05)
...
Datasets = 1, 2
Confidence Method = confidence
Iterative Fit Method = None
Fitting Method = levmar
Statistic = chi2datavar
confidence 1.64-sigma (89.8995%) bounds:
Param Best-Fit Lower Bound Upper Bound
----- -------- ----------- -----------
g15.Sigma 0.000997626 -0.000907834 0.000597058
This occurs where confidence cannot locate the root (minimum value of the fit statistic function) even though the root is bracketed within an interval (perhaps due to poor resolution of the data or a discontinuity). In such cases, when the openinterval option of confidence is set to False (default), the confidence function will not be able to find the root within the set tolerance and the function will return the average of the open interval which brackets the root. If the openinterval option is set to True, then confidence will print the minimal open interval which brackets the root (not to be confused with the lower and upper bound of the confidence interval). The most accurate thing to do is to return an open interval where the root is localized/bracketed rather than the average of the open interval (since the average of the interval is not a root within the specified tolerance).
The output from confidence may be checked by setting 'set_conf_opt('verbose',1)', and then re-running confidence for the relevant parameter(s).