What parameter space did the fit search?

[New] New in CIAO 4.18 is the record_steps argument for the fit command. Setting it to `True` will cause Sherpa to store the values of the thawed parameters for each iteration. This can then be recovered from the record_steps argumenof the FitResults object returned by get_fit_results.

sherpa> fit(record_steps=True)
Dataset               = 1
Method                = levmar
Statistic             = chi2gehrels
Initial fit statistic = 2.95653e+08
Final fit statistic   = 25.6852 at function evaluation 269
Data points           = 41
Degrees of freedom    = 37
Probability [Q-value] = 0.919273
Reduced statistic     = 0.694194
Change in statistic   = 2.95653e+08
   gal.nH         0            +/- 0.0414875
   src.kT         3.86495      +/- 0.685231
   src.Abundanc   0.247734     +/- 0.22332
   src.norm       0.000630125  +/- 8.76926e-05
WARNING: parameter value gal.nH is at its minimum boundary 0.0
sherpa> fr = get_fit_results()
sherpa> steps = fr.record_steps

The steps argument is a 2D array, where each row is the iteration and the columns are a combination of the numnber of model evaluations, the statistic value, and the thawed parameter values.

sherpa> print(steps[0])
(0, 295652615.26603246, 1.0, 1.0, 1.0, 1.0)
sherpa> print(steps[-1])
(270, 25.68516437091545, 0.0, 3.864952109508104, 0.24773416325255948, 0.0006301253463394781)

These columns can be accessed by name:

sherpa> print(steps.dtype)
[('nfev', '<i8'), ('statistic', '<f8'), ('gal.nH', '<f8'), ('src.kT', '<f8'), ('src.Abundanc', '<f8'), ('src.norm', '<f8')]

which lets us plot information about the fit, such as how the stasistic varies

sherpa> plot_trace(steps["statistic"], name="statistic", ylog=True)

Figure 1: How does the statistic vary?

[The statistic starts up very large and then drops to a sensible value (approximately 7 orders of magnitude drop) after about 60 iterations, it then stays there until there are two jumps where it gets very-large again before falling back (within about 30 iterations each time).]

[Print media version: The statistic starts up very large and then drops to a sensible value (approximately 7 orders of magnitude drop) after about 60 iterations, it then stays there until there are two jumps where it gets very-large again before falling back (within about 30 iterations each time).]

Figure 1: How does the statistic vary?

or how the parameters change:

sherpa> plot_scatter(steps["src.kT"], steps["src.Abundanc"], xlabel="kT", ylabel="Abundanc", name="fitting")

Figure 2: How do kT and Abundanc vary?

[The temperature varies between 1 to 5 keV abd the abudance covers 0 to 1.4, but there is a pattern (the abundance is mostly close to 0.3 but there is a curve with kT and there are a few places where the values jump about).]

[Print media version: The temperature varies between 1 to 5 keV abd the abudance covers 0 to 1.4, but there is a pattern (the abundance is mostly close to 0.3 but there is a curve with kT and there are a few places where the values jump about).]

Figure 2: How do kT and Abundanc vary?

Prior to CIAO 4.18 the outfile argument to fit would have had to be used. This writes the information to an ASCII file (or a StringIO buffer) which could then be read back in to get at the statistic and thawed-parameter values.

sherpa> from io import StringIO
sherpa> get_source().reset()  # reset the parameter values
sherpa> buffer = StringIO()
sherpa> fit(outfile=buffer)
sherpa> out = buffer.getvalue()
sherpa> print(out)
# nfev statistic gal.nH src.kT src.Abundanc src.norm
0.000000e+00 2.956526e+08 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00
1.000000e+00 2.956526e+08 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00
2.000000e+00 2.953944e+08 1.000345e+00 1.000000e+00 1.000000e+00 1.000000e+00
...
2.680000e+02 2.571938e+01 1.520523e-04 3.852037e+00 2.273156e-01 6.392440e-04
2.690000e+02 2.594376e+01 2.697721e-03 3.935248e+00 2.592980e-01 6.310050e-04
2.700000e+02 2.568516e+01 0.000000e+00 3.864952e+00 2.477342e-01 6.301253e-04