Sherpa Overview

Observed data

This is the data recorded by the instrument and can be stored in either ASCII or binary (FITS) form. Error values can also be read in for each element, calculated from the data (if one of the chi-square statistice is being used), or set to equal a mathematical expression. Systematic errors can also be included if necessary.

More than one file can be loaded into Sherpa at a time and these datasets can then be fit simultaneously or independently. For instance if you have LETGS data then loading the PHA-II file will load the different orders into separate datasets. Similarly, if you have XMM imaging data you can fit the same spatial model to the data from the MOS1, MOS2, and PN detectors.

When analysing X-ray spectra it is best to use PHA-I or PHA-II files since Sherpa can recognise them and set up things accordingly. For instance, Sherpa will recognise the grouping information added by dmgroup and will then used the grouped data for fitting.

Background data

While Sherpa does not require any background data, you can load in a data file using the same formats described for the source data file. Once loaded the background data can be subtracted from the source dataset or used to fit a background model. This latter step is necessary when using the statistics based on the Cash and Bayesian maximum-likelihood functions.

If using PHA-I or PHA-II data then the background will be automatically set to use the contents of the filename stored in the BACKFILE keyword in the header of your source file. Unlike X-Spec, the background will not be automatically subtracted from the data and so you will need to use the SUBTRACT command if you want this.

The instrument model

The instrument model describes the mapping between the physical quantities of the source model and those of the detector. Setting up the instrument model is often simple - in fact its automatic for PHA-I files if the ANCRFILE and RESPFILE keywords are correctly defined - although it is also possible to create combinations of instrument models if required.

When fitting spectral data the instrument model contains both the effective area (read in as the ARF) and - if not grating data - the spectral blurring of the detector (read in as the RMF). For radial profiles the PSF of the instrument can be used to convolve the model before fitting to the data. With imaging data the effective area can be included by using an exposure map and the model can also be convolved by the instrumental PSF. It is also possible to set up a separate instrument model for the background dataset.

It is also possible to fit a dataset without setting up an instrument model. This may be because one is not needed - such as fitting a lightcurve - or because the instrumental effects would not significantly change the results - e.g. when fitting a radial profile to a source that covers most of an ACIS chip, the 0.5 arcsecond resolution of the instrument can normally be ignored.

Filtering the observed data

It is often the case that you do not want to fit all the data you have loaded: you may want to exclude areas where the particle background dominates your X-ray spectrum or exclude point sources when fitting an image of an extended source. Sherpa allows you to define what ranges of your data you want included or excluded from the fit. These regions can be changed at any time, which allows you to see how sensitive the fit results are to the chosen data range.

If an instrument model is supplied which contains a mapping between instrument and physical quantities - such as that between energy and the PI channel of an X-ray spectrum - then filters can also be defined in physical units.

The source model

Sherpa contains a large number of components that can be combined to define the source model that is to be fitted to the data. These range from one-dimensional and two-dimensional functions such as power law, polynomial, and gaussian to X-ray specific models (many of the X-Spec models are available). It is also possible to write your own models using either S-Lang or a compiled language like C and FORTRAN. If the background is being fit, rather than subtracted from the data or ignored, a separate source expression can be created to define the background model.

Each model component has a set of parameters that define its form, such as the FWHM, position, and ampliude of a one-dimensional gaussian. These values can be fixed (frozen) so they are not changed during fitting or freed (thawed) so they are. You can also change the value of a parameter and see how that changes the resulting source model.

When fitting multiple datasets simultaneously there are several ways that constraints can be applied. The same component (or components) can be used in the source expressions for the different datasets so that the same model is fit to all the datasets. Or, for a more fine-grained approach, individual parameters can be linked so that they are the same; for instance you can fit two power laws, with independent normalisations, but force their slopes to be the same.

For the more advanced cases it it also possible to create nested and joint-mode models.

Modelling detector noise

Both the source and background models are passed through the full instrument model when calculating the predicted data. At times it may be useful to be able to include a model that is not affected by the effective area (be it ARF or exposure map) of the instrument. An example would be when trying to include a model of the particle background in your fits to Chandra data. The noise model is used in such situations.

Predicted data

The predicted data is used when fitting the model but may also be of interest on its own, such as when simulating a source spectrum for a proposal or to estimate parameter errors. It is formed by combining the source and background models, after multiplication with the effective area (be it an ARF or an exposure map), together with the noise model. The resulting sum is then blurred by the instrument response (the RMF for spectral data and a PSF for imaging data) to create the predicted data.

Fit statistic

There are a number of statistics to use to compare the observed and predicted datasets: there is a set of choices based on the chi-square statistic with different ways for the errors to be calculated; maximum-likelihood functions based on the Cash statistic; a Bayesian maximum-likelihood function; and the ability to write your own statistic.

Fit optimisation

The optimisation method controls how the fitted parameters are changed so as to improve the fit after each iteration. Sherpa provides a range of optimization methods - each with its own advantages and disadvantages - because there is no guarantee that, for a given situation, there will not be multiple local minima in the fit statistic (i.e. search space). There are two broad classes of optimiser available within Sherpa: "single-shot" and "scatter-shot" methods.

The single-shot methods start at a single position in the search space (the initial parameter values) and then vary them until a local minimum is found. Within this class there are trade-offs to be made between speed and robustness (i.e. how likely you are to find the local minimum).

The scatter-shot methods are used to allow the search to be made from a range of initial parameter values spread throughout the search space. They may therefore be more likely to find the global, rather than local, minimum of fit statistic. Some of these methods include the single-shot methods - so the fit is optimised starting at each grid point - which can provide the best of both worlds but can take a very long time to run.

It is possible to change the optimisation method and re-fit, to see how reliable the fit results are.

Estimating errors on best-fit parameters

Once the best-fit location has been found, Sherpa provides a number of routines for calculating the errors on the parameters. It is also possible to calculate a confidence map showing how two parameters are correlated.

The Sherpa command language

Sherpa commands are entered on the command line:

sherpa> READ DATA data1.dat 1 2
sherpa> READ ERRORS data1.dat 1 3
sherpa> LPLOT DATA
sherpa> SOURCE = POLY[model1]
sherpa> THAW model1.c1

Note that within this document, as well as in the Sherpa Reference Manual, non-variable command elements are capitalized. However, Sherpa commands are not case sensitive and therefore may be issued in either upper or lower case.

Using S-Lang within Sherpa

The Sherpa command language is intended to allow a user to quickly load in a dataset, set up the appropriate source model, and fit it. It was not designed for users who want to script their fitting sessions, or want the ability to manipulate (read and write) the data and results in Sherpa.

As with ChIPS, S-Lang commands can be entered at the Sherpa prompt; unlike the Sherpa command language case is important for S-Lang commands. In the following example we call the S-Lang function run_fit(), which is part of the Sherpa S-Lang module:

sherpa> res = run_fit()

Note that we have been able to ignore both the trailing semi-colon and the need to declare res as a variable as Sherpa does this for us. When used from a script - executed either via the evalfile() command or via the slsh tool - you would have had to say something like:

variable res = run_fit();

Using scripts

Both sets of commands can also be used from scripts: the use command for Sherpa scripts and the evalfile() function to run S-Lang scripts. This is described in detail in the Sherpa and Scripts thread.

Saving the current state

The SAVE command will create an ASCII file listing all commands necessary to re-create the current state of Sherpa. If you are in the middle of an analysis then

sherpa> SAVE ALL state.shp

creates the text file state.shp.

Restoring the state

The text file (here state.shp) can then be used from a new Sherpa to restore the analysis by saying:

sherpa> ERASE ALL
sherpa> USE state.shp

The call to ERASE ALL may not be necessary but ensures that any previous settings are cleared before the old session is restored. The data analysis can be resumed after the use of USE.

An alternative technique is to start Sherpa and give it the file name:

unix% sherpa state.shp

Recovering a Sherpa session

If for any reason, Sherpa exits with a nonzero status, a log file of the session is saved in the user's home directory to assist in recovering the Sherpa session. This log file contains contains all commands that were successfully executed up to the point that Sherpa failed. The name of the saved file is $HOME/.sherpa-session-PID, where PID is the process ID of the Sherpa session that failed.

To recover the session sherpa needs to be launched with the pointer to the saved file.For example, if the PID of a failed Sherpa session was 2048, then the command:

unix% sherpa ~/.sherpa-session-2048

will start Sherpa and execute all the commands from that failed session, up to but not including the command that caused a failure.

When Sherpa exits normally, with an exit status of zero, no such log file is left behind.