Skip to the navigation links
Last modified: 12 December 2019

URL: https://cxc.cfa.harvard.edu/sherpa/index.html

CIAO's modeling and fitting package


Sherpa is the CIAO modeling and fitting application. It enables the user to construct complex models from simple definitions and fit those models to data, using a variety of statistics and optimization methods (see the Gallery of Examples).

Latest Version

Sherpa version for CIAO 4.12 was released on December 17, 2019. Sherpa in CIAO runs under Python 3.5 (when installed with ciao-install) or Python 3.7, 3.6, or 3.5 (when installed using the conda package manager). The full list of the Sherpa updates is given in the Release Notes. The major updates in this release include:

  • Plotting is now handled by Matplotlib as ChIPS has been deprecated in CIAO 4.12. Please see the ChiPS to Matplotlib Conversion Guide for help converting from ChIPS, or contact the CXC Helpdesk if you need help.

    The sherpa application will automatically convert to Matplotlib if you still have ChIPS as your preference setting, and you will see the following message displayed:

    WARNING: chips is not supported in CIAO 4.12+, falling back to matplotlib.
    WARNING: Please consider updating your $HOME/.sherpa.rc file to suppress this warning.
    

    As discussed in the Sherpa FAQ on the ChIPS warning, this warning can be stopped by changing the plot_pkg line in your $HOME/.sherpa.rc file so that it is set to pylab - for example:

    unix% grep plot_pkg $HOME/.sherpa.rc
    plot_pkg   : pylab
    

    The use of the IPython magic command "%matplotlib" is no-longer needed in the Sherpa environment to see the plots on screen. Users who import Sherpa into IPython sessions or Jupyter notebooks will still need to set up the Matplotlib event loop to see the plots.

  • The plot_xxx family of routines - such as plot_data and plot_fit - can now accept plot settings as keyword arguments. The allowable keywords are the same as those returned by the get_xxx_plot_prefs routines, such as xlog, ylog, and color. This means that you can now say:

    sherpa> plot_fit(ylog=True)
    sherpa> plot_model(2, overplot=True, color='gray', linestyle='dotted')
    

    Two new routines have been added to display the current fit along with the ratio of the model to data: plot_fit_ratio() and plot_bkg_fit_ratio(). Please use help(plot_fit_ratio) from Sherpa for more information.

    The number of warnings about displaying error bars in plots when error bars have been turned off has been decreased (in many cases you will no longer see this, but they can still occur for the combined fit-and-residual plot functions).

    The plot_pvalue function has seen improvements when used with PHA spectra.

  • Sherpa now supports the use of a PSF image with a finer resolution than the resolution of 2D data images. 1D models can also be evaluated on a finer grid than the grid of the input data arrays. The models are evaluated on the resolution given by the PSF pixel scale or the 1D arbitrary grid. The evaluated models are rebin to match the original data scales for calculating the statistic during the fit.

  • Asymmetric errors are now supported for calculating parameter uncertainties via bootstrap. The data with asymmetric errors can be entered using a new 'load_ascii_with_errors' function or 'Data1DAsymmetricErrs' class.

  • The paging performed by the show_xxx set of commands - such as show_data and show_all - is now handled by Python rather than using an external command. This means that the output will now appear in Jupyter notebooks, and that the PAGER environment variable is no longer checked.

  • Update to version 12.10.1n of the XSPEC model library, which adds support for the following additive models: xsagnsed, xscph, xskyrline, xsqsosed, xsvcph, xszbknpower, and xszlogpar.

    The update to version 12.10.1 of XSPEC has meant that the default cross-section table has changed: it now defaults to vern when it was bcmc in CIAO 4.11 and earlier. Note that this default setting is now read from your $HOME/.xspec/Xspec.init configuration file (if it exists), using the XSECT setting.

    The default abundance table will now be set from your $HOME/.xspec/Xspec.init file, if the ABUND line is set.

The CIAO release of Sherpa is based on the GitHub-developed version of Sherpa. Please consider contributing to Sherpa development, whether by adding code, fixing bugs, or changing documentation. The updated documentation for this version of Sherpa contains information about new functions and might be useful, but is not aimed at CIAO users.

Sherpa lets you:
  • fit 1-D data sets (simultaneously or individually), including:
    spectra, surface brightness profiles, light curves, general ASCII arrays;

  • fit 2-D images/surfaces in the Poisson/Gaussian regime;

  • visualize the data with Matplotlib and DS9;

  • access the internal data arrays;

  • build complex model expressions;

  • import and use your own models;

  • choose appropriate statistics for modeling Poisson or Gaussian data;

  • import new statistics, with priors if required by analysis;

  • visualize a parameter space with simulations or using 1-D/2-D cuts of
    the parameter space;

  • calculate confidence levels on the best-fit model parameters;

  • choose a robust optimization method for the fit: Levenberg-Marquardt,
    Nelder-Mead Simplex or Monte Carlo/Differential Evolution;

  • perform Bayesian analysis with Poisson Likelihood and priors, using
    Metropolis or Metropolis-Hastings algorithm in the MCMC (Markov-Chain Monte Carlo);

  • and use Python to create complex analysis and modeling functions,
    build the batch mode analysis or extend the provided functionality
    to meet the required needs.

[] []
[] []
[] []

The Sherpa infrastructure greatly enhances the default Sherpa functions, and provides users with an environment for developing complex and sophisticated analysis.

Sherpa is designed for use in a variety of modes: as a user-interactive application and in batch mode. Sherpa is an importable module for the Python the scripting language and is available as a C/C++ library for software developers. In addition, users may write their own Python scripts for use in Sherpa.

The About Sherpa page outlines key features of the software, and the Latest Updates page describes new functionality and recent changes. See also the SciPy2009 conference proceedings for detailed information about Sherpa's capabilities.

If you have ideas about how to enhance or improve Sherpa, please contribute ideas (and code) to the Sherpa GitHub repository.

Please send feedback and questions on Sherpa to the CXC Helpdesk or the Sherpa Issues list on GitHub.


Citing Sherpa in a Publication

If you are writing a paper and would like to cite Sherpa, we recommend the following papers and presentations:

Sherpa: a mission-independent data analysis application (ADS)
P. E. Freeman, S. Doe, A. Siemiginowska
SPIE Proceedings, Vol. 4477, p.76, 2001

\bibitem[Freeman et al.(2001)]{2001SPIE.4477...76F} Freeman, P., Doe, S., 
\& Siemiginowska, A.\ 2001, \procspie, 4477, 76

The specific version of CIAO and CALDB (if applicable) used for the analysis should be mentioned as well.

A reference for the Python interface to Sherpa is also available:

Developing Sherpa with Python (ADS)
S. Doe, et al.
Astronomical Data Analysis Software and Systems XVI, 376, 543

\bibitem[Doe et al.(2007)]{2007ASPC..376..543D} Doe, S., et al.\ 2007, 
Astronomical Data Analysis Software and Systems XVI, 376, 543

Sherpa: 1D/2D modeling and fitting in Python (SciPy 2009)
B. Refsdal, S. Doe, D. Nguyen, A. Siemiginowska, N. Bonaventura, D. Burke, I. Evans, J. Evans, A. Fruscione, E. Galle, J. Houck, M. Karovska, N. Lee, M. Nowak
Proceedings of the 8th Python in Science Conference (SciPy 2009), G. Varoquaux, S. van der Walt, J. Millman (Eds.), pp. 51-57 2009

Fitting and Estimating Parameter Confidence Limits with Sherpa (SciPy 2011),
B. Refsdal, S. Doe, D. Nguyen, A. Siemiginowska, V. Kashyap
Proceedings of the 19th Python in Science Conference (SciPy 2011), S. van der Walt, J. Millman (Eds.), pp. 4-10 2011

Further guidelines are available from the Acknowledgment of Use of Chandra Resources.