Sherpa: Modeling and Fitting in Python¶
Sherpa is a modeling and fitting application for Python. It contains a powerful language for combining simple models into complex expressions that can be fit to the data using a variety of statistics and optimization methods. It is easily extensible to include user models, statistics and optimization methods.
What can you do with Sherpa?¶
- Model generic 1D/2D (N-D) data arrays.
- Fit 1D (multiple) data including: spectra, surface brightness profiles, light curves, arrays.
- Fit 2D images/surfaces in Poisson/Gaussian regime.
- Build complex model expressions.
- Import, define and use your own models.
- Simulate predicted data based on defined models.
- Use appropriate statistics for modeling Poisson or Gaussian data
- Use Classic Maximum Likelihood or Bayesian Framework.
- Import, define the new statistics, with priors if required by analysis.
- Visualize a parameter space with simulations or using 1D/2D cuts of the parameter space
- Calculate confidence levels on the best fit model parameters
- Use a robust optimization method for the fit: Levenberg-Marquardt, Nelder-Mead Simplex or Monte Carlo/Differential Evolution.
- Sherpa supports wcs, responses, psf, convolution.
- Use Sherpa as part of astropy.modeling with Sherpa Bridge to Astropy - SABA
What’s new in Sherpa 4.9.0¶
Sherpa 4.9.0 was released on January 30, 2017. This release is for both Python 2.7 and Python 3.5. This version fixes many bugs in the Python 3 support. Moreover, it includes a significant refactoring of the Fit and Stat classes that made it possible to fix several bugs related to the recent wstat implementation while making these classes more maintainable and extensible.
The complete 4.9.0 Release Notes can be found here.
The code is developed for and has been tested on Linux and Macintosh (OS X 10.8 and later) machines. Sherpa works on both Python 2.7 and 3.5. There has been minimal testing with Python 3.6, and support for versions 3.3 and 3.4 would require community support.
A simple example of using the high-level Sherpa interfaces is provided in the Sherpa Quick Start Jupyter Notebook. Additional notebooks showcasing a range of Sherpa functionality are available from the Jupyter Notebooks section of the Sherpa wiki.
Sherpa can be installed from a binary distribution or built from sources. The binary distribution is suited for people wanting to have Sherpa up and running as soon as possible in its standard form. The binaries are built and tested on Linux and Mac OSX (>=10.8)
Source installation is available for platforms incompatible with the binary builds, or for users wanting to customize the way Sherpa is built and installed.
The code and installation instructions are available on GitHub.
If you are using Conda then it is as easy as:
conda config --add channels https://conda.anaconda.org/sherpa conda install sherpa
which is provided for Python 2.7, 3.5, and 3.6 versions (the Python 3.6 version has seen minimal testing).
There is no binary version of Sherpa which includes support for the XSPEC model library. Please follow the Source Build instructions, taking note of the XSPEC sub-section in the “Custom source build~ section, if XSPEC support is needed.
Although not required, Sherpa will use the configuration file $HOME/sherpa-standalone.rc to configure its behavior when it is run. If this file is not present Sherpa will use the default internal configuration file with the IO and Plotting back-ends set to pyfits (Astropy) and pylab (Matplotlib).
matplotlib comes with a configuration file matplotlibrc. For smooth behavior with Sherpa, be sure to indicate interactive=True in ~/.matplotlib/matplotlibrc.
You can import Sherpa into your ipython session:
(conda)$ ipython --pylab Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul 2 2016, 17:43:17) Type "copyright", "credits" or "license" for more information. IPython 5.1.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. Using matplotlib backend: MacOSX In : from sherpa.astro.ui import * WARNING: imaging routines will not be available, failed to import sherpa.image.ds9_backend due to 'RuntimeErr: DS9Win unusable: Could not find ds9 on your PATH'
The standard warnings are issued if you do not have ds9 models in your path. The image with ds9 will not be available. See the Dependencies section below.
Now to simulate a simple shape (a parabola with errors):
In : x = np.arange(-5, 5.1) In : y = x*x + 23.2 + np.random.normal(size=x.size) In : e = np.ones(x.size)
The data can now be loaded into Sherpa:
In : load_arrays(1, x, y, e) In : plot_data()
For this example we know what model to use, so pick a polynomial and free-up some of the parameters:
In : set_source(polynom1d.poly) In : print(poly) polynom1d.poly Param Type Value Min Max Units ----- ---- ----- --- --- ----- poly.c0 thawed 1 -3.40282e+38 3.40282e+38 poly.c1 frozen 0 -3.40282e+38 3.40282e+38 poly.c2 frozen 0 -3.40282e+38 3.40282e+38 poly.c3 frozen 0 -3.40282e+38 3.40282e+38 poly.c4 frozen 0 -3.40282e+38 3.40282e+38 poly.c5 frozen 0 -3.40282e+38 3.40282e+38 poly.c6 frozen 0 -3.40282e+38 3.40282e+38 poly.c7 frozen 0 -3.40282e+38 3.40282e+38 poly.c8 frozen 0 -3.40282e+38 3.40282e+38 poly.offset frozen 0 -3.40282e+38 3.40282e+38 In : thaw(poly.c1, poly.c2)
With everything set up, the data can be fit using the standard optimization method levmar and chi2 statistics:
In : fit() Dataset = 1 Method = levmar Statistic = chi2 Initial fit statistic = 12190 Final fit statistic = 5.40663 at function evaluation 8 Data points = 11 Degrees of freedom = 8 Probability [Q-value] = 0.713361 Reduced statistic = 0.675829 Change in statistic = 12184.6 poly.c0 22.2341 poly.c1 0.109262 poly.c2 1.06812 In : plot_fit_resid()
and an estimate of 1 sigma parameters uncertainties is:
In : conf() poly.c0 lower bound: -0.455477 poly.c1 lower bound: -0.0953463 poly.c0 upper bound: 0.455477 poly.c2 lower bound: -0.0341394 poly.c1 upper bound: 0.0953463 poly.c2 upper bound: 0.0341394 Dataset = 1 Confidence Method = confidence Iterative Fit Method = None Fitting Method = levmar Statistic = chi2gehrels confidence 1-sigma (68.2689%) bounds: Param Best-Fit Lower Bound Upper Bound ----- -------- ----------- ----------- poly.c0 22.2341 -0.455477 0.455477 poly.c1 0.109262 -0.0953463 0.0953463 poly.c2 1.06812 -0.0341394 0.0341394
Data I/O support and plotting need astropy and matplotlib. Imaging requires both ds9 and XPA. The XSPEC model library provides a number of models of use when analysing Astronomical spectra (primarily in the X-ray regime).
- Matplotlib: Hunter, JD (2007). Matplotlib: A 2D graphics environment.
Computing in Science and Engineering. 9: 90-95. http://matplotlib.org