Table Of Contents

This Page

_images/sherpa_logo.gif

Sherpa: Modeling and Fitting in Python

Sherpa is a modeling and fitting application for Python. It contains a powerful language for combining simple models into complex expressions that can be fit to the data using a variety of statistics and optimization methods. It is easily extensible to include user models, statistics and optimization methods.

What can you do with Sherpa?

  • fit 1D (multiple) data including: spectra, surface brightness profiles, light curves, general ASCII arrays
  • fit 2D images/surfaces in Poisson/Gaussian regime
  • build complex model expressions
  • import and use your own models
  • use appropriate statistics for modeling Poisson or Gaussian data
  • import the new statistics, with priors if required by analysis
  • visualize a parameter space with simulations or using 1D/2D cuts of the parameter space
  • calculate confidence levels on the best fit model parameters
  • choose a robust optimization method for the fit: Levenberg-Marquardt, Nelder-Mead Simplex or Monte Carlo/Differential Evolution.

For detailed documentation see: http://cxc.harvard.edu/sherpa

Download and Install

The binary installation of Sherpa 4.7b1 was released on September 26, 2014. It has been tested on Linux 32, Linux 64 and Mac OSX (10.8 and 10.9).

1. Binary installer

The binary installer takes care of unpacking a full binary installation and it contains Sherpa’s dependencies (including Python). It is suitable for users who do not typically have Python environment setup or who want Sherpa to be installed in a separate Python environment. See Section 2. Anaconda Python for Anaconda installation.

We provide a binary self-extracting installer for the supported platforms:

To run the installer, just type:

$ bash sherpa_<version>_<platform>_installer.sh

The installer will ask where to install Sherpa and its dependencies.

It will also provide you with information on how to add Sherpa to your PATH:

Once you enable the Sherpa environment you can check that the installation works with the command:

$ sherpa_test

The standard warnings will be printed if the configuration file is set for CIAO or there is no matplotlib or pyFITS installed:

Failed importing sherpa.astro.io: No module named pycrates
WARNING: failed to import sherpa.plot.chips_backend; plotting routines will not be available
WARNING: failed to import sherpa.astro.io; FITS I/O routines will not be available
WARNING: failed to import sherpa.astro.xspec; XSPEC models will not be available


WARNING: Couldn't load ChIPS. continue plotting w/ BLT.
WARNING: Couldn't load ChIPS. continue plotting w/ BLT.
WARNING: Couldn't load ChIPS. continue plotting w/ BLT.
WARNING: Couldn't load ChIPS. continue plotting w/ BLT.
WARNING: Couldn't load ChIPS. continue plotting w/ BLT.
WARNING: Couldn't load ChIPS. continue plotting w/ BLT.

If there are any issues please contact sherpadev at head.cfa.harvard.edu.

2. Anaconda Python

Sherpa binaries can be seamlessly installed into Anaconda Python <http://continuum.io/downloads>. You need to add the Chandra X-Ray Center’s channel to your configuration, and then install Sherpa:

$ conda config --add channels https://conda.binstar.org/cxc
$ conda install sherpa

To test that your installation works type:

$ sherpa_test

To update Sherpa:

$ conda update sherpa

Configuration Files

Sherpa comes with a configuration file sherpa.rc which is located in the $PYTHON/lib/site-packages/sherpa/. This file will be used if there is no ~/.sherpa.rc present. You need to copy the file to the home directory as ~/.sherpa.rc and update the verbosity to avoid the issues with standard python tracebacks (See: Known Issues). Be sure to indicate the IO and Plotting back-ends as pyfits and pylab depending on configuration.

matplotlib comes with a configuration file matplotlibrc. For smooth behavior with Sherpa, be sure to indicate interactive=True in ~/.matplotlib/matplotlibrc.

Run Sherpa

You can import Sherpa into your ipython session:

(conda)unix: ipython --pylab
Python 2.7.8 |Continuum Analytics, Inc.| (default, Aug 21 2014, 18:22:21)
Type "copyright", "credits" or "license" for more information.

IPython 2.2.0 -- An enhanced Interactive Python.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
Using matplotlib backend: Qt4Agg

In [1]: from sherpa.astro.ui import *
WARNING: imaging routines will not be available,
failed to import sherpa.image.ds9_backend due to
'RuntimeErr: DS9Win unusable: Could not find ds9 on your PATH'
WARNING: failed to import sherpa.astro.xspec; XSPEC models will not be available

The standard warnings are issued if you do not have ds9 or XSPEC models in your path. The image with ds9 and use of the XSPEC models will not be available. See the Dependencies section below.

Now to simulate a simple shape (a parabola with errors):

In [2]: x = np.arange(-5, 5.1)

In [3]: y = x*x + 23.2 + np.random.normal(size=x.size)

In [4]: e = np.ones(x.size)

The data can now be loaded into Sherpa:

In [5]: load_arrays(1, x, y, e)

In [6]: plot_data()
_images/data.png

For this example we know what model to use, so pick a polynomial and free-up some of the parameters:

In [7]: set_source(polynom1d.poly)

In [8]: print(poly)
polynom1d.poly
   Param        Type          Value          Min          Max      Units
   -----        ----          -----          ---          ---      -----
   poly.c0      thawed            1 -3.40282e+38  3.40282e+38
   poly.c1      frozen            0 -3.40282e+38  3.40282e+38
   poly.c2      frozen            0 -3.40282e+38  3.40282e+38
   poly.c3      frozen            0 -3.40282e+38  3.40282e+38
   poly.c4      frozen            0 -3.40282e+38  3.40282e+38
   poly.c5      frozen            0 -3.40282e+38  3.40282e+38
   poly.c6      frozen            0 -3.40282e+38  3.40282e+38
   poly.c7      frozen            0 -3.40282e+38  3.40282e+38
   poly.c8      frozen            0 -3.40282e+38  3.40282e+38
   poly.offset  frozen            0 -3.40282e+38  3.40282e+38

In [9]: thaw(poly.c1, poly.c2)

With everything set up, the data can be fit:

In [10]: fit()
Dataset               = 1
Method                = levmar
Statistic             = chi2
Initial fit statistic = 12190
Final fit statistic   = 5.40663 at function evaluation 8
Data points           = 11
Degrees of freedom    = 8
Probability [Q-value] = 0.713361
Reduced statistic     = 0.675829
Change in statistic   = 12184.6
   poly.c0        22.2341
   poly.c1        0.109262
   poly.c2        1.06812

In [11]: plot_fit_resid()
_images/fit.png

and an estimate of the errors is:

In [12]: conf()
poly.c0 lower bound:        -0.455477
poly.c1 lower bound:        -0.0953463
poly.c0 upper bound:        0.455477
poly.c2 lower bound:        -0.0341394
poly.c1 upper bound:        0.0953463
poly.c2 upper bound:        0.0341394
Dataset               = 1
Confidence Method     = confidence
Iterative Fit Method  = None
Fitting Method        = levmar
Statistic             = chi2gehrels
confidence 1-sigma (68.2689%) bounds:
   Param            Best-Fit  Lower Bound  Upper Bound
   -----            --------  -----------  -----------
   poly.c0           22.2341    -0.455477     0.455477
   poly.c1          0.109262   -0.0953463    0.0953463
   poly.c2           1.06812   -0.0341394    0.0341394

Dependencies

Data I/O support and plotting can be supplemented using PyFITS and “matplotlib”. Imaging requires ds9/XPA.

[pyfits]http://www.stsci.edu/resources/software_hardware/pyfits
[mpl]Hunter, JD (2007). Matplotlib: A 2D graphics environment. Computing in Science and Engineering. 9: 90-95. http://matplotlib.sourceforge.net.
[ds9]http://hea-www.harvard.edu/RD/ds9/
[xpa]http://hea-www.harvard.edu/saord/xpa/

Known Issues

  • The normal Python tracebacks are broken when sherpa.ui is imported. The screen output shows:

    In [1]: 1/0
    ---------------------------------------------------------------------------
    ZeroDivisionError                         Traceback (most recent call last)
    <ipython-input-1-05c9758a9c21> in <module>()
    ----> 1 1/0
    
    ZeroDivisionError: integer division or modulo by zero
    
    In [2]: from sherpa import ui
    WARNING: imaging routines will not be available,
    failed to import sherpa.image.ds9_backend due to
    'RuntimeErr: DS9Win unusable: Could not find ds9 on your PATH'
    
    In [3]: 1/0
    ERROR: Internal Python error in the inspect module.
    Below is the traceback from this internal error.
    
    Traceback (most recent call last):
    AssertionError
    
    Unfortunately, your original traceback can not be constructed.

The traceback can be recovered by modifying the verbosity in .sherpa.rc file:

[verbosity]
# Sherpa Chatter level
# a non-zero value will
# display full error traceback
level      : 2

You can also overwrite .sherpa.rc settings with:

import sys
sys.tracebacklimit = 100