.. Sherpa documentation master file, created by sphinx-quickstart on Thu Sep 25 15:44:42 2014. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. .. image:: sherpa_logo.gif Sherpa: Modeling and Fitting in Python ========================================== .. toctree:: :maxdepth: 2 Sherpa is a modeling and fitting application for Python. It contains a powerful language for combining simple models into complex expressions that can be fit to the data using a variety of statistics and optimization methods. It is easily extensible to include user models, statistics and optimization methods. What can you do with Sherpa? ---------------------------- * fit 1D (multiple) data including: spectra, surface brightness profiles, light curves, general ASCII arrays * fit 2D images/surfaces in Poisson/Gaussian regime * build complex model expressions * import and use your own models * use appropriate statistics for modeling Poisson or Gaussian data * import the new statistics, with priors if required by analysis * visualize a parameter space with simulations or using 1D/2D cuts of the parameter space * calculate confidence levels on the best fit model parameters * choose a robust optimization method for the fit: Levenberg-Marquardt, Nelder-Mead Simplex or Monte Carlo/Differential Evolution. To install Sherpa see Sections Source_ and Binary_ Installation. For an example of a Sherpa session check :ref:`run-sherpa` section. More examples are given in a few Sherpa IPython Notebooks: `Sherpa Quick Start `_ `Image Fitting `_ `Template Fitting `_ `A Simple Bayes Analysis of an X-ray Spectrum `_ For detailed documentation see: http://cxc.harvard.edu/sherpa Install Sherpa ============== Sherpa can be installed from a binary distribution or built from sources. The binary distribution is suited for people wanting to have Sherpa up and running as soon as possible in its standard form. The binaries are built and tested on Linux 32, Linux 64 and Mac OSX (10.8 and 10.9) Source installation is available for platforms incompatible with the binary builds, or for users wanting to customize the way Sherpa is built and installed. .. _Source: Source Installation =================== It takes only a few simple steps to build and install Sherpa in any Python 2.7 environment on Linux or Mac OSX. For example Anaconda Python Distribution contains many scientific software components needed for the analysis and Sherpa fits seamlessly into that environment. The Sherpa source code is available on GitHub: https://github.com/sherpa/sherpa Download the Sherpa source tar or zip file: `sherpa-4.7.zip `_ or `sherpa-4.7.tar.gz `_ Unpack the files:: $ unzip sherpa-4.7.zip or $ tar xvf sherpa-4.7.tar.gz To install:: $ cd sherpa $ python setup.py install The Sherpa code will be built and installed in the directory `${prefix}/lib/python2.7/site-packages/sherpa`, where `${prefix}` can be determined with:: $ python -c 'import sys; print(sys.prefix)' Sherpa requires the standard Python packages and system compilers for the build:: Python: setuptools, numpy System: gcc, g++, gfortran, make, flex, bison In addition I/O requires `pyfits` for reading FITS files and `matplotlib` for standard plotting, `ds9` for imaging. After the installation you can run the test to check if the installation was successful:: $ sherpa_test Note that the `${prefix}/bin` should be in the $PATH to run the test. The tests should succeed, but there could be two expected failures if 'ds9' or/and 'XSPEC' models are not found. These are not necessary for the use of Sherpa and only needed if you plan to perform X-ray analysis or use 'ds9' to display images. Copy the source code repository ------------------------------- If you want to look at the code, use the code and maybe develop your own code in Git you can 'clone' the full repository structure with the default 'git clone': $ git clone https://github.com/sherpa/sherpa.git This would also be useful if you want to contribute your own code to Sherpa via 'pull request'. The contributions are welcome. We advise to follow the `astropy guidlines `_ for developing your contributions. We require the code to have a description of the code, unit and integration test and a user documentation. Custom Source Build ==================== The costumization of the source build could be made by updating updating the `setup.cfg` file in 'sherpa' directory. For example to use the `fftw` local library one needs to set the following configuration options:: fftw=local fftw-include-dirs=/usr/local/include fftw-lib-dirs=/usr/local/lib fftw-libraries=fftw3 where `/usr/local` path may need to be changed to the local directories with the header (.h) files and the `libfftw3.so` shared object file. It can also allow you to build XSPEC models. XSPEC Models ------------ Sherpa does not support XSPEC models for X-ray spectral analysis by default. However, it is possible to instruct Sherpa to build its XSPEC extension module by changing the build configuration options. You may need to build XSPEC yourself, and in any case to point Sherpa to existing binary libraries for XSPEC, cfitsio, and CCfits. Additionaly, you will need to point Sherpa to the libgfortran shared library. These dependencies are required to build XSPEC itself, so they are assumed to be present on the system where you want to build Sherpa with XSPEC support. Here we assume that XSPEC has been already build.:: $ git clone https://github.com/sherpa/sherpa.git $ cd sherpa Edit the `setup.cfg` file. Find the XSPEC configuration section in the file, uncomment the relative options and make sure they point to the location of the XSPEC, cfitsio, CCfits, and gfortran libraries. For instance:: with-xspec=True xspec_lib_dirs=/opt/xspec/lib cfitsio_lib_dirs=/usr/local/lib ccfits_lib_dirs=/usr/local/lib gfortran_lib_dirs=/usr/local/lib You may need to change the values in the above example to reflect the actual directories where the libraries are to be found on your system. Then, build Sherpa in the standard way:: $ python setup.py install .. _Binary: Binary Install ============== The binary installation of Sherpa 4.7 was released on April 17, 2015. It has been tested on Linux 32, Linux 64 and Mac OSX (10.8 and 10.9). Sherpa binaries can be seamlessly installed into `Anaconda Python `_. You need to add Sherpa channel to your configuration, and then install Sherpa:: $ conda config --add channels https://conda.binstar.org/sherpa $ conda install sherpa To test that your installation works type:: $ sherpa_test To update Sherpa:: $ conda update sherpa If you do not have Anaconda Python and want to create the minimum environment for trying Sherpa check the `README on GitHub `_. Configuration Files =================== Sherpa expects a user configuration file `sherpa-standalone.rc` to be in the $HOME directory. If this file is not present Sherpa will use the default internal configuration file with the IO and Plotting back-ends set to `pyfits` and `pylab`. matplotlib comes with a configuration file `matplotlibrc`. For smooth behavior with Sherpa, be sure to indicate `interactive=True` in `~/.matplotlib/matplotlibrc`. .. _run-sherpa: Run Sherpa ========== You can import Sherpa into your ipython session:: (conda)$ ipython --pylab Python 2.7.9 |Anaconda 1.9.2 (x86_64)| (default, Dec 15 2014, 10:37:34) Type "copyright", "credits" or "license" for more information. IPython 2.3.1 -- An enhanced Interactive Python. Anaconda is brought to you by Continuum Analytics. Please check out: http://continuum.io/thanks and https://binstar.org ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. Using matplotlib backend: MacOSX In [1]: from sherpa.astro.ui import * WARNING: imaging routines will not be available, failed to import sherpa.image.ds9_backend due to 'RuntimeErr: DS9Win unusable: Could not find ds9 on your PATH' The standard warnings are issued if you do not have ds9 models in your path. The image with ds9 will not be available. See the Dependencies section below. Now to simulate a simple shape (a parabola with errors):: In [2]: x = np.arange(-5, 5.1) In [3]: y = x*x + 23.2 + np.random.normal(size=x.size) In [4]: e = np.ones(x.size) The data can now be loaded into Sherpa:: In [5]: load_arrays(1, x, y, e) In [6]: plot_data() .. image:: data.png For this example we know what model to use, so pick a polynomial and free-up some of the parameters:: In [7]: set_source(polynom1d.poly) In [8]: print(poly) polynom1d.poly Param Type Value Min Max Units ----- ---- ----- --- --- ----- poly.c0 thawed 1 -3.40282e+38 3.40282e+38 poly.c1 frozen 0 -3.40282e+38 3.40282e+38 poly.c2 frozen 0 -3.40282e+38 3.40282e+38 poly.c3 frozen 0 -3.40282e+38 3.40282e+38 poly.c4 frozen 0 -3.40282e+38 3.40282e+38 poly.c5 frozen 0 -3.40282e+38 3.40282e+38 poly.c6 frozen 0 -3.40282e+38 3.40282e+38 poly.c7 frozen 0 -3.40282e+38 3.40282e+38 poly.c8 frozen 0 -3.40282e+38 3.40282e+38 poly.offset frozen 0 -3.40282e+38 3.40282e+38 In [9]: thaw(poly.c1, poly.c2) With everything set up, the data can be fit using the standard optimization method `levmar` and chi2 statistics:: In [10]: fit() Dataset = 1 Method = levmar Statistic = chi2 Initial fit statistic = 12190 Final fit statistic = 5.40663 at function evaluation 8 Data points = 11 Degrees of freedom = 8 Probability [Q-value] = 0.713361 Reduced statistic = 0.675829 Change in statistic = 12184.6 poly.c0 22.2341 poly.c1 0.109262 poly.c2 1.06812 In [11]: plot_fit_resid() .. image:: fit.png and an estimate of 1 sigma parameters uncertainties is:: In [12]: conf() poly.c0 lower bound: -0.455477 poly.c1 lower bound: -0.0953463 poly.c0 upper bound: 0.455477 poly.c2 lower bound: -0.0341394 poly.c1 upper bound: 0.0953463 poly.c2 upper bound: 0.0341394 Dataset = 1 Confidence Method = confidence Iterative Fit Method = None Fitting Method = levmar Statistic = chi2gehrels confidence 1-sigma (68.2689%) bounds: Param Best-Fit Lower Bound Upper Bound ----- -------- ----------- ----------- poly.c0 22.2341 -0.455477 0.455477 poly.c1 0.109262 -0.0953463 0.0953463 poly.c2 1.06812 -0.0341394 0.0341394 Dependencies ============ Data I/O support and plotting can be supplemented using PyFITS and "matplotlib". Imaging requires ds9/XPA. .. [pyfits] http://www.stsci.edu/resources/software_hardware/pyfits .. [mpl] Hunter, JD (2007). *Matplotlib: A 2D graphics environment.* Computing in Science and Engineering. 9: 90-95. http://matplotlib.sourceforge.net. .. [ds9] http://hea-www.harvard.edu/RD/ds9/ .. [xpa] http://hea-www.harvard.edu/saord/xpa/ Release Notes Sherpa 4.7 for Python =================================== April 17, 2015 The Sherpa 4.7 code released on April 17, 2015 is the same as the CIAO 4.7 version. There are, however, small differences required for the standalone build. Differences between Standalone 4.7 and CIAO 4.7 ------------------------------------------------ - Standalone Sherpa Python and CIAO Sherpa now look for different configuration files in the $HOME directory. For the standalone this is `$HOME/.sherpa-standalone.rc`. If this file is not present, Sherpa falls back to the internal configuration file. The default IO and plotting backends are set to pyfits and pylab in the Standalone Sherpa, and he stack trace is not silenced. Users can update the settings and override the configuration file location by exporting the SHERPARC environment variable, as supported by the previous versions of Sherpa and CIAO. - All source files now have copyright and licensing information. A summary is included upfront in the repository main directory. - Orphan code that was not actually used has been removed. - Fixed code triggering deprecation warnings from Numpy 1.9. - Fixed code triggering compiler warnings. - Added documentation (README, ipython notebook). - The version string (`sherpa.__version__`) depends on the git `commit/tag`, unlike in CIAO where it is fixed to 40701. - Some classes from the template module were not exposed by `__all__`, and template models were not imported in the sherpa session. - The standalone repository contains some code originated in CIAO and also includes other OTS: the code coming from CIAO has been updated to 4.7. There is a caveat: - the datastack module is not imported if no plotting packages are available. `matplotlib` has to be installed to use that module.