---

Sherpa is the CIAO modeling and fitting application for Python made available by the Chandra X-ray Center (CXC). Development takes place at the Sherpa GitHub repository and contibutions - whether new code, bug fixes, or documentation input - are welcome. Information on recent releases and citation information for Sherpa is available using the Digital Object Identifier (DOI) 10.5281/zenodo.593753.

Sherpa can be used for analysis of images, spectra and time series from many telescopes, including optical telescopes such as Hubble. It can also be easily used with non-Astronomical data. Sherpa is flexible, modular and extensible. It has an IPython user interface and it is also an importable Python module. Sherpa models, optimization and statistic functions are available via both C++ and Python for software developers wishing to link such functions directly to their own compiled code.

Sherpa supports fitting of 1-D X-ray spectra from Chandra and other X-ray missions, as well as 1-D non-X-ray data, including generic data arrays, radial profiles, and lightcurves. The options for grating data analysis include fitting the spectrum with multiple response files required for overlapping orders in LETG observations. Modeling of 2-D spatial data is fully supported, including the PSF and exposure maps. User specified models can be added to Sherpa with advanced "user model" functionality.

A list of recent publications featuring the various uses of Sherpa may be found at the Sherpa ADS library.

The Sherpa website concentrates on using Sherpa within CIAO. There is also documentation for the "standalone" version of Sherpa which advanced users may find useful.

Features

Python environment

Sherpa is an importable module for the Python scripting language, providing users with the ability to write their own Python scripts for use in Sherpa. This easily allows for creating complex analysis and modeling functions, building batch-mode analysis, and extending the provided functionality to meet required needs.

Multiple startup options

There are six command-line options which may be specified when Sherpa is started:

-h

Provide help on the options.

-x

launch sherpa shell in separate display terminal.

-b

Run in batch mode.

-n

Do not print banner on start up.

-rcfile <filename>

Specify a specific .sherpa.rc file to use; refer to "ahelp sherparc" for details.

-norcfile

Do not load any .sherpa.rc file; overrides -rcfile if both are set.

The Sherpa application uses the $HOME/.ipython-ciao/ directory to store configuration information. The first time Sherpa s started you may see a message about old IPython profile information being found.

unix% sherpa

ATTENTION: The IPython profile found in $HOME/.ipython-ciao//profile_sherpa is not compatible with this version of sherpa.

    Update to a compatible version? [Y/N]:

Unless you have manually changed the file it is safe to answer "Y" here. The outdated file is renamed with a timestamp to preserve it.

Visualization

For plotting, Sherpa uses Matplotlib and bokeh which is added as an eperimental plotting backend in 4.17.0. Image visualization is provided by DS9.

Parallelization

The following computationally intensive functions are parallelized to make use of multi-core systems, providing users with such systems significant improvements in efficiency compared to previous releases of Sherpa. Each function has a "numcores" option to specify the number of cores to use in parallelization; the default is to use all cores available.

• conf
• projection
• int_proj
• int_unc
• get_int_proj
• get_int_unc
• reg_proj
• reg_unc
• get_reg_proj
• get_reg_unc
• plot_photon_flux
• plot_energy_flux
• get_photon_flux_hist
• get_energy_flux_hist
• gridsearch
• normal_sample
• t_sample
• uniform_sample
• plot_pvalue
• get_pvalue_plot

Access to data structures

In addition to the normal mode of loading data from files, Sherpa allows you to manipulate the data arrays internally at the Python prompt and then "re-load" them before fitting and modeling further. The resulting data may be manipulated without writing it to a file, making it easy to incorporate into scripts.

The analysis of data from modern instruments requires a flexible modeling and fitting system. The Sherpa infrastructure provides the user with an environment to develop complex and sophisticated analysis that goes beyond the default Sherpa functionality. Examples of user-contributed Python packages which extend Sherpa's functionality are Deproject and Cosmocalc (although the cosmology routines provided by AstroPy are to be preferred to Cosmocalc nowadays). The DataStack support was a user-provided extension but was integrated into Sherpa in CIAO 4.7.

The standalone Sherpa project allows users to install - and develop - Sherpa outside of CIAO. This allows users to combine Sherpa with external packages - such as SciPy and AstroML - which may not be easily installed into the CIAO directory structure.

Support for installing Python packages into CIAO has been greatly improved with the CIAO 4.11 release, and CIAO 4.12 was the first release of CIAO (and hence Sherpa including the XSPEC model library) using the conda package manager.

User Interface

There are many functions available for accessing internal data, model data, fit and projection results, plotting objects, etc., in a Sherpa session. These provide the user with easy access and interaction with the system required by modern data analysis. Examples of such functions include:

• show_data()
• show_bkg()
• show_model()
• show_filter()
• show_fit()
• show_all()
• show_xsabund()
• get_arf()
• get_rmf()
• get_response()
• get_bkg_scale()
• get_backscal()
• get_conf_results()
• get_data_plot_prefs()
• get_delchi_plot()

The much longer list of get*, set*, show*, etc., functions may be accessed by typing, e.g., "*get*?" at the Sherpa prompt.

Optimization methods

There are four optimization methods available in Sherpa: "levmar", a modification of the Levenberg-Marquardt algorithm which uses the LMDIF algorithm; "neldermead", A Nelder-Mead Simplex direct search; "moncar", a Monte Carlo method - differential evolution - based on the paper by Hedar & Fukushima; and "gridsearch", a grid searching method available for fitting data with Sherpa template models. These algorithms are robust and show very good convergence required by modern data analysis.

The Sigma-rejection iterative fitting method in Sherpa allow the chosen optimization method to be called multiple times during a fit, instead of just once, until the fit can no longer be improved. the Sigma-rejection method excludes poorly fit data points at each successive fit. Primini method was removed in CIAO 4.15.

Statistics

There are several choices of fit statistic in Sherpa, including Maximum-Likelihood based ones (cash, cstat, and wstat) and many options of weighted χ² statistics. The full list is:

• cash
• cstat
• wstat
• χ² statistics:
  • leastsq
  • chi2constvar (CHI PARENT | CVAR)
  • chi2datavar (CHI DVAR)
  • chi2gehrels (CHI GEHRELS)
  • chi2modvar (CHI MVAR)
  • chi2xspecvar

Descriptions of all available fit statistics are provided on the Sherpa Statistics page.

The Sherpa interface also provides an easy way to define new statistics required for analysis by the user. The load_user_stat function accommodates user-defined functions for a statistic and statistical errors, in addition to defining a list of model parameters and hyperparameters for prior distributions (if prior desired).

Parameter Uncertainty and Confidence

Confidence and covariance calculate the confidence range for the best-fit model parameters; interval-projection and region-projection provide a way to visualize the model constraints on the best-fit parameter values. The results are easily accessible with the user functions get_conf_results and get_covar_results.

The PPP statistic function plot_pvalue (with associated get_pvalue_results and get_pvalue_plot functions) performs a likelihood ratio test to compare a fit to data done with a simple, null model versus a more complex, alternative model. This function helps users determine if the fitted model is statistically significant compared to the null model.

The sample_flux function calculates the flux and flux uncertainty due to a sub-component of a Sherpa model, returning the model parameter values and associated fluxes from each of the user-specified number of simulations, along with the median flux value and the lower and upper quantiles defined by the confidence level supplied to the function.

Models

Sherpa allows the user to define customized models and provides several flexible options for defining complex model expressions for fitting data.

Multi-component, multi-response expressions

Complex model expressions which apply response matrices, or PSFs, to some model components, while not applying the response or the PSF to the rest of the model expression, may be defined with the set_full_model and set_bkg_full_model functions. This allows for including both convolved and unconvolved model components in a single model expression for fitting source and/or background data.

There are plotting functions available for visualizing the contribution of individual model components to a fit of a multi-component model to data. These functions can accept one or multiple Sherpa model components. Similarly, users can view 2D model components in DS9.

• plot_model_component()
• plot_model_components()
• plot_source_component()
• plot_source_components()
• get_model_component_plot()
• get_source_component_plot()
• image_model_component()
• image_source_component()
• get_model_component_image()
• get_source_component_image()
• get_model_component()

It is possible to simultaneously fit multiple data sets with each independent source model convolved by a different PSF model.

User models

The add_model function may be used to register a user-defined Sherpa model class as a Sherpa model type, to create instances of models that are recognized automatically in Sherpa model expressions.

The Sherpa table model reads data from a file or crate and assigns it to an instance of the table model class that can be combined arithmetically with the source model. It supports additive and multiplicative XSpec-style table models, as well as linear interpolation of data points on the data set grid from the model grid supplied from file.

The Sherpa template model can read in a collection of templates from a directory full of template files, and can compare a data set to all the templates in that collection. Sherpa finds the template that best matches the data, and reports back the parameter values associated with that template.

Both the table and template models support linear, nearest-neighbor, and polynomial interpolation.

Fitting

The fit_bkg function is available for fitting only background data sets when fitting with PHA spectral data. The function's list of arguments is the same as the fit function.

Iterative fitting can be done with the Primini and Sigma-rejection methods using the following functions:

• set_iter_method
• get_iter_method_name
• list_iter_methods
• get_iter_method_opt
• set_iter_method_opt

calc_stat_info and get_stat_info give the full statistical information based on the current models.

Filtering and Grouping Data

The notice and ignore functions are the basis for restricting the data used in a fit for 1D data. For 2D data the commands are notice2d and ignore2d. There are versions of these commands ending in _id when the filter is only to be applied to a single dataset: for example, notice_id and notice2d_id.

In order to simplify common use cases, if the first filter command is a notice call then it is treated as if you have ignored all the data first. Subsequent notice calls do not have this behaviour for the dataset. This means that

load_pha("src.pi")
set_analysis("energy")

notice(0.5, 3)
notice(4, 6)

acts as if you have ignored everything and then notices the 0.5 to 3 keV and the 4 to 6 keV ranges. To help identify what happens, these commands will display the old and new selected ranges; as an example the two commands will produce output something like:

dataset 1: 0.00146:14.9504 -> 0.4964:3.0076 Energy (keV)
dataset 1: 0.4964:3.0076 -> 0.4964:3.0076,3.9858:6.0006 Energy (keV)

(the actual values depends on the dataset and the respose information).

The following grouping functions are avaible for grouping PHA channel data. These functions use the CIAO group module to group data in various ways:

• group_bins - Group into a specified number of bins
• group_width - Group so there are N channels in each group
• group_counts - Group by a number of counts
• group_snr - Group by a signal-to-noise value
• group_adapt - Adaptively bin based on the number of counts
• group_adapt_snr - Adaptively bin based on a signal-to-noise value

It is strongly suggested that filtering is applied before setting up the grouping. This is to ensure that the grouping is only applied to the noticed channel range.

Saving the Sherpa Session

save_all command saves all the settings which define the current Sherpa session to an ASCII file. The resulting file can be read into a future Sherpa session to recreate the data, models, filter, grouping and quality flags, statistic and method used at that time.

The save command creates a binary file that can be restored in Sherpa with the restore. Please note that this file is likely to better reflect the actual analysis than the ASCII file, but

• it is not guaranteed to be usable with older or newer versions of CIAO,
• it can not be edited (unlike the ASCII file),
• and should only be loaded if you trust the person who created it, as the loaded code could execute random code.

Advanced options

Bayesian analysis with pyBLoCXS

pyBLoCXS, a sophisticated Markov chain Monte Carlo based algorithm designed to carry out Bayesian Low-Count X-ray Spectral (BLoCXS) analysis in the Sherpa environment, is available via the functions listed below. (For more information on BLoCXS, see van Dyk, D.A., Connors, A., Kashyap, V.L., & Siemiginowska, A. 2001, Ap.J., 548, 224, "Analysis of Energy Spectra with Low Photon Counts via Bayesian Posterior Simulation")

• get_draws()
• list_priors()
• set_prior()
• get_prior()
• list_samplers()
• set_sampler()
• set_sampler_opt()
• get_sampler()
• get_sampler_opt()
• get_sampler_name()

The code is a Python extension to Sherpa that explores parameter space at a suspected minimum using a predefined Sherpa model to high-energy X-ray spectral data. The get_draws function runs a pyBLoCXS chain using fit information associated with the specified data set(s), and the currently set sampler and parameter priors, for a specified number of iterations. It returns an array of statistic values, an array of acceptance Booleans, and a 2-D array of associated parameter values.