Latest Updates
This document highlights important changes and additions to Sherpa functionality in the CIAO 4.3 Sherpa release.
To compare the features and functionality of Sherpa in CIAO 4.3 to Sherpa in CIAO 3.4, refer to the pages About Sherpa and Sherpa 3.4 to 4.3 Conversion.
 Startup Options

 Removal of Slang Support

In CIAO 4.3, the SLang interface to Sherpa is not supported. There is no longer any means of importing Sherpa as a SLang module, as the SLang version of the Sherpa highlevel UI has been removed.
 It is no longer possible to do 'require("sherpa");' in slsh, or any other SLang application.
 Sherpa no longer reads the environment variables CIAO_SCRIPT_LANG or SHERPA_SCRIPT_LANG, and will issue error messages if these variables are set to "slang".
 There is no longer an option to use slsh as the interpreter in the Sherpa application script. The "l" commandline option has been removed. The following are the remaining commandline options which may be specified when Sherpa is started:
% sherpa h  sherpa usage: sherpa [x] [n] [b] [rcfile file] [norcfile] <file> x launch sherpa shell in separate display terminal n do not print banner b batch mode rcfile load a user specifiable preference file norcfile do not load any user specifiable preference file (overrides rcfile) <file> appropriate python command file to execute
 .sherpa.rc

 The option "shell" is no longer needed in the user's .sherpa.rc file. If this is set to "slang", the application will exit with an error:
Sherpa ERROR: Sherpa no longer offers a SLang interface. The 'shell' option in /home/username/.sherpa.rc is no longer used.
 The new section [parallel] in the .sherpa.rc file includes a switch "numcores"; more information is available in the "Parallelization" section, below. The default value for "numcores" is None. To specify the number of cores to use for all Sherpa concurrent functions, change the default value:
[parallel] numcores : 3
 Parallelization

Several computationally intensive functions, such as projection and conf, were parallelized in the CIAO 4.2 release, to make use of multicore systems (i.e., laptops or desktops with 2 or 4 cores). In Sherpa in CIAO 4.3, a new option, "numcores" has been added to the following functions to specify the number of cores to use in parallelization. The default is to use all cores available.
 Loading, Filtering, and Viewing Data

The new function get_bkg_scale returns the coefficient which is used to scale background counts during background subtraction of a source spectrum, or a simultaneous fit of source and background spectra. The complete scaling factor used to scale the background counts in these cases consists of the product of the sourcetobackground exposure and extraction region area (backscal) ratios (the backscal value may be retrieved with the get_backscal function). The complete background scale is returned by the show_data and show_all functions.
The load_filter function has been updated to include the ability to read FITS images that hold filter information, e.g., an image file containing 1s and 0s to mark which pixels should be filtered. There is a new keyword argument 'ignore' that indicates whether the filter information should be used to notice or ignore data points. "ignore" is False by default.
sherpa> load_filter(id=1, filename, bkg_id=None, ignore=False, ncols=2) sherpa> set_filter(id=1, val, bkg_id=None, ignore=False)

The plot_source command now supports the 'factor' setting of set_analysis. This means that calling plot_source while the set_analysis 'factor' setting is '1' will plot E F(E) versus E in keV (XSPEC plot eufspec), or λ f(λ) versus λ in Angstroms, depending on which units are set for the spectral analysis. A set_analysis 'factor=2' setting will plot E^{2} F(E) versus E, or λ^{2} f(λ) versus λ (XSPEC plot eeufspec).
There are new plotting functions to plot individual model components contributing to the fit of a multicomponent model to data. Users can pass one or more Sherpa model components to the plotting function to quickly view the contribution to the model. Similarly, users can view 2D model components in DS9.
 plot_model_component()
 plot_source_component()
 get_model_component_plot()
 get_source_component_plot()
 image_model_component()
 image_source_component()
 get_model_component_image()
 get_source_component_image()
 get_model_component()
The following new functions set the x and y axis scales for Sherpa plots:
These functions accept arguments similar to the generic plot command: "data", "model", "source", "delchi", etc.
 Models

Sherpa models now have a "cache" attribute for users to control behavior. The default value for "cache" is a nonzero value indicating that caching will be turned on only if all parameters in the model are frozen. Indicating zero will turn off caching even if all model parameters are frozen.
Model caching is now available for XSPEC and 1D analytic models; caching is turned on by default. 2D Sherpa analytical models are not cached by default due to the potential impact on memory usage. Compared to fit results in previous releases, there should be no change to any calculated value. The only difference that may be seen is in reduced program execution time.
sherpa> set_source( xsphabs.abs1 * powlaw1d.p1 ) sherpa> thaw(abs1) sherpa> abs1.cache=0
The new 1D integration helper function, "integrate1d", can be used to define 1D numerical integration on a particular arbitrary Sherpa model expression. This function is used inside a Sherpa model definition as an explicit indication that Sherpa should numerically integrate the expression in parenthesis. The helper function numerically integrates the expression as a whole, correctly, instead of each component individually. Note that the model expression should not include any XSPEC additive models, as those models perform integration on themselves.
sherpa> set_model(integrate1d.int1(beta1d.b1*gauss1d.g1)) sherpa> print int1 integrate1d.int1 Param Type Value Min Max Units       int1.epsabs frozen 2.22045e16 3.40282e+38 3.40282e+38 int1.epsrel frozen 0 3.40282e+38 3.40282e+38 int1.maxeval frozen 10000 3.40282e+38 3.40282e+38
This helper function is used in lieu of the "integrate" flag found on Sherpa models.
 New Models

The following models are now available in Sherpa:
 scale1d  1D constant amplitude model
 scale2d  2D constant amplitude model
 logparabola  a logparabolic function
The scale1d model is simply the const1d model with the integrate flag turned off by default. If scale1d is set as a model for fitting an integrated data set, it will behave as a simple constant by default.
The logparabola model is defined by a logparabolic function of the following form:
f(x) = A (x/x_ref)^{[gamma  beta*log10(x/x_ref)]}
New XSpec models included in Sherpa are:
 xssirf
The new XSPEC functions set_xsxset and get_xsxset() are associated with the XSPEC function XSET for defining model environment variables.
sherpa> set_xsxset("NEIVERS", "2.0") sherpa> set_xsxset("NEIAPECROOT", "/home/brefsdal/apec") sherpa> set_model(xsvnei.nn) sherpa> get_xsxset("NEIAPECROOT") sherpa> '/home/user/apec'
 PSF Instrument Models

2D PSF models are no longer required to have equal (x,y) size parameters; rectangular PSFs are now supported. It is also now possible to simultaneously fit multiple data sets with each independent source model convolved by a different PSF model.
 Table Models

Additive and multiplicative XSPECstyle table models are now supported by the Sherpa table model using load_table_model, e.g.:
sherpa> load_table_model("tab1", "bhmodel.fits") sherpa> set_model(tab1)
A table model contained in a file written in the format required by the XSpec table model, i.e. a file containing an Ndimensional grid of model spectra, can be loaded and set as a source model in Sherpa.
The Sherpa table model has also been updated to support linear interpolation of data points on the data set grid from the model grid supplied from file. The grids need not be of constant or comparable bin size, and it is no longer necessary to match the length of the table model column to that of the data. If the table model grid is not sorted, Sherpa will sort it in ascending order.
 User Models

The new function add_model is available as an alternative to using load_user_model together with add_user_pars for explicitly defining user models as Python classes for use in Sherpa. The add_model command registers a userdefined Sherpa model class as a Sherpa model type, to allow users to create instances of models that are recognized automatically in Sherpa model expressions.
sherpa> from sherpa.models import PowLaw1D sherpa> class MyPowLaw1D(PowLaw1D): pass sherpa> add_model(MyPowLaw1D) sherpa> set_model(xswabs.abs1*mypowlaw1d.p1)
 Definining Model Expressions

Sherpa now allows the user to define model expressions that apply response matrices, or PSFs, to some models, while not applying the response or the PSF to the rest of the model expression. An example of this kind of model is an expression where a spectral model is defined in energy space, and folded through a response matrix; then, a background model defined in counts, which is not folded through the response, is added to the model expression.
The new functions set_full_model() and set_bkg_full_model() allow users to explicitly define instrument responses and convolutions that are applied to specified model components. PSF and table models may be used in model expressions defined with these functions.
Legacy functionality is still supported with set_source() and set_model(); CIAO 4.3 scripts using these functions will continue to work in the current Sherpa.
Automatic Manual Definition ============= ================= set_source() set_full_model() set_model() set_bkg_source() set_bkg_full_model() set_bkg_model()
 Instrument Responses

The new function get_response returns the instrument response model associated with a PHA data set, which may be used together with the set_full_model() and set_bkg_full_model() functions for explicitly defining complex convolved model expressions.
The get_arf and get_rmf functions have been updated with the ability to return the 'instrument' version of an ARF and RMF data set, allowing the output of these functions to be used like that of the get_response function for explicitly defining convolved model expressions. The multiplication with the exposure time is implicit in such a model expression. The following two sets of commands produce equivalent results.
sherpa> arf = get_arf() sherpa> rmf = get_rmf() sherpa> set_full_model(rmf(arf(xsphabs.abs1*powlaw1d.p1))) sherpa> rsp = get_response() sherpa> set_full_model(rsp(xsphabs.abs1*powlaw1d.p1))
 Fitting

The Primini and sigmarejection iterative fitting methods which were available in Sherpa 3.4 are now available in Sherpa 4. Several new functions are available to allow users to set the iterative fitting method, to find out what the current iterative fitting method is, and to get and set options for this method. These functions are:
 set_iter_method  choices are "none", "primini", "sigmarej"
 get_iter_method_name  print the name of the current iterative fitting method
 list_iter_methods  list all available iterative fitting methods
 get_iter_method_opt  get the value of the named option for the current iterative fitting method. If no argument, list all options.
 set_iter_method_opt  Set the value of the specified option for the current iterative fitting method
If the iterative fitting method is "none" (the default value), then no iterative fitting is done  when "fit()" is called, the optimization method is called once, and Sherpa otherwise operates as expected. The fit statistic and optimization methods are selected independently of the iterative fitting method
Primini's method is used for recalculating statistical errors using the bestfit model parameters from the previous fit, until the fit can no longer be improved.
The sigmarejection method is based on the IRAF SFIT function. In successive fits, data points for which ((data  model) / error) exceeds some threshold are added to the filter, and automatically excluded from the next fit.
Primini's method and sigmarejection can only be called when the fit statistic is a χ^{2} statistic; it cannot be used with leastsquares, Cash or Cstatistic.
The new functions calc_stat_info and get_stat_info implement functionality similar to the GOODNESS command from Sherpa 3.4. Goodnessoffit information usually found in a Sherpa fit result is now also calculated separately. It is no longer necessary to perform a new fit just to return goodnessoffit information.
 Additional Enhancements & Bug Fixes
 Parallelization

In Sherpa in CIAO 4.3, users will now see that when CtrlC is issued while proj() or conf() is running, Sherpa will correctly kill off slave processes from any parallel evaluation. Also, parallel evaluation of proj() and conf() is now automatically turned off on single core machines.
 Models

New arguments have been added to the list_models command; valid arguments are "all", "xspec", "1d", and "2d".
The Sherpa powlaw1d power law model no longer suffers from a numerical instability with a gamma setting close to 1, while the integrate flag is switched on.

XSPEC model 'refflag' parameters are now always frozen.

The XSPEC function set_xsabund is updated to accept userdefined files containing solar abundances.

The atten model now correctly ignores the integrate setting.

The error messages which are printed during model evaluation when arrays are different sizes, have been improved.
 Data Display

The plot functions int_proj and int_unc include dashed lines indicating the best fit statistic and best fit parameter value.
The plot_bkg_unconvolved command has been removed; the identical plot_bkg_source command should be used, instead.
The show_all command now correctly displays information on background data sets loaded into the Sherpa session.

Sherpa now uses a private DS9 window for communication, with ID 'sherpa' for session imaging.

Axis units for plots created with the contour_* functions are now correctly returned in pixels when the coordinate setting for the session is 'physical'.

When the statistical errors resulting from a fit are calculated by the chosen fit statistic and the systematic errors are set by the user, the error bars are now correctly displayed on plots of the fit.

The get_arf function no longer erroneously returns the ARF for data set 1, when a data set other than 1 is explicitly entered as the argument.

The show_bkg_source command now correctly recognizes a specified bkg_id value
 Data Properties

For grating data sets, set_analysis("wave") now correctly sets both the source and background to wavelength units.
 Saving Data

The save function now includes XSPEC module settings in the saved binary file. These settings are the abundance, chatter level, cosmology, electron crosssection, and model variables set via the XSPEC "xsxset" function.
The restore function reads these settings back in and calls the appropriate XSPEC module functions to reset the values.
The save_all function, which saves attributes of a Sherpa session to a humanreadable text file, has been modified. The text file created by save_all now includes:
 The iterative fitting method, if that has been set;
 Source model assignment via either set_source or set_full_model, as appropriate;
 XSPEC module settings for abundances, chatter level, cosmology, electron crosssection, and model settings via the "xsxset" function.
Text files created with the CIAO 4.2 version of save_all can still be used in the CIAO 4.3 version of Sherpa.
 Modules

The utils module include a new utility function, parallel_map(), which is a parallelized version of the native Python map which evaluates a function over a list of arguments.
sherpa> from sherpa.utils import parallel_map sherpa> parallel_map(lambda x: x*x, [1,2,3]) > [1,4,9]
The follow statistical PDF functions for multivariate normal and multivariate student T distributions have also been added to Sherpa utils module.
sherpa> dof = get_fit_results().dof sherpa> sigma = get_covar_results().extra_output sherpa> mu = get_model().thawedpars sherpa> x = numpy.random.multivariate_normal(mu, sigma) sherpa> from sherpa.utils import multinormal_pdf, multit_pdf sherpa> t_pdf = multit_pdf(x, mu, sigma, dof) sherpa> normal_pdf = multinormal_pdf(x, mu, sigma)
The new vectorized function "interpolate" may be used as follows:
sherpa> from sherpa.utils import interpolate sherpa> yout = interpolate(xout, xin, yin, method='linear''nearest')