/ sherpa / notebook_plotter / notebook_plotter.html

notebook_plotter: An Interactive Tool for Model Plotting and Fitting in Sherpa

The sherpa contributed tool notebook_plotter utilizes Jupyter Widgets to interactively view models and data loaded into Sherpa. The purpose of the tool is help provide intuition about spectral model fitting by visualizing how effective area and instrumental response modify the shape of input theoretical models. This tool is useful for exploring the parameter space for fitting astrophysical models to data and for proposal planning purposes.

If you have never used sherpa with jupyter notebooks, you should use this guide for understanding the small differences compared to using sherpa via the command line. Notebook plotter must be used with sherpa and notebooks, whether the classic notebook interface or the newer lab environment. The What is Jupyter? page provides a lot of information on why people may want to use "computational notebooks".

Run a more detailed version of the below tutorial on your own machine

CIAO versions 4.17 and later come with a pre-installed jupyter notebook tutorial file for building model fitting intuition with notebook_plotter. This enhanced tutorial simulates a dataset allowing you to explore all the notebook_plotter tool features without needing to provide your own data. It also includes an example science case how notebook_plotter can be utilized for proposal planning. It is advised to copy the tutorial file into a new working directory before running it so you don't overwrite the original file. After initializing CIAO, the following commands will copy the tutorial into a new 'notebooks' directory and run it in a jupyter notebook environment:

unix% mkdir notebooks  
unix% cp $ASCDS_INSTALL/share/sherpa/notebooks/*ipynb notebooks/  
unix% jupyter lab --notebook-dir notebooks

The sections of this document are:

Starting the notebook environment

The lab or notebook environments can be used. For example, after ensuring that the Jupyter environment is installed:

unix% jupyter lab
...
[I 2025-05-01 15:41:55.575 ServerApp] Jupyter Server 2.14.2 is running at:
[I 2025-05-01 15:41:55.575 ServerApp] http://localhost:8888/lab?token=big-long-hexadecimal-string
[I 2025-05-01 15:41:55.575 ServerApp]     http://127.0.0.1:8888/lab?token=big-long-hexadecimal-string
[I 2025-05-01 15:41:55.575 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2025-05-01 15:41:55.671 ServerApp] 
    
    To access the server, open this file in a browser:
        file:///...a path
    Or copy and paste one of these URLs:
        http://localhost:8888/lab?token=big-long-hexadecimal-string
        http://127.0.0.1:8888/lab?token=big-long-hexadecimal-string
...

The screen output should include the web page to navigate to, if it hasn't already been selected. The locations used may well differ to those listed in the example above.

Loading Sherpa and notebook_plotter

As well as Sherpa, it is often useful to ensure that NumPy and Matplotlib are loaded:

import numpy as np
from matplotlib import pyplot as plt  

from sherpa.astro.ui import *
from sherpa_contrib.notebook_plotter import notebook_plotter

Note

Unlike other CIAO sherpa notebook threads, this document does not import Sherpa as ui ("from sherpa.astro import ui") because it can be inconvenient to include 'ui' in front of every sherpa function in a notebook. However, one advantage to using the explicit ui name is that it makes it easier to use tab-completion - e.g. after entering ui.plot tab completion will only look for Sherpa commands that start with plot.

The notebook_plotter Interface

The notebook_plotter interface consists of a plotting window and two sets of jupyter widgets below the plot. The 'Plotting Options' set of widgets allow interactive control of the plotting window and the 'Model Parameter' set of widgets allows control over the model parameter values. Please see the notebook_plotter help file e.g., help(notebook_plotter) for a complete description of every parameter.

The plot window options and model parameters can be controlled by interacting with the individual widgets. Some widgets provide a dropdown box to select from a preset range of parameters while others have values that can be entered manually or buttons that can be dragged left and right to increase or decrease their values. The information plotted in the window will be updated live as the parameter values are changed.

Note

The figures on this webpage showing the notebook_plotter windows are static. When running notebook_plotter in a jupyter_notebook, users will see models and data update live.

A Basic Example of Using notebook_plotter with Data

The notebook_plotter tool utilizes standard sherpa commands so data and/or models must be loaded first following the appropriate syntax for using sherpa in a notebook. First, let's load a spectral model into sherpa and visualize it with notebook_plotter.

abs_apec_model = xstbabs.ma1 * xsapec.p1
abs_apec_model

xstbabs.ma1 * xsapec.p1
   Param        Type          Value          Min          Max      Units
   -----        ----          -----          ---          ---      -----
   ma1.nH       thawed            1            0        1e+06 10^22 atoms / cm^2
   p1.kT        thawed            1        0.008           64        keV
   p1.Abundanc  frozen            1            0            5
   p1.Redshift  frozen            0       -0.999           10
   p1.norm      thawed            1            0        1e+24

The notebook_plotter tool makes it easy to visualize model parameters but sometimes sherpa 'min' and 'max' model parameters covers a very large range making it inconvenient for plotting purposes. Before initializing notebook_plotter, lets first set more realistic minimum and maximum values for ma1.nH. Modifying the min and max values should always be done carefully with a particular science case in mind.

ma1.nH.min = 0.01
ma1.nH.max = 40

Now we can simply initialize notebook_plotter using the loaded model we called 'abs_apec_model'.

notebook_plotter(abs_apec_model)

All of the plotting window and model parameters can be adjusted with a mouse and keyboard in the notebook_plotter window. However, you can also set all of these parameters when initializing notebook_plotter in case you don't like the default values.

ma1.nH = 10
p1.kT = 2.0
notebook_plotter(abs_apec_model, autoscale='none', xlim_low=0.5, xlim_high=8.0, ylim_low=1E-4, ylim_high = 10, log_axis='ylog')

Note

It is important to understand that changing a model parameter value in the notebook_plotter window updates the model parameter value in your sherpa session. When you advance to the next cell in a notebook, the model values in 'abs_apec_model' will be identical to the final state of the notebook_plotter window. You can override these values by setting the model parameters in subsequent cells as you typically would with sherpa (e.g., ma1.nH = 1.0).

If your notebook has multiple interactive plotting windows open it is not advised to go to previous cells and continue interacting with previous plotter windows. The model values in these previous cells will not be automatically updated.

Now that we have a model setup, we can load in a spectrum and see what it looks like with our model convolved with the detector response. More information about this process can be found in the Extract Spectra and Response Files for a Pointlike Source and fit PHA spectra threads.

load_pha('spectrum.pi')

read ARF file spectrum_arffile.arf
read RMF file spectrum_rmffile.rmf

Note

If the ARF and RMF (responses) are not loaded in automatically with load_pha() then you can manually load them with load_arf() and load_rmf(), respectively.

# ignore everything but 0.5-8.0 keV part of the spectrum and then group the spectrum by 10 counts per bin
notice()
ignore(hi=0.5)
ignore(lo=8)
group_counts(1, 10)

set_source(abs_apec_model) #set the model spectrum to the input dataset

dataset 1: 0.0073:14.9504 Energy (keV) (unchanged)
dataset 1: 0.0073:14.9504 -> 0.511:14.9504 Energy (keV)
dataset 1: 0.511:14.9504 -> 0.511:7.9862 Energy (keV)
dataset 1: 0.511:7.9862 Energy (keV) (unchanged)

notebook_plotter(abs_apec_model, log_axis = 'ylog', autoscale = 'y', figsize_y = 3, plot_type = 'data_and_convolved_model')

The orange line in the plot is the absorbed apec plasma model convolved with the ACIS instrument responses. This is how the previous unconvolved models from above (purple) will appear when observed with Chandra. This model clearly is not a good fit at the moment to the data (blue crosses). Before fitting the data, you could choose to explore the model parameter space by adjusting the model parameters until the orange model better fits the data. In many cases, the sherpa fit function will find appropriate values without needing manual intervention. However, the notebook_plotter tool offers a good way to visually adjust parameters and build intuition on how certain model parameters may be impacting spectral fits.

fit()

Dataset               = 1
Method                = levmar
Statistic             = chi2gehrels
Initial fit statistic = 3.27828e+08
Final fit statistic   = 188.391 at function evaluation 58
Data points           = 277
Degrees of freedom    = 274
Probability [Q-value] = 0.999979
Reduced statistic     = 0.687558
Change in statistic   = 3.27828e+08
   ma1.nH         0.137072     +/- 0.0279393   
   p1.kT          1.89406      +/- 0.0424396   
   p1.norm        0.00100342   +/- 2.29621e-05

After running the fit, the parameters have been updated in sherpa to their best-fit values. Let's replot the data and best-fit model in notebook_plotter.

notebook_plotter(abs_apec_model, log_axis = 'ylog', autoscale = 'y', plot_type = 'data_and_convolved_model')

Here we can see that the model is a reasonable fit to the data. Let's see how the underlying unconvolved model affects the convolved model fit to the spectrum by changing the notebook_plotter 'plot_type' to 'both_models'. You can either do this by clicking the 'plot_type' widget or by manually declaring it in the function call to notebook_plotter.

notebook_plotter(abs_apec_model, log_axis = 'ylog', autoscale = 'y', plot_type = 'both_models')

In the above plot we can see how the underlying (unconvolved) model affects the model fit to the data. In this particular case, we have an absorbed thermal plasma with a continuum and several emission features that make the convolved spectrum 'bumpy'. For example, the emission component at ~6.7 keV is a forest of Fe emission lines and the strength of these features depends strongly on the plasma temperature.

The notebook_plotter tool can also be used on data in wavelength space. We can change the analysis to units of wavelength using sherpa's set_analysis function and call notebook_plotter again.

set_analysis('wavelength')
notebook_plotter(abs_apec_model, log_axis = 'ylog', autoscale = 'y', plot_type = 'both_models', xlim_high=20)

The notebook_plotter Parameters

The notebook_plotter help file contains details about all parameter options and includes some common examples.

help(notebook_plotter)

Help on function notebook_plotter in module sherpa_contrib.notebook_plotter:

notebook_plotter(model_var, dataset_id=1, model_resolution=0.001, autoscale='xy', xlim_low=0.3, xlim_high=10, ylim_low=0.001, ylim_high=10, log_axis='none', plot_type='model_unconvolved', figsize_x=10, figsize_x_min=0.1, figsize_x_max=30, figsize_y=5, figsize_y_min=0.1, figsize_y_max=30, limit_style='soft', widget_step=0.001)
        
    This functions creates the interactive jupyter notebook plotting window allowing users to visualize changes in model parameters and how that may affect spectral fitting. Users can adjust the 'Model Parameters' section of the interactive plot to visualize real-time changes in the underlying spectral model. Users can modify the Plotting Options in the interactive plot to customize what is plotted. This tool is currently a prototype and is meant to demonstrate the usefulness of python ipywidgets in visualizing sherpa data. For full functionality, users should have a dataset loaded with proper responses and a model set (e.g., with load_pha() and set_source() ). Please note, running this tool and modifying the model parameters will change the model values in your sherpa session.
        
            Parameters
            ----------
                model_var: sherpa.models.Model object
                A variable assigned to a sherpa  model. This cannot be a string.
                Example: my_model = (xstbabs.a1 * xsapec.p1)
    
                dataset_id: int
                    The sherpa dataset identifier. The default value is dataset id 1.
    
                figsize_x_min: float
                    The minimum allowable X-axis size of the interactive plotting window.
    
                figsize_x_max: float
                    The maximum allowable X-axis size of the interactive plotting window.
    
                figsize_y_min: float
                    The minimum allowable Y-axis size of the interactive plotting window.
    
                figsize_y_max: float
                    The maximum allowable Y-axis size of the interactive plotting window.           
    
                limit_style: string
                    This parameter can be either 'soft' (default) or 'hard'. Soft limits will set the user-defined limits when plotting the maximum and minumum model parameter values. Hard limits will set the model-defined maximum and minimum model parameter values.
    
                widget_step: float
                    The minimun step to increment the value for all 'Model Parameter' widget sliders.                
    
                Description of Interactive Plotting Options
                -------------------------------
               model_resolution: float
                    Desired spectral resolution of the plotted model in units of keV. The spectral model will be evalulated over an energy/wavelength grid every resolution element. If plotting in wavelength space, model_resolution is automatically converted from keV to Ang.
    
                autoscale: string
                    Whether or not to autoscale the respective axes using the values plotted. 'None' means no autoscaling, 'x' means x-axis only autoscaling, 'y' means y-axis only autoscaling, and 'xy' means both x and y axes are autoscaled.                
    
                xlim_low: float
                    The minimum value for the x-axis plot widget.
    
                xlim_high: float
                    The maximum value of the x-axis plot widget.
    
                ylim_low: float
                    The minumum value for the y-axis plot widget.
    
                ylim_high: float
                    The maximum value for the y-axis plot widget.
    
                log_axis: string
                    Whether or not to plot axes in log-space. 'None' means no log space plotting, 'xlog' means xlog only plotting, 'ylog' means ylog only plotting, and 'loglog' means both x and y are set to log plotting.
    
                plot_type: string
                    This parameter determines what is plotted. 'model_unconvolved' plots the source model using the model resolution and without folding in the instrument response. 'model_convolved' plots the source model convolved with the instruments (ARF+RMF) responses. 'both_models' plots both unconvolved (purple) and convolved models (orange) on the same plot. For this plot_type, plotting options control the unconvolved model plot while the y-axis of the convolved model autoscales to remain within the x and y limits set for the unconvolved model. 'data_and_convolved_model' plots the loaded dataset with the source model convolved with the response. When plotting a convolved model or data, the model_var must be set to the same dataset value as indicated with dataset_id.
    
                figsize_x: float
                    Sets the initial widget x-axis figure size.
    
                figsize_y: float
                    Sets the initial widget y-axis figure size.
    
             Notes
             -------
             Interactive plotting options can be set by selecting them in the plotting window or directly in the call to the the notebook_plotter() function. 
    
             Units of model_resolution parameter should always be in keV regardless whether or not set_analysis is set to 'energy' or 'wavelength'. If plotting in wavelength, keV resolution elements are converted to Angstrom.  This tool does not work with other set_analysis() options.              
    
             Some of the parameter widget boxes and sliders have clickable up and down arrows to adjust parameter values. Only the minimum 'Model Parameter' slider increments can be adjusted using the widget_step parameter. Users can override most parameter values by clicking the values and entering new values using a keyboard.
             
             Examples
             -------
    
             Define a model and plot the unconvolved model with default notebook_plotter parameters. Here we set a maximum to nH to better facilitate viewing using widgets:
    
             >>> my_model = (xstbabs.a1 * xsapec.p1)
             >>> a1.nH.max = 50
             >>> notebook_plotter(my_model)
    
             Plot the same model as previous example with a new resolution of 0.1 keV. Modify the plotting parameters by setting the y-axis in log-scale, setting limits on the x-axis and y figure size while plotting the unconvolved model:  
    
             >>> notebook_plotter(my_model, model_resolution = 0.1, log_axis = 'ylog', xlim_low = 0.3, xlim_high =10, figsize_y = 3, plot_type = 'model_unconvolved')
    
             Load a dataset and associated responses with id=2, set a model to the data and then plot the data with the convolved model overlaid:
    
             >>> load_pha(2,'src2.pi')
             >>> load_arf(2, 'src2.arf')
             >>> load_rmf(2, 'src2.rmf')
             >>> my_model_2 = (xsapec.p1 + xsapec.p2)
             >>> set_source(2, my_model_2)
             >>> notebook_plotter(my_model_2, dataset_id = 2, plot_type = 'data_and_convolved_model')
    
             Compare the unconvolved and convolved models in the previous example in wavelength (Angstrom) and adjust the minimum widget slider increment for model parameters to be at least 0.1:
    
             >>> set_analysis('wavelength')
             >>> notebook_plotter(my_model_2, dataset_id = 2, plot_type = 'both_models', widget_step  = 0.1)

/ sherpa / notebook_plotter / notebook_plotter.html