Using XSpec User Models in Sherpa
![[CXC Logo]](/ciao/imgs/cxc-logo.gif){kind=logo}
Sherpa Threads (CIAO 4.17 Sherpa)
Overview
Synopsis:
The experimental script convert_xspec_user_model provides support for using some XSpec contributed models—whether written youself or obtained from the new XSpec models page—from Sherpa.
Last Update: 22 Aug 2023 - Rewritten for CIAO 4.15; examples using binaries and Conda distribution of CIAO on Linux and MacOS platforms with different user models.
Contents
- Preamble
- Compiling a user model for the standard CIAO binaries distribution
- Compiling a user model for the Conda CIAO distribution
- What to do if the build fails
- Testing the model in Sherpa
- Example: a model that successfully compiles to completion but fails at execution
- History
- Images
Preamble
The convert_xspec_user_model script is provided as part of the CIAO contributed-scripts package. The script underwent significant enhancements and 'under-the-hood' re-working that was introduced alongside the CIAO 4.15 release.
For the purposes of the thread,
- Version 2.3 of the relxill collection of relativistic and non-relativistic reflection models will be used for the example of building the models for the pre-compiled CIAO binaries installed with the ciao-install script.
- Version 1.0.1 of the RELTRANS reverberation mapping models will be used for the example of building with a Conda distribution of CIAO.
- Version 2.47 of the WARMABS XSpec package—a collection of warm absorption and photoionized emission models derived from the XSTAR program that calculates physical conditions and spectra of photoionized gasses.
will be used for the examples herein. The convert_xspec_user_model script is similar to the XSpec initpackage command, so follow the model instructions to make any changes that would be needed before running initpackage, and then run convert_xspec_user_model instead.
![[TIP]](../../imgs/tip.png)
Often, compilers require access to system files to correctly build programs and a standardized set for MacOS is available through XCode's Software Development Kit (SDK).
The CIAO binary installation already requires XCode presently, so you may set:
(t)csh unix% setenv SDKROOT `xcrun --show-sdk-path` bash/zsh unix% export SDKROOT=`xcrun --show-sdk-path`
For Conda, it may be also necessary to set CONDA_BUILD_SYSROOT,
(t)csh unix% setenv CONDA_BUILD_SYSROOT `xcrun --show-sdk-path` bash/zsh unix% export CONDA_BUILD_SYSROOT=`xcrun --show-sdk-path`
but if XCode is not available on your system for Conda, the SDK may be externally acquired. The environment variable may then be explicitly set to where it has been saved to, for example if it is located in /opt/MacOSX10.10.sdk:
(t)csh unix% setenv CONDA_BUILD_SYSROOT /opt/MacOSX10.10.sdk bash/zsh unix% export CONDA_BUILD_SYSROOT=/opt/MacOSX10.10.sdk
Compiling a user model for the standard CIAO binaries distribution
In this example, the relxill models package will be used. The file relxill_model_v2.3.tgz is assumed to have been downloaded and unpacked, for example:
unix% cd /data/models/relxill_v2 unix% tar -xzf relxill_model_v2.3.tgz unix% ls -F common.h ModelParams.h Relprofile.h Relxill.cpp compile_relxill.sh README.txt Relreturn_BlackBody.cpp Relxill.h config.h Relbase.cpp Relreturn_BlackBody.h writeOutfiles.c donthcomp.c Relbase.h Relreturn_Corona.cpp writeOutfiles.h IonGradient.cpp Relcache.cpp Relreturn_Corona.h Xillspec.cpp IonGradient.h Relcache.h relreturn_datastruct.c Xillspec.h LICENSE Rellp.cpp relreturn_datastruct.h xilltable.c lmodel_relxill.dat Rellp.h relreturn_table.c xilltable.h LocalModel.cpp Relmodels.cpp relreturn_table.h XspecSpectrum.h LocalModel.h Relmodels.h reltable.c xspec_wrapper_lmodels.cpp ModelDatabase.h relphysics.c reltable.h xspec_wrapper_lmodels.h ModelInfo.h relphysics.h relutility.c ModelParams.cpp Relprofile.cpp relutility.h
Setting up the model
For version 2.x of the relxill model, in the directory with the unpacked source code, the line #include "fftw/fftw3.h" needs to be changed to #include "fftw3.h" since the fftw3 header file is in a different relative location when it is built for CIAO/Sherpa than where it is located for HEASoft/XSpec.
Linux (GNU): unix% sed -i 's,#include "fftw/fftw3.h",#include "fftw3.h",' common.h unix% sed -i 's,#include "fftw/fftw3.h",#include "fftw3.h",' Relbase.cpp MacOS (BSD): unix% sed -i '' -e 's,#include "fftw/fftw3.h",#include "fftw3.h",' common.h unix% sed -i '' -e 's,#include "fftw/fftw3.h",#include "fftw3.h",' Relbase.cpp
![[NOTE]](../../imgs/note.png)
Many models require no set up or code changes.
Compile the model
After any setup, the convert_xspec_user_model script is used to compile the model. For many cases you just need a name for the Python module containing the model, or models, and the name of the "model.dat" file. In this case we use 'relxill_v2' for the model name; this will be used to load in the model for testing later.
(t)csh
unix% setenv CC `which gcc`
unix% setenv CXX `which g++`
bash/zsh
unix% export CC=`which gcc`
unix% export CXX=`which g++`
unix% convert_xspec_user_model relxill_v2 lmodel_relxill.dat
convert_xspec_user_model: 21 April 2023
name: relxill_v2
modelfile: lmodel_relxill.dat
clobber: False
Found no Fortran files.
Found 8 C files.
Found 13 C++ files.
Processing 12 models.
Using language interface: C style
Using XSPEC version: 12.12.1c
..............................
... a lot of screen output ...
..............................
Successfully built relxill-v2
Installing collected packages: relxill-v2
Successfully installed relxill-v2-1.0
Testing out importing the model ...
Error: unable to import the module; possible reasons are:
- incompatible version of gcc/gfortran used.
- the model uses the udmget set of routines which are currently
unsupported
- missing a library or include directory
# convert_xspec_user_model (21 April 2023): ERROR Unable to import model:
${ASCDS_OTS}/lib/python3.9/site-packages/relxill_v2/_models.cpython-39-x86_64-linux-gnu.so: undefined symbol: fftw_plan_dft_c2r_1d
The initial attempt to build the model fails with an error associated with a fftw-related library. Examining the available fftw library files installed on the system with ldconfig (on Linux):
unix% ldconfig -p | grep fftw libfftw3_threads.so.3 (libc6,x86-64) => /lib64/libfftw3_threads.so.3 libfftw3_omp.so.3 (libc6,x86-64) => /lib64/libfftw3_omp.so.3 libfftw3.so.3 (libc6,x86-64) => /lib64/libfftw3.so.3 (t)csh unix% setenv LDFLAGS "-l:libfftw3.so.3" bash/zsh unix% export LDFLAGS="-l:libfftw3.so.3" *or* (t)csh unix% setenv LDFLAGS "-lfftw3" bash/zsh unix% export LDFLAGS="-lfftw3"
we then set a linker flag in the LDFLAGS environment variable for the compiler to explicitly look for the libfftw3.so.3 file (prepended by -l:) in the usual library directory locations (specific syntax use of the '-l' flag will be dependent on the compilers/OS used; the use of '-lfftw3' is more generalized and would not necessarily find the version specific .so.3 library).
In some Linux distributions, for example Ubuntu 18.04 LTS, it may be necessary to also specify the library directories too. For example, ldconfig points to the use /usr/lib/x86_64-linux-gnu and /usr/lib on the system:
unix% ldconfig -p | grep fftw
libfftw3f_threads.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfftw3f_threads.so.3
libfftw3f_omp.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfftw3f_omp.so.3
libfftw3f.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfftw3f.so.3
libfftw3_threads.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfftw3_threads.so.3
libfftw3_omp.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfftw3_omp.so.3
libfftw3.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfftw3.so.3
bash/zsh
unix% export LDFLAGS="-l:libfftw3.so.3 -L/usr/lib/x86_64-linux-gnu -L/usr/lib"
(t)csh
unix% setenv LDFLAGS "-l:libfftw3.so.3 -L/usr/lib/x86_64-linux-gnu -L/usr/lib"
*or*
bash/zsh
unix% export LDFLAGS="-lfftw3 -L/usr/lib/x86_64-linux-gnu -L/usr/lib"
(t)csh
unix% setenv LDFLAGS "-lfftw3 -L/usr/lib/x86_64-linux-gnu -L/usr/lib"
where the -L linker flag is used to specify additional directories to look for library dependencies, in addition to the specified library using -l.
This example is done on a Red Hat Enterprise Linux 8.6 system with the C/C++ compilers from the GNU Compiler Collection (GCC) 8.5.0; however, for other operating systems and/or compilers, setting LDFLAGS/CFLAGS/CXXFLAGS/FFLAGS may be completely unnecessary. For this particular version of the relxill model and the compiler used, a lot of warnings are emitted, which can be suppressed by setting CFLAGS to use the set of flags:
- -Wno-unused-value
- -Wno-unused-variable
- -Wno-array-bounds
for example,
(t)csh unix% setenv CFLAGS '-Wno-unused-value -Wno-unused-variable -Wno-array-bounds' bash/zsh unix% export CFLAGS='-Wno-unused-value -Wno-unused-variable -Wno-array-bounds'
and re-run the script to build the model, also including --clobber flag:
unix% convert_xspec_user_model relxill_v2 lmodel_relxill.dat --clobber
convert_xspec_user_model: 21 April 2023
name: relxill_v2
modelfile: lmodel_relxill.dat
clobber: True
Found no Fortran files.
Found 8 C files.
Found 13 C++ files.
Processing 12 models.
Using language interface: C style
Using XSPEC version: 12.12.1c
...
... a lot of screen output
...
Installing collected packages: relxill-v2
Attempting uninstall: relxill-v2
Found existing installation: relxill-v2 1.0
Uninstalling relxill-v2-1.0:
Successfully uninstalled relxill-v2-1.0
Successfully installed relxill-v2-1.0
Testing out importing the model ...
Import succeeded
------------------------------------------------------------------
Finished. You should be able to now say
import relxill_v2.ui
from Sherpa.
The following 12 models are available:
xsumrelline
xsumrelconv
xsumrelline_lp
xsumrelconv_lp
xsumrelxill
xsumrelxilllp
xsumxillver
xsumxillvercp
xsumrelxillcp
xsumrelxilllpcp
xsumxillverns
xsumrelxillns
pip can then be used to confirm that the package is installed into CIAO for use with Sherpa.
unix% pip list | grep relxill
relxill-v2 1.0
unix% pip show relxill-v2
Name: relxill-v2
Version: 1.0
Summary: XSPEC user models in Sherpa
Home-page:
Author:
Author-email:
License:
Location: ${ASCDS_INSTALL}/ots/lib/python3.9/site-packages
Requires:
Required-by:
In principle the compilers used to build XSpec models should be the same as that used to build the CIAO/Sherpa distribution installed on your system. The Clang/LLVM compilers that come packaged with XCode should be used to build C/C++ software for CIAO/Sherpa distributed through the binaries (e.g. via ciao-install) on MacOS and that on a typical system, gcc and g++ are linked to use clang and clang++ respectively by default. GNU C and C++ may sometimes be used successfully in lieu of Clang. Pre-built distributions of GNU Fortran compilers are also available for MacOS.
It is worth mentioning that as of relxill 2.3, the native Clang compiler has trouble building the model. There has been limited success building with the x86-64 GNU (gcc/g++) compilers into the CIAO binary distribution and buildable into the Conda CIAO distribution using the Conda compilers for MacOS.
An important step to using this collection of models is to make sure they have access to the tabulated data the models are dependent upon. The relxill documentation notes that the RELXILL_TABLE_PATH environment variable should be used to define this location. Assuming the table files are downloaded and unpacked to /path/to/directory/with/relxill/table/files, then within a Sherpa or Python environment with the models imported, the os.environ Python object should be used to internally set this environment variable:
sherpa> import relxill_v2.ui Adding additive XSPEC local model: xsumrelline Adding convolution XSPEC local model: xsumrelconv Adding additive XSPEC local model: xsumrelline_lp Adding convolution XSPEC local model: xsumrelconv_lp Adding additive XSPEC local model: xsumrelxill Adding additive XSPEC local model: xsumrelxilllp Adding additive XSPEC local model: xsumxillver Adding additive XSPEC local model: xsumxillvercp Adding additive XSPEC local model: xsumrelxillcp Adding additive XSPEC local model: xsumrelxilllpcp Adding additive XSPEC local model: xsumxillverns Adding additive XSPEC local model: xsumrelxillns sherpa> os.environ["RELXILL_TABLE_PATH"] = "/path/to/directory/with/relxill/table/files"
to set the location for the models to look in. If you would like to avoid declaring the os.environ["RELXILL_TABLE_PATH"] whenever the relxill model package is initially imported, the path can be hardcoded into the common.h file before compiling the package, changing the line (line 117 as of v2.3):
#define RELXILL_TABLE_PATH "./"
to
#define RELXILL_TABLE_PATH "/path/to/directory/with/relxill/table/files"
Compiling a user model for the Conda CIAO distribution
Compiling software compatible with a Conda environment and other Conda packages should not be anymore involved than building with the system's default compilers for the CIAO binaries, but requires a different tack.
In this example, the RELTRANS models package will be used. The file reltrans_v1.0.1.tar.gz is assumed to have been downloaded and unpacked; for example:
unix% cd /data/models/reltrans_v1 unix% tar -xzf reltrans_v1.0.1.tar.gz unix% cd reltrans_v1.0.1/ unix% ls -F amodules.f90 lmodel_relxill.dat rellp.c reltable.h revmakefile_laptop clean.xcm* load.xcm rellp.h relutility.c srevtrans.f90 common.h README.txt relmodels.c relutility.h subroutines/ compile_xspec.sh* relbase.c relmodels.h revmakefile xilltable.c donthcomp.c relbase.h reltable.c revmakefile_g xilltable.h
Out of the box no additional file modifications are needed for the the RELTRANS v1 models. Please note that the minor modifications to the unpacked relxill source code, illustrated earlier in this thread, are applicable for a Conda build too.
Setting up the Conda environment for model compilation
Packages on a Conda distribution channel are built using a standardized set of compilers and build dependenices, also available on that given channel. The Conda distribution of CIAO are compiled using the Conda compiler packages:
Linux: gcc, gxx, gfortran MacOS: clang, clangxx, gfortran
Since the Conda distribution of CIAO was built using the compilers from the conda-forge channel, we install them into the environment:
Linux: unix% conda install -n ciao-4.15 -c conda-forge gcc gxx gfortran MacOS: unix% conda install -n ciao-4.15 -c conda-forge clang clangxx gfortran
where the Conda environment with CIAO installed is called 'ciao-4.15' in this example. The respective compiler environment variables are also set:
unix% conda activate ciao-4.15
Linux:
(t)csh
(ciao-4.15) unix% setenv CC ${CONDA_PREFIX}/bin/gcc
(ciao-4.15) unix% setenv CXX ${CONDA_PREFIX}/bin/g++
(ciao-4.15) unix% setenv FC ${CONDA_PREFIX}/bin/gfortran
bash/zsh
(ciao-4.15) unix% export CC=${CONDA_PREFIX}/bin/gcc
(ciao-4.15) unix% export CXX=${CONDA_PREFIX}/bin/g++
(ciao-4.15) unix% export FC=${CONDA_PREFIX}/bin/gfortran
MacOS:
(t)csh
(ciao-4.15) unix% setenv CC ${CONDA_PREFIX}/bin/clang
(ciao-4.15) unix% setenv CXX ${CONDA_PREFIX}/bin/clang++
(ciao-4.15) unix% setenv FC ${CONDA_PREFIX}/bin/gfortran
bash/zsh
(ciao-4.15) unix% export CC=${CONDA_PREFIX}/bin/clang
(ciao-4.15) unix% export CXX=${CONDA_PREFIX}/bin/clang++
(ciao-4.15) unix% export FC=${CONDA_PREFIX}/bin/gfortran
and the C compiler flag specifying the CIAO C header files might also be needed in this example:
(t)csh
(ciao-4.15) unix% setenv CFLAGS "-I${ASCDS_OTS}/include"
bash/zsh
(ciao-4.15) unix% export CFLAGS="-I${ASCDS_OTS}/include"
Note that depending on the model package, defining every compiler environment variable (e.g. CC, CXX, FC) may be unnecessary and even detrimental.
Compile the model
(ciao-4.15) unix% convert_xspec_user_model reltrans_v1 lmodel_relxill.dat --clobber
convert_xspec_user_model: 21 April 2023
name: reltrans_v1
modelfile: lmodel_relxill.dat
clobber: True
Found 2 Fortran files.
Found 7 C files.
Found no C++ files.
Processing 15 models.
Using language interfaces: C style, Fortran - single precision
WARNING: reltrans needs to be re-calculated per spectrum; this is untested.
WARNING: reltransCp needs to be re-calculated per spectrum; this is untested.
Using XSPEC version: 12.12.1c
..............................
... a lot of screen output ...
..............................
Successfully built reltrans-v1
Installing collected packages: reltrans-v1
Successfully installed reltrans-v1-1.0
Testing out importing the model ...
Import succeeded
------------------------------------------------------------------
Finished. You should be able to now say
import reltrans_v1.ui
from Sherpa.
The following 15 models are available:
xsumrelline
xsumrelconv
xsumrelline_lp
xsumrelconv_lp
xsumrelxill
xsumrelxilllp
xsumxillver
xsumrelxilld
xsumrelxilllpd
xsumxillverd
xsumrelxillcp
xsumrelxilllpcp
xsumxillvercp
xsumreltrans
xsumreltranscp
Please note the following 2 problems:
model reltrans needs to be re-calculated per spectrum; this is untested.
model reltransCp needs to be re-calculated per spectrum; this is untested.
If the script gets this far—that is, the model can be imported and a message listing the available model, or models, is displayed—then the model has compiled and you can check it out in Sherpa. Otherwise, follow on to the errors section.
What to do if the build fails
There are a number of reasons why the build might fail; please contact the CXC Helpdesk if you are unable to solve the problem. The --clobber flag may be needed if convert_xspec_user_model is called multiple times, and sometimes directly deleting the src and build sub-directories generated by the script in the directory with the model source code may clear things up.
-
In CIAO 4.8, Only additive and multiplicative models are supported. Convolution model support was added in CIAO contributed scripts package 4.9.2 and in CIAO 4.13, XSpec convolution models are directly supported by Sherpa.
-
Support for models that require the -udmget or -udmget64 flag when calling XSpec's initpackage command was added in the CIAO contributed scripts package 4.15.2 using the --udmget or --udmget64, respectively.
-
There can be problems if your version of gcc and gfortran is much different than the version used to build CIAO 4.15 (GCC version 7.4.0 for the binaries and 12.1.0 for Conda on Linux CentOS 7.9 and Clang 11.0.3/gfortran 11.2.0 for the binaries and Clang 14.0.4/gfortran 11.3.0 for Conda on MacOS 1.15). This can lead to link errors such as:
undefined symbol: _gfortran_copy_string
-
The script does not guarantee to compile files in the same order as XSpec does, so if an ordering is required, then the build may fail.
-
If there are link or build issues, then the flags --no-xspec and --cfitsio may be useful: --no-xspec means that no XSpec libraries are linked to and --cfitsio can be added if the model only uses the CFITSIO routines.
There may also be a version conflict between the XSpec version needed by the model and that provided by Sherpa, which uses version 12.12.1c in CIAO 4.15.
-
If there is a problem when building the interface the resulting error message is unlikely to be useful. In this case try running
unix% pip install . --verbose
as it should provide more information.
Possible problems are models that refer to XSPEC code from older (or newer) versions of XSPEC than used by CIAO, as well as access to certain parts of the XSPEC build system. Some of these can be worked around—such as removing the problematic models or tweaking how the extension module is built in setup.py.
Testing the model in Sherpa
Once the model has been compiled, it can be used from Sherpa. For the relxill model, we can try it without any data or response information using the following. First, load in the module and set the RELXILL_TABLE_PATH environment variable /path/to/directory/with/relxill/table/files (replace with the location of your model's tabulated data dependency):
sherpa> import relxill_v2.ui Adding additive XSPEC local model: xsumrelline Adding convolution XSPEC local model: xsumrelconv Adding additive XSPEC local model: xsumrelline_lp Adding convolution XSPEC local model: xsumrelconv_lp Adding additive XSPEC local model: xsumrelxill Adding additive XSPEC local model: xsumrelxilllp Adding additive XSPEC local model: xsumxillver Adding additive XSPEC local model: xsumxillvercp Adding additive XSPEC local model: xsumrelxillcp Adding additive XSPEC local model: xsumrelxilllpcp Adding additive XSPEC local model: xsumxillverns Adding additive XSPEC local model: xsumrelxillns sherpa> os.environ["RELXILL_TABLE_PATH"] = "/path/to/directory/with/relxill/table/files"
Now we create a simple data space over which the model can be evaluated and the model expression (note that the user model used in the same manner as the built-in models). The default 'xsum' prefix of the model names is shorthand for "XSpec user model"; the "--prefix" flag can be used to alter the prefix, taking care to avoid using the lower case 'xs' since it is already used for the standard set of XSpec models in Sherpa and risk the possibility of overwriting the models if it has the same name. The xsumrelconv convolution model will be used in this example:
sherpa> dataspace1d(0.1, 10, step=0.01, id=1, dstype=Data1DInt) sherpa> set_source(xsumrelconv.rel1(const1d.c0)) sherpa> show_source() Model: 1 xsumrelconv.rel1(const1d.c0) Param Type Value Min Max Units ----- ---- ----- --- --- ----- rel1.Index1 frozen 3 -10 10 rel1.Index2 frozen 3 -10 10 rel1.Rbr frozen 15 1 1000 rel1.a thawed 0.998 -0.998 0.998 rel1.Incl frozen 30 3 87 deg rel1.Rin frozen -1 -100 -1 rel1.Rout frozen 400 1 1000 rel1.limb frozen 0 0 2 c0.c0 thawed 1 -3.40282e+38 3.40282e+38
The parameter names specified in the XSpec "model.dat" file are not always valid for Python, so a transformation is made, converting unusable characters—such as + and /—into the underscore character (_).
Two versions of the model are now plotted on the same graph; first for the default parameter values and then after changing the inclination angle, Incl, parameter, which creates Figure 1:
sherpa> plot_source() *** loading RELXILL model (version 2.3) *** sherpa> rel1.Incl = 80 sherpa> plot_source(overplot=True) sherpa> plt.loglog()
Figure 1: Comparison of two model plots
![[comparison of the relconv model convolving a constant model component by altering the inclination angle.]](compare-curves.png)
Figure 1: Comparison of two model plots
The blue line shows the model using the default parameters, whereas the orange line shows the model after changing the Incl parameter to 80 degrees.
Since there is no response information here—as no RMF has been loaded—then the axes have no units.
Many models require, or allow, external configuration. The set_xsxset command is the equivalent of XSpec's xset command. As an example:
sherpa> set_xsxset('CARBATM', '/data/models/data/carbatm')
sets the XSpec CARBATM variable.
The equivalent location to XSpec's spectral/ directory is ${ASCDS_INSTALL}/ots/spectral/.
Other models will access data dependencies through a specified shell environment variable, which may be set in Sherpa using the os.environ Python object, as demonstrated earlier in this thread,
sherpa> os.environ["RELXILL_TABLE_PATH"] = "/path/to/directory/with/relxill/table/files"
Use of set_xsxset versus os.environ will be completely model dependent, which should be informed based on the specific model's user-documentation.
Example: a model that successfully compiles to completion but fails at execution
The WARMABS Fortran model readily compiles on Conda without setting any specific compiler or linker flags, but is problematic when building it against the CIAO 4.15 Linux binary distribution on a RHEL 8 system and its native compilers (GCC 8.5.0).
In this example, (t)csh syntax is used to set environment variables—which can be easily substituted with the equivalent export call in bash/zsh—and the WARMABS model package is assumed to have been downloaded and unpacked,
unix% mkdir warmabs_v247 unix% tar -xzf warmabs247.tar.gz -C warmabs_v247 unix% ls -F warmabs_v247 atdbwarmabs.fits PARAMX xo01_detah4.fits xo01_detal3.fits coheatwarmabs.dat README xo01_detahl.fits xo01_detal4.fits fphotems.f90 xo01_detah2.fits xo01_detail.fits lmodel.dat xo01_detah3.fits xo01_detal2.fits
For organizational purposes, the data tables used by the models are separated from the source code and moved to a separate directory:
unix% ls -F /path/to/warmabs_v2.4x/data/files atdbwarmabs.fits xo01_detah3.fits xo01_detail.fits xo01_detal4.fits coheatwarmabs.dat xo01_detah4.fits xo01_detal2.fits xo01_detah2.fits xo01_detahl.fits xo01_detal3.fits
The model package will be built with the module name 'warmabs_v247' herein. It seemingly compiles successfully using convert_xspec_user_model with no build failure messages but an error is thrown upon using the model with Sherpa:
sherpa> import warmabs_v247.ui Adding additive XSPEC local model: xsumhotemis Adding multiplicative XSPEC local model: xsumhotabs Adding additive XSPEC local model: xsumphotemis Adding multiplicative XSPEC local model: xsumwarmabs Adding multiplicative XSPEC local model: xsummultabs sherpa> os.environ["WARMABS_DATA"] = "/path/to/warmabs_v2.4x/data/files" sherpa> dataspace1d(0.1, 10, step=0.01, id=1, dstype=Data1DInt) sherpa> set_source(xsumwarmabs.abs1 * xspowerlaw.p1) sherpa> show_model() Model: 1 (xsumwarmabs.abs1 * xspowerlaw.p1) Param Type Value Min Max Units ----- ---- ----- --- --- ----- abs1.column thawed 0 -3 2 cm^-2 abs1.rlogxi thawed 0 -4 5 abs1.Cabund frozen 1 0 1000 abs1.Nabund frozen 1 0 1000 abs1.Oabund frozen 1 0 1000 abs1.Fabund frozen 1 0 1000 abs1.Neabund frozen 1 0 1000 abs1.Naabund frozen 1 0 1000 abs1.Mgabund frozen 1 0 1000 abs1.Alabund frozen 1 0 1000 abs1.Siabund frozen 1 0 1000 abs1.Pabund frozen 1 0 1000 abs1.Sabund frozen 1 0 1000 abs1.Clabund frozen 1 0 1000 abs1.Arabund frozen 1 0 1000 abs1.Kabund frozen 1 0 1000 abs1.Caabund frozen 1 0 1000 abs1.Scabund frozen 1 0 1000 abs1.Tiabund frozen 1 0 1000 abs1.Vabund frozen 1 0 1000 abs1.Crabund frozen 1 0 1000 abs1.Mnabund frozen 1 0 1000 abs1.Feabund frozen 1 0 1000 abs1.Coabund frozen 1 0 1000 abs1.Niabund frozen 1 0 1000 abs1.Cuabund frozen 1 0 1000 abs1.Znabund frozen 1 0 1000 abs1.write_outfile frozen 0 0 1 abs1.outfile_idx frozen 0 0 10000 abs1.vturb frozen 0 0 10000 km/s abs1.Redshift frozen 0 0 10 p1.PhoIndex thawed 1 -3 10 p1.norm thawed 1 0 1e+24 sherpa> plot_source() doing setup: 0 1 0 0 0 photemis/warmabs v2.47d Fortran runtime error: Insufficient number of elements in TARRAY.
Taking a quick look at the library dependencies of the built module with ldd
unix% ldd build/lib.linux-x86_64-cpython-39/warmabs_v247/_models.cpython-39-x86_64-linux-gnu.so linux-vdso.so.1 (0x00007ffdc97db000) libXSModel.so => not found libXSFunctions.so => not found libXSUtil.so => not found libXS.so => not found libhdsp_6.30.so => not found libCCfits_2.6.so => not found libcfitsio.so.9 => not found libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fca97e26000) libm.so.6 => /lib64/libm.so.6 (0x00007fca97aa4000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fca9788c000) libc.so.6 => /lib64/libc.so.6 (0x00007fca974c7000) /lib64/ld-linux-x86-64.so.2 (0x00007fcad9d7e000)
which is found in the temporary 'build' sub-directory that convert_xspec_user_model generates in the unpacked source code directory.
All of the "not found" libraries are CIAO OTS libraries (which upon further inspection with the 'ldd -u' flag shows they are ultimately unused by the model). In this case, the linker flag is set before attempting to do another re-build of the package:
sherpa> setenv LDFLAGS -Wl,-rpath,${ASCDS_OTS}/lib
sherpa> convert_xspec_user_model warmabs_v247 lmodel.dat --clobber
..............................
... a lot of screen output ...
..............................
unix% ldd build/lib.linux-x86_64-cpython-39/warmabs_v247/_models.cpython-39-x86_64-linux-gnu.so
linux-vdso.so.1 (0x00007ffd5ccd2000)
libXSModel.so => ${ASCDS_OTS}/lib/libXSModel.so (0x00007f05fb703000)
libXSFunctions.so => ${ASCDS_OTS}/lib/libXSFunctions.so (0x00007f05f9d5f000)
libXSUtil.so => ${ASCDS_OTS}/lib/libXSUtil.so (0x00007f05f98eb000)
libXS.so => ${ASCDS_OTS}/lib/libXS.so (0x00007f05f96cc000)
libhdsp_6.30.so => ${ASCDS_OTS}/lib/libhdsp_6.30.so (0x00007f05f92b4000)
libCCfits_2.6.so => ${ASCDS_OTS}/lib/libCCfits_2.6.so (0x00007f05f8e47000)
libcfitsio.so.9 => ${ASCDS_OTS}/lib/libcfitsio.so.9 (0x00007f05f8943000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f05f85ae000)
libm.so.6 => /lib64/libm.so.6 (0x00007f05f822c000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f05f8014000)
libc.so.6 => /lib64/libc.so.6 (0x00007f05f7c4f000)
libtinfo.so.5 => ${ASCDS_OTS}/lib/../lib/libtinfo.so.5 (0x00007f05f7a1c000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f05f7818000)
libgfortran.so.4 => ${ASCDS_OTS}/lib/../lib/libgfortran.so.4 (0x00007f05f72fe000)
libquadmath.so.0 => ${ASCDS_OTS}/lib/../lib/libquadmath.so.0 (0x00007f05f70b7000)
libwcs-7.7.so => ${ASCDS_OTS}/lib/../lib/libwcs-7.7.so (0x00007f05f6d24000)
libhdutils_6.30.so => ${ASCDS_OTS}/lib/../lib/libhdutils_6.30.so (0x00007f05f6a1f000)
libfgsl.so.1 => ${ASCDS_OTS}/lib/../lib/libfgsl.so.1 (0x00007f05f676c000)
libgsl.so.25 => ${ASCDS_OTS}/lib/../lib/libgsl.so.25 (0x00007f05f61f9000)
libgslcblas.so.0 => ${ASCDS_OTS}/lib/../lib/libgslcblas.so.0 (0x00007f05f5fa6000)
libreadline.so.8 => ${ASCDS_OTS}/lib/../lib/libreadline.so.8 (0x00007f05f5d4e000)
libz.so.1 => ${ASCDS_OTS}/lib/../lib/libz.so.1 (0x00007f05f5b30000)
/lib64/ld-linux-x86-64.so.2 (0x00007f063da90000)
libncurses.so.5 => ${ASCDS_OTS}/lib/../lib/../lib/libncurses.so.5 (0x00007f05f5902000)
unix% ldd -u build/lib.linux-x86_64-cpython-39/warmabs_delme/_models.cpython-39-x86_64-linux-gnu.so
Unused direct dependencies:
${ASCDS_OTS}/lib/libXSModel.so
${ASCDS_OTS}/lib/libXSFunctions.so
${ASCDS_OTS}/lib/libXSUtil.so
${ASCDS_OTS}/lib/libXS.so
${ASCDS_OTS}/lib/libhdsp_6.30.so
${ASCDS_OTS}/lib/libCCfits_2.6.so
${ASCDS_OTS}/lib/libcfitsio.so.9
/lib64/libm.so.6
/lib64/libgcc_s.so.1
/lib64/libpthread.so.0
unix% ldd -v build/lib.linux-x86_64-cpython-39/warmabs_delme/_models.cpython-39-x86_64-linux-gnu.so | grep fortran
libgfortran.so.4 => ${ASCDS_OTS}/lib/../lib/libgfortran.so.4 (0x00007fd10738b000)
libgfortran.so.4 (GFORTRAN_7) => ${ASCDS_OTS}/lib/../lib/libgfortran.so.4
libgfortran.so.4 (GFORTRAN_7) => ${ASCDS_OTS}/lib/../lib/libgfortran.so.4
${ASCDS_OTS}/lib/../lib/libgfortran.so.4:
libgfortran.so.4 (GFORTRAN_7) => ${ASCDS_OTS}/lib/../lib/libgfortran.so.4
Since the CIAO 4.15 Linux binaries were built using GCC 7.9 and the ligbfortran.so.4 library, there are incompatibilities with the version of GCC 8.x that comes with the OS, which does not support compiler backwards compatiblity. Using ldconfig to find the libgfortran that comes with the operating system,
unix% ldconfig -p |grep gfortran libgfortran.so.5 (libc6,x86-64) => /lib64/libgfortran.so.5
the linker flag is then modified to use the libgfortran.so.5 version of the library that is compatible with the compiler, and then attempt another build of the models:
unix% setenv LDFLAGS "-l:libgfortran.so.5 -Wl,-rpath,/lib64 -Wl,-rpath,${ASCDS_OTS}/lib"
sherpa> convert_xspec_user_model warmabs_v247 lmodel.dat --clobber
..............................
... a lot of screen output ...
..............................
Installing collected packages: warmabs-v247
Successfully installed warmabs-v247-1.0
Testing out importing the model ...
Import succeeded
------------------------------------------------------------------
Finished. You should be able to now say
import warmabs_v247.ui
from Sherpa.
The following 5 models are available:
xsumhotemis
xsumhotabs
xsumphotemis
xsumwarmabs
xsummultabs
and once again, the model seems to successfully compile. Trying to use the model, strangely, it will seemingly run, and sometimes will run to completion with the model displayed,
sherpa> import warmabs_v247.ui
Adding additive XSPEC local model: xsumhotemis
Adding multiplicative XSPEC local model: xsumhotabs
Adding additive XSPEC local model: xsumphotemis
Adding multiplicative XSPEC local model: xsumwarmabs
Adding multiplicative XSPEC local model: xsummultabs
sherpa> os.environ["WARMABS_DATA"] = "/path/to/warmabs_v2.4x/data/files"
sherpa> dataspace1d(0.1, 10, step=0.01, id=1, dstype=Data1DInt)
sherpa> set_source(xsumwarmabs.abs1 * xspowerlaw.p1)
sherpa> plot_source()
doing setup: 0 1 0 0 0
photemis/warmabs v2.47d
xstar version 2.58e
Loading Atomic Database...
Loading Atomic Database...
Atomic Data Version: 2021-08-03T18:36:21
number of pointers= 1216828
number of reals= 199199716
number of integers= 6205342
number of chars= 753844
initializng database...
initializng database...
number of lines= 736256
number of rrcs= 301301
max levels per ion: 744
habund 0.1000E+01 0.1000E+01
heabund 0.1000E+01 0.1000E+00
liabund 0.0000E+00 0.0000E+00
beabund 0.0000E+00 0.0000E+00
babund 0.0000E+00 0.0000E+00
cabund 0.1000E+01 0.3700E-03
nabund 0.1000E+01 0.1100E-03
oabund 0.1000E+01 0.6800E-03
fabund 0.1000E+01 0.3980E-07
neabund 0.1000E+01 0.2800E-04
naabund 0.1000E+01 0.1780E-05
mgabund 0.1000E+01 0.3500E-04
alabund 0.1000E+01 0.2450E-05
siabund 0.1000E+01 0.3500E-04
pabund 0.1000E+01 0.3310E-06
sabund 0.1000E+01 0.1600E-04
clabund 0.1000E+01 0.3980E-06
arabund 0.1000E+01 0.4500E-05
kabund 0.1000E+01 0.8910E-07
caabund 0.1000E+01 0.2100E-05
scabund 0.1000E+01 0.1660E-08
tiabund 0.1000E+01 0.1350E-06
vabund 0.1000E+01 0.2510E-07
crabund 0.1000E+01 0.7080E-06
mnabund 0.1000E+01 0.2510E-06
feabund 0.1000E+01 0.2500E-04
coabund 0.1000E+01 0.1260E-06
niabund 0.1000E+01 0.2000E-05
cuabund 0.1000E+01 0.3160E-07
znabund 0.1000E+01 0.1580E-07
Reading pop files
using files xout_detail.fits etc. 219
constructing energy grid 9999
constructing energy grid 7571 6377
${ASCDS_INSTALL}/binexe/sherpa: line 187: 2822848 Segmentation fault (core dumped) ipython --profile sherpa
but the model execution is inconsistent, and more often than not, a segmentation fault is thrown with a core dump.
Exploring the various GFortran compiler flags for code generation,
unix% setenv FFLAGS -fcheck=all # insert runtime error checking ... At line 9599 of file src/_xspeclmodels/fphotems.f90 Fortran runtime error: Index '-1' of dimension 1 of array 'nlbin' below lower bound of 1
setting -fmax-stack-var-size=10 (or using the -fno-automatic flag) to avoid a stack overflow can work around the error by avoiding to write variables/arrays to stack memory (at the expense of a minor performance penalty for the flexibility offered by writing to heap memory).
unix% setenv FFLAGS -fmax-stack-var-size=10
unix% convert_xspec_user_model warmabs_v247 lmodel.dat
convert_xspec_user_model: 21 April 2023
name: warmabs_v247
modelfile: lmodel.dat
clobber: False
Found one Fortran file.
Found no C files.
Found no C++ files.
Processing 5 models.
Using language interface: Fortran - single precision
Using XSPEC version: 12.12.1c
...
... a lot of screen output
...
Successfully built warmabs-v247
Installing collected packages: warmabs-v247
Successfully installed warmabs-v247-1.0
Testing out importing the model ...
Import succeeded
------------------------------------------------------------------
Finished. You should be able to now say
import warmabs_v247.ui
from Sherpa.
The following 5 models are available:
xsumhotemis
xsumhotabs
xsumphotemis
xsumwarmabs
xsummultabs
Testing out the multiplcative xsumwarmabs absorption model in Sherpa with a constant model component of unity, we see that the model plots without an error being thrown:
sherpa> import warmabs_v247.ui Adding additive XSPEC local model: xsumhotemis Adding multiplicative XSPEC local model: xsumhotabs Adding additive XSPEC local model: xsumphotemis Adding multiplicative XSPEC local model: xsumwarmabs Adding multiplicative XSPEC local model: xsummultabs sherpa> os.environ["WARMABS_DATA"] = "/path/to/warmabs_v2.4x/data/files" sherpa> dataspace1d(0.1, 10, step=0.01, id=1, dstype=Data1DInt) sherpa> set_source(xsumwarmabs.abs1 * xspowerlaw.p1) sherpa> p1.PhoIndex = 0 sherpa> show_source() Model: 1 (xsumwarmabs.abs1 * xspowerlaw.p1) Param Type Value Min Max Units ----- ---- ----- --- --- ----- abs1.column thawed 0 -3 2 cm^-2 abs1.rlogxi thawed 0 -4 5 abs1.Cabund frozen 1 0 1000 abs1.Nabund frozen 1 0 1000 abs1.Oabund frozen 1 0 1000 abs1.Fabund frozen 1 0 1000 abs1.Neabund frozen 1 0 1000 abs1.Naabund frozen 1 0 1000 abs1.Mgabund frozen 1 0 1000 abs1.Alabund frozen 1 0 1000 abs1.Siabund frozen 1 0 1000 abs1.Pabund frozen 1 0 1000 abs1.Sabund frozen 1 0 1000 abs1.Clabund frozen 1 0 1000 abs1.Arabund frozen 1 0 1000 abs1.Kabund frozen 1 0 1000 abs1.Caabund frozen 1 0 1000 abs1.Scabund frozen 1 0 1000 abs1.Tiabund frozen 1 0 1000 abs1.Vabund frozen 1 0 1000 abs1.Crabund frozen 1 0 1000 abs1.Mnabund frozen 1 0 1000 abs1.Feabund frozen 1 0 1000 abs1.Coabund frozen 1 0 1000 abs1.Niabund frozen 1 0 1000 abs1.Cuabund frozen 1 0 1000 abs1.Znabund frozen 1 0 1000 abs1.write_outfile frozen 0 0 1 abs1.outfile_idx frozen 0 0 10000 abs1.vturb frozen 0 0 10000 km/s abs1.Redshift frozen 0 0 10 p1.PhoIndex thawed 0 -3 10 p1.norm thawed 1 0 1e+24 sherpa> plot_source() doing setup: 0 1 0 0 0 photemis/warmabs v2.47d xstar version 2.58e Loading Atomic Database... Loading Atomic Database... Atomic Data Version: 2021-08-03T18:36:21 number of pointers= 1216828 number of reals= 199199716 number of integers= 6205342 number of chars= 753844 initializng database... initializng database... number of lines= 736256 number of rrcs= 301301 max levels per ion: 744 habund 0.1000E+01 0.1000E+01 heabund 0.1000E+01 0.1000E+00 liabund 0.0000E+00 0.0000E+00 beabund 0.0000E+00 0.0000E+00 babund 0.0000E+00 0.0000E+00 cabund 0.1000E+01 0.3700E-03 nabund 0.1000E+01 0.1100E-03 oabund 0.1000E+01 0.6800E-03 fabund 0.1000E+01 0.3980E-07 neabund 0.1000E+01 0.2800E-04 naabund 0.1000E+01 0.1780E-05 mgabund 0.1000E+01 0.3500E-04 alabund 0.1000E+01 0.2450E-05 siabund 0.1000E+01 0.3500E-04 pabund 0.1000E+01 0.3310E-06 sabund 0.1000E+01 0.1600E-04 clabund 0.1000E+01 0.3980E-06 arabund 0.1000E+01 0.4500E-05 kabund 0.1000E+01 0.8910E-07 caabund 0.1000E+01 0.2100E-05 scabund 0.1000E+01 0.1660E-08 tiabund 0.1000E+01 0.1350E-06 vabund 0.1000E+01 0.2510E-07 crabund 0.1000E+01 0.7080E-06 mnabund 0.1000E+01 0.2510E-06 feabund 0.1000E+01 0.2500E-04 coabund 0.1000E+01 0.1260E-06 niabund 0.1000E+01 0.2000E-05 cuabund 0.1000E+01 0.3160E-07 znabund 0.1000E+01 0.1580E-07 Reading pop files using files xout_detail.fits etc. 219 constructing energy grid 9999 constructing energy grid 7571 6377
Figure 2: warmabs curve
![[plot of the warmabs absorption model with default settings]](warmabs247.png)
Figure 2: warmabs curve
Plot of the [xsum]warmabs warm absorber model, expressed by multiplying the absorption component with a constant model component expressed as a powerlaw with photon index of 0.
History
| 30 Sep 2014 | original version |
| 15 Dec 2014 | Updated for CIAO 4.7 |
| 29 Feb 2016 | Updated for CIAO 4.8 and contributed scripts 4.8.2 |
| 25 Jan 2017 | Updated for CIAO 4.9; convolution models are still not supported. |
| 13 Jul 2018 | Updated for CIAO 4.10; convolution models supported again. |
| 22 Aug 2023 | Rewritten for CIAO 4.15; examples using binaries and Conda distribution of CIAO on Linux and MacOS platforms with different user models. |