Ska Runtime Environment 0.11

Content changes overview

• num_libs_bin: new binary install of “num_libs.cfg”. Using procedure in NOTES.make_num_libs_bin, create a package with arch-dependent binaries for num_libs compiled from source (which can take a long time). Do this on the target host if possible. (Building num_libs on VMware Centos-5 VM takes at least 2 hours and is completely the wrong CPU).
• NumPy 1.5 source install
• SciPy 0.9 source install
• New pycore.cfg which installs numpy, scipy, matplotlib from source.

Review

Notes and testing were reviewed by Jean Connelly. Command states and module testing were run independently by Jean.

Build

Install (or git pull) skare on 32-bit or 64-bit virtual CentOS-5 machine.

# Get skare repository on virtual CentOS-5 machine
cd ~/git

git clone git@github.com:sot/skare.git
git checkout master

# Choose prefix (dev or flight) and configure
prefix=/proj/sot/ska/dev
prefix=/proj/sot/ska
./configure --prefix=$prefix

# Choose 64 or 32-bit installation
make basedirs libs python_64 git meld sherpa
make basedirs libs python_32 git meld sherpa

# Create tarfile output (dev or flight) [replace 64 -> 32 as needed]
cd $prefix
tar zcf skare-0.11-64-dev.tar.gz arch bin lib build/*/*/.installed
tar zcf skare-0.11-64.tar.gz arch bin lib build/*/*/.installed
scp skare-0.11-64.tar.gz ccosmos:/proj/sot/ska/dist/

Pre-install testing in development area

Starcheck

skadev
cd ~/hg/starcheck
(unska; /proj/sot/ska/bin/starcheck -dir AUG0104A -fid_char fid_CHARACTERIS_JUL01 -out test.7cb31b)
make test
diff test.7cb31b.txt test.txt

==> OK

Eng_archive

cd ~/hg
hg clone /proj/sot/ska/hg/eng_archive
cd eng_archive

Follow the steps for “Regression test for new skare (..) in $ska/dev” in NOTES.test.

==> OK

Commanded states

skadev
cd ~/hg/Chandra.cmd_states_dev
python setup.py install
cd ~/hg/cmd_states_dev
make install
cd ~/hg/timelines_dev
make install

nosetests

==>

cd ~/hg/Chandra.cmd_states
python setup.py install
cd ~/hg/cmd_states
make install
cd ~/hg/timelines
make install

nosetests

==>

psmc_check

skadev
cd ~/hg/psmc
ln -s $ska/data/telem_archive $ska/dev/data/
make install
python ./psmc_check.py --run_start_time='2011:001' --outdir regress_skadev

# NEW WINDOW
ska
python ./psmc_check.py --run_start_time='2011:001' --outdir regress_ska

diff regress_ska{,dev}/validation_quant.csv
rm $ska/dev/data/telem_archive

==> OK

Other modules

• Ska.Table: OK (with patch). Updated because recent pyfits throws IOError, not caught properly. This was true of Flight Ska, just did not run tests for pyfits 2.2.2 install.
• Ska.DBI: OK (need to run as aca and link to aspect_authorization)
• Quaternion: OK
• Ska.ftp: OK
• Ska.Numpy: OK
• Ska.ParseCM: OK
• Ska.quatutil: OK
• Ska.Shell: OK
• asciitable: OK

Installation on HEAD network

Copy the skare tar distribution skare-0.11-64.tar.gz to /proj/sot/ska/dist.

# Do everything as aca
su -l aca

# Stop all cron jobs
touch /proj/sot/ska/data/task_schedule/master_heart_attack

cd /proj/sot/ska/dist
mkdir skare-0.11-64
cd skare-0.11-64
tar xf ../skare-0.11-64.tar.gz

# preview updates in key areas
rsync --size-only --dry-run -av bin/ /proj/sot/ska/bin/
rsync --size-only --dry-run -av lib/ /proj/sot/ska/lib/
# NO CHANGES

ls        /proj/sot/ska/dist/skare-0.11-64/
mv /proj/sot/ska/build/x86_64-linux_CentOS-5{,.bak}
mv /proj/sot/ska/arch/x86_64-linux_CentOS-5{,.bak}

cd /proj/sot/ska/dist
rsync -av /proj/sot/ska/dist/skare-0.11-64/ /proj/sot/ska/ >& install-0.11-64.log

# Make modules that cannot be made on virtual machine, e.g. Sybase, and
# ensure completeness.
# First clone the skare installer repo, then
cd /proj/sot/ska/dist
(ska; git clone ~aldcroft/git/skare)
cd skare
git branch # confirm correct branch
./configure --prefix=/proj/sot/ska
make python_modules

# TEST per instructions below

# Allow all cron jobs to resume
rm /proj/sot/ska/data/task_schedule/master_heart_attack

Installation on GRETA network

Perform GRETA network installation after a soak period of about one week on the HEAD network. Start by copying the skare tar distribution skare-0.11-32.tar.gz to /proj/sot/ska/dist on the GRETA network. Then do the installation steps:

# Do everything as aca
su -l aca

# Stop all cron jobs
crontab -e

cd /proj/sot/ska/dist
mkdir skare-0.11-32
cd skare-0.11-32
tar xf ../skare-0.11-32.tar.gz

# preview updates in key areas
rsync --size-only --dry-run -av bin/ /proj/sot/ska/bin/
rsync --size-only --dry-run -av lib/ /proj/sot/ska/lib/

mv /proj/sot/ska/build{,.bak}
ls /proj/sot/ska/dist/skare-0.11-32/

# DO THE INSTALL
rsync -av /proj/sot/ska/dist/skare-0.11-32/ /proj/sot/ska/ >& install.log

# Make modules that cannot be made on virtual machine, e.g. Sybase, and
# ensure completeness.
make python_modules

# TEST per instructions below (as applicable for GRETA)

# Allow all cron jobs to resume
crontab -e

Fallback

Only three files are expected to changed (the rest go in a new arch directory):

• bin/sysarch
• bin/syspathsubst
• lib/perl/CXC/Envs/Flight.pm.

Confirm with:

rsync --dry-run -av /proj/sot/.snapshot/nightly.0/ska/bin/ /proj/sot/ska/bin/
rsync --dry-run -av /proj/sot/.snapshot/nightly.0/ska/lib/ /proj/sot/ska/lib/

Restore with:

cp -p /proj/sot/.snapshot/nightly.0/ska/bin/sysarch /proj/sot/ska/bin/
cp -p /proj/sot/.snapshot/nightly.0/ska/bin/syspathsubst /proj/sot/ska/bin/
cp -p /proj/sot/.snapshot/nightly.0/ska/lib/perl/CXC/Envs/Flight.pm /proj/sot/ska/lib/perl/CXC/Envs/
rm -rf /proj/sot/ska/build
mv /proj/sot/ska/build{.bak,}

Post-install testing

Starcheck

cd ~/hg/starcheck
/proj/sot/ska/bin/starcheck -dir AUG0104A -fid_char fid_CHARACTERIS_JUL01 -out test.new
diff test.7cb31b.txt test.new.txt

==> OK

Eng_archive

cd ~/hg
hg clone /proj/sot/ska/hg/eng_archive
cd eng_archive

Follow the steps for “Regression test for new skare in /proj/sot/ska” in NOTES.test.

==> OK

Commanded states

cd ~/hg/timelines
nosetests

==> PENDING...

psmc_check

ska
cd ~/hg/psmc
python ./psmc_check.py --run_start_time='2011:001' --outdir regress_ska.new

diff regress_ska{.new,.7cb31b}/validation_quant.csv

==> OK

Other modules

• Ska.Table: OK
• Ska.DBI: OK
• Quaternion (nose): OK
• Ska.ftp (nose): OK
• Ska.Numpy: OK
• Ska.ParseCM: OK
• Ska.quatutil: OK
• Ska.Shell: OK
• asciitable: OK