Prev   Index   Next

kalman_watch3: Sat Mar 28 08:35:09 2026

File: /proj/sot/ska/data/kalman_watch3/logs/daily.0/kalman_watch.log

Task Status AgeMaxAge Last Date
kalman_watch3 OK 0.091.0 Sat Mar 28 08:35:09 2026

No errors

File contents:

2026-03-28 08:31:36,103 log_run_info: ******************************************
2026-03-28 08:31:36,103 log_run_info: Running: /proj/sot/ska3/aca-test/lib/python3.13/site-packages/kalman_watch/kalman_perigee_mon.py
2026-03-28 08:31:36,103 log_run_info: Version: 0.6.2
2026-03-28 08:31:36,103 log_run_info: Time: Sat Mar 28 08:31:36 2026
2026-03-28 08:31:36,103 log_run_info: User: aca
2026-03-28 08:31:36,103 log_run_info: Machine: kady.cfa.harvard.edu
2026-03-28 08:31:36,103 log_run_info: Processing args:
2026-03-28 08:31:36,103 log_run_info: {'data_dir': PosixPath('/proj/sot/ska3/aca-test/www/ASPECT/kalman_watch3'),
2026-03-28 08:31:36,103 log_run_info: 'emails': ['aca_alert@cfa.harvard.edu'],
2026-03-28 08:31:36,103 log_run_info: 'lookback': 14,
2026-03-28 08:31:36,103 log_run_info: 'make_html': True,
2026-03-28 08:31:36,103 log_run_info: 'rad_data': PosixPath('/proj/sot/ska3/aca-test/data/stk/radiation_data.fits'),
2026-03-28 08:31:36,104 log_run_info: 'stop': None}
2026-03-28 08:31:36,104 log_run_info: ******************************************
2026-03-28 08:31:36,112 read_kalman_stats: Reading kalman perigee data from /proj/sot/ska3/aca-test/www/ASPECT/kalman_watch3/perigees/kalman_perigees.ecsv
2026-03-28 08:31:36,146 get_evts_perigee: Getting perigee events between 2026:073:12:31:36.104 and 2026:087:12:31:36.104
2026-03-28 08:31:40,823 _get_tlm: Getting telemetry for 2026:087:09:29:33.757
2026-03-28 08:31:41,669 get_evts_perigee: No TLM found for perigee event at 2026:087:09:29:33.757, skipping
2026-03-28 08:31:41,669 get_evts_perigee: Found 0 new perigee event(s)
2026-03-28 08:31:41,669 main: No new perigee events, exiting
2026-03-28 08:31:49,349 get_manvrs_perigee: Getting maneuvers from 2026:042:12:31:49.348 to 2026:086:12:31:49.348
2026-03-28 08:31:53,572 get_manvrs_perigee: Getting intervals of Earth blocks from 2026:042:12:31:49.348 to 2026:086:12:31:49.348
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:044:21:55:01.124 to 2026:044:22:01:47.024
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:045:05:52:19.626 to 2026:045:06:02:34.626
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:047:10:15:39.638 to 2026:047:10:19:43.588
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:051:14:43:38.087 to 2026:051:14:44:46.762
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:053:18:31:58.900 to 2026:053:18:35:36.200
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:053:22:05:40.625 to 2026:053:22:09:59.950
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:054:02:16:07.126 to 2026:054:02:22:08.951
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:055:12:33:21.735 to 2026:055:12:35:25.760
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:056:17:38:49.067 to 2026:056:17:44:45.767
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:058:00:06:02.049 to 2026:058:00:09:57.799
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:060:21:42:54.565 to 2026:060:21:57:43.240
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:063:12:13:23.080 to 2026:063:12:34:31.005
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:066:03:26:46.395 to 2026:066:04:01:47.646
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:068:22:13:35.036 to 2026:068:22:16:28.261
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:070:04:48:24.144 to 2026:070:04:53:49.069
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:071:12:51:40.701 to 2026:071:13:48:41.127
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:076:19:25:09.132 to 2026:076:20:00:25.757
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:079:10:02:30.722 to 2026:079:10:12:05.747
2026-03-28 08:32:05,171 get_manvrs_perigee: Excluding Earth block from 2026:083:17:47:18.196 to 2026:083:17:52:54.396
2026-03-28 08:32:05,172 get_manvrs_perigee: Excluding Earth block from 2026:084:18:38:35.577 to 2026:084:18:59:39.402
2026-03-28 08:32:05,544 get_mon_dataset: Getting MON data from 2026:045:01:28:17.225 to 2026:045:01:55:55.675
2026-03-28 08:32:05,545 get_aca_images_cached: Getting ACA images from 2026:045:01:28:17.225 to 2026:045:01:55:55.675
2026-03-28 08:32:05,814 get_ir_thresholds: Getting IR thresholds from 2023:100 to 2023:200
2026-03-28 08:34:26,171 get_mon_dataset: Getting MON data from 2026:047:16:22:34.590 to 2026:047:16:55:56.415
2026-03-28 08:34:26,171 get_aca_images_cached: Getting ACA images from 2026:047:16:22:34.590 to 2026:047:16:55:56.415
2026-03-28 08:34:26,453 get_mon_dataset: Getting MON data from 2026:050:07:25:17.280 to 2026:050:07:55:56.130
2026-03-28 08:34:26,453 get_aca_images_cached: Getting ACA images from 2026:050:07:25:17.280 to 2026:050:07:55:56.130
2026-03-28 08:34:26,747 get_mon_dataset: Getting MON data from 2026:052:23:29:21.770 to 2026:052:23:55:55.645
2026-03-28 08:34:26,748 get_aca_images_cached: Getting ACA images from 2026:052:23:29:21.770 to 2026:052:23:55:55.645
2026-03-28 08:34:27,269 get_mon_dataset: Getting MON data from 2026:055:15:42:40.785 to 2026:055:16:05:55.811
2026-03-28 08:34:27,269 get_aca_images_cached: Getting ACA images from 2026:055:15:42:40.785 to 2026:055:16:05:55.811
2026-03-28 08:34:27,524 get_mon_dataset: Getting MON data from 2026:058:06:19:53.150 to 2026:058:06:50:55.575
2026-03-28 08:34:27,524 get_aca_images_cached: Getting ACA images from 2026:058:06:19:53.150 to 2026:058:06:50:55.575
2026-03-28 08:34:27,748 get_mon_dataset: Getting MON data from 2026:058:07:44:59.701 to 2026:058:08:32:50.726
2026-03-28 08:34:27,748 get_aca_images_cached: Getting ACA images from 2026:058:07:44:59.701 to 2026:058:08:32:50.726
2026-03-28 08:34:28,258 get_mon_dataset: Getting MON data from 2026:060:21:37:57.315 to 2026:060:21:42:54.565
2026-03-28 08:34:28,258 get_aca_images_cached: Getting ACA images from 2026:060:21:37:57.315 to 2026:060:21:42:54.565
2026-03-28 08:34:28,431 get_mon_dataset: Getting MON data from 2026:060:21:57:43.240 to 2026:060:22:15:55.891
2026-03-28 08:34:28,431 get_aca_images_cached: Getting ACA images from 2026:060:21:57:43.240 to 2026:060:22:15:55.891
2026-03-28 08:34:28,633 get_mon_dataset: Getting MON data from 2026:063:14:22:46.431 to 2026:063:14:48:55.706
2026-03-28 08:34:28,634 get_aca_images_cached: Getting ACA images from 2026:063:14:22:46.431 to 2026:063:14:48:55.706
2026-03-28 08:34:29,115 get_mon_dataset: Getting MON data from 2026:063:15:42:59.831 to 2026:063:16:21:23.006
2026-03-28 08:34:29,115 get_aca_images_cached: Getting ACA images from 2026:063:15:42:59.831 to 2026:063:16:21:23.006
2026-03-28 08:34:29,333 get_mon_dataset: Getting MON data from 2026:066:05:58:33.521 to 2026:066:06:32:55.821
2026-03-28 08:34:29,333 get_aca_images_cached: Getting ACA images from 2026:066:05:58:33.521 to 2026:066:06:32:55.821
2026-03-28 08:34:29,559 get_mon_dataset: Getting MON data from 2026:068:19:19:24.136 to 2026:068:19:55:55.586
2026-03-28 08:34:29,559 get_aca_images_cached: Getting ACA images from 2026:068:19:19:24.136 to 2026:068:19:55:55.586
2026-03-28 08:34:29,767 get_mon_dataset: Getting MON data from 2026:068:22:05:29.186 to 2026:068:22:13:35.036
2026-03-28 08:34:29,767 get_aca_images_cached: Getting ACA images from 2026:068:22:05:29.186 to 2026:068:22:13:35.036
2026-03-28 08:34:29,951 get_mon_dataset: Getting MON data from 2026:068:22:16:28.261 to 2026:068:22:55:56.012
2026-03-28 08:34:29,951 get_aca_images_cached: Getting ACA images from 2026:068:22:16:28.261 to 2026:068:22:55:56.012
2026-03-28 08:34:30,428 get_mon_dataset: Getting MON data from 2026:071:10:36:59.601 to 2026:071:11:15:55.576
2026-03-28 08:34:30,429 get_aca_images_cached: Getting ACA images from 2026:071:10:36:59.601 to 2026:071:11:15:55.576
2026-03-28 08:34:30,906 get_mon_dataset: Getting MON data from 2026:071:12:50:00.251 to 2026:071:12:51:40.701
2026-03-28 08:34:30,906 get_aca_images_cached: Getting ACA images from 2026:071:12:50:00.251 to 2026:071:12:51:40.701
2026-03-28 08:34:31,070 get_mon_dataset: Getting MON data from 2026:071:13:48:41.127 to 2026:071:13:50:06.202
2026-03-28 08:34:31,070 get_aca_images_cached: Getting ACA images from 2026:071:13:48:41.127 to 2026:071:13:50:06.202
2026-03-28 08:34:31,229 get_mon_dataset: Getting MON data from 2026:074:03:07:42.066 to 2026:074:03:45:56.016
2026-03-28 08:34:31,229 get_aca_images_cached: Getting ACA images from 2026:074:03:07:42.066 to 2026:074:03:45:56.016
2026-03-28 08:34:31,797 get_mon_dataset: Getting MON data from 2026:076:19:11:34.257 to 2026:076:19:25:09.132
2026-03-28 08:34:31,797 get_aca_images_cached: Getting ACA images from 2026:076:19:11:34.257 to 2026:076:19:25:09.132
2026-03-28 08:34:32,010 get_mon_dataset: Getting MON data from 2026:076:20:00:25.757 to 2026:076:20:01:35.457
2026-03-28 08:34:32,010 get_aca_images_cached: Getting ACA images from 2026:076:20:00:25.757 to 2026:076:20:01:35.457
2026-03-28 08:34:32,171 get_mon_dataset: Getting MON data from 2026:076:20:29:08.782 to 2026:076:20:36:20.307
2026-03-28 08:34:32,171 get_aca_images_cached: Getting ACA images from 2026:076:20:29:08.782 to 2026:076:20:36:20.307
2026-03-28 08:34:32,198 get_mon_dataset: Not enough images for slot 0 found 0)
2026-03-28 08:34:32,199 get_mon_dataset: Not enough images for slot 1 found 0)
2026-03-28 08:34:32,200 get_mon_dataset: Not enough images for slot 2 found 0)
2026-03-28 08:34:32,201 get_mon_dataset: Not enough images for slot 3 found 0)
2026-03-28 08:34:32,202 get_mon_dataset: Not enough images for slot 4 found 0)
2026-03-28 08:34:32,203 get_mon_dataset: Not enough images for slot 5 found 0)
2026-03-28 08:34:32,204 get_mon_dataset: Not enough images for slot 6 found 0)
2026-03-28 08:34:32,204 get_mon_dataset: Not enough images for slot 7 found 0)
2026-03-28 08:34:32,205 get_mon_dataset: Getting MON data from 2026:079:09:53:06.972 to 2026:079:10:02:30.722
2026-03-28 08:34:32,205 get_aca_images_cached: Getting ACA images from 2026:079:09:53:06.972 to 2026:079:10:02:30.722
2026-03-28 08:34:32,584 get_mon_dataset: Getting MON data from 2026:079:10:12:05.747 to 2026:079:10:30:55.297
2026-03-28 08:34:32,584 get_aca_images_cached: Getting ACA images from 2026:079:10:12:05.747 to 2026:079:10:30:55.297
2026-03-28 08:34:32,785 get_mon_dataset: Getting MON data from 2026:079:11:24:09.197 to 2026:079:11:31:20.722
2026-03-28 08:34:32,785 get_aca_images_cached: Getting ACA images from 2026:079:11:24:09.197 to 2026:079:11:31:20.722
2026-03-28 08:34:32,836 get_mon_dataset: Not enough images for slot 0 found 0)
2026-03-28 08:34:32,837 get_mon_dataset: Not enough images for slot 1 found 0)
2026-03-28 08:34:32,838 get_mon_dataset: Not enough images for slot 2 found 0)
2026-03-28 08:34:32,839 get_mon_dataset: Not enough images for slot 3 found 0)
2026-03-28 08:34:32,839 get_mon_dataset: Not enough images for slot 4 found 0)
2026-03-28 08:34:32,840 get_mon_dataset: Not enough images for slot 5 found 0)
2026-03-28 08:34:32,841 get_mon_dataset: Not enough images for slot 6 found 0)
2026-03-28 08:34:32,842 get_mon_dataset: Not enough images for slot 7 found 0)
2026-03-28 08:34:32,843 get_mon_dataset: Getting MON data from 2026:082:02:43:23.062 to 2026:082:03:20:56.012
2026-03-28 08:34:32,843 get_aca_images_cached: Getting ACA images from 2026:082:02:43:23.062 to 2026:082:03:20:56.012
2026-03-28 08:34:33,053 get_mon_dataset: Getting MON data from 2026:082:03:40:40.912 to 2026:082:04:27:45.812
2026-03-28 08:34:33,053 get_aca_images_cached: Getting ACA images from 2026:082:03:40:40.912 to 2026:082:04:27:45.812
2026-03-28 08:34:33,773 get_mon_dataset: Getting MON data from 2026:084:18:34:50.077 to 2026:084:18:38:35.577
2026-03-28 08:34:33,774 get_aca_images_cached: Getting ACA images from 2026:084:18:34:50.077 to 2026:084:18:38:35.577
2026-03-28 08:34:33,950 get_mon_dataset: Getting MON data from 2026:084:18:59:39.402 to 2026:084:19:25:55.852
2026-03-28 08:34:33,950 get_aca_images_cached: Getting ACA images from 2026:084:18:59:39.402 to 2026:084:19:25:55.852
2026-03-28 08:34:34,144 get_kalman_drops_nman: processing NMAN data
2026-03-28 08:34:34,215 get_kalman_drops_nman: Skipping NMAN from 2026:060:21:37:57.315 to 2026:060:21:42:54.565 due to insufficient monitor window data within 100 minutes of perigee
2026-03-28 08:34:34,283 get_kalman_drops_nman: Skipping NMAN from 2026:071:12:50:00.251 to 2026:071:12:51:40.701 due to insufficient monitor window data within 100 minutes of perigee
2026-03-28 08:34:34,288 get_kalman_drops_nman: Skipping NMAN from 2026:071:13:48:41.127 to 2026:071:13:50:06.202 due to insufficient monitor window data within 100 minutes of perigee
2026-03-28 08:34:34,309 get_kalman_drops_nman: Skipping NMAN from 2026:076:20:00:25.757 to 2026:076:20:01:35.457 due to insufficient monitor window data within 100 minutes of perigee
2026-03-28 08:34:34,346 get_kalman_drops_nman: Skipping NMAN from 2026:084:18:34:50.077 to 2026:084:18:38:35.577 due to insufficient monitor window data within 100 minutes of perigee
2026-03-28 07:34:35,414 _get_tlm: Getting telemetry for 2026:042:10:36:04.628
2026-03-28 07:34:36,169 obss: Getting observations from kadi commands for 2026/Feb-11
2026-03-28 07:34:36,181 _get_tlm: Getting telemetry for 2026:045:02:03:56.696
2026-03-28 07:34:36,532 obss: Getting observations from kadi commands for 2026/Feb-14
2026-03-28 07:34:36,547 _get_tlm: Getting telemetry for 2026:047:17:31:28.743
2026-03-28 07:34:36,994 obss: Getting observations from kadi commands for 2026/Feb-16
2026-03-28 07:34:37,008 _get_tlm: Getting telemetry for 2026:050:08:58:37.782
2026-03-28 07:34:37,501 obss: Getting observations from kadi commands for 2026/Feb-19
2026-03-28 07:34:37,514 _get_tlm: Getting telemetry for 2026:053:00:26:17.845
2026-03-28 07:34:37,865 obss: Getting observations from kadi commands for 2026/Feb-22
2026-03-28 07:34:37,879 _get_tlm: Getting telemetry for 2026:055:15:54:54.637
2026-03-28 07:34:38,385 obss: Getting observations from kadi commands for 2026/Feb-24
2026-03-28 07:34:38,399 _get_tlm: Getting telemetry for 2026:058:07:23:48.691
2026-03-28 07:34:38,769 obss: Getting observations from kadi commands for 2026/Feb-27
2026-03-28 07:34:38,780 _get_tlm: Getting telemetry for 2026:060:22:52:08.597
2026-03-28 07:34:39,076 obss: Getting observations from kadi commands for 2026/Mar-01
2026-03-28 07:34:39,090 _get_tlm: Getting telemetry for 2026:063:14:19:39.343
2026-03-28 07:34:39,400 obss: Getting observations from kadi commands for 2026/Mar-04
2026-03-28 07:34:39,413 _get_tlm: Getting telemetry for 2026:066:05:46:52.246
2026-03-28 07:34:39,710 obss: Getting observations from kadi commands for 2026/Mar-07
2026-03-28 07:34:39,724 _get_tlm: Getting telemetry for 2026:068:21:14:15.444
2026-03-28 07:34:40,013 obss: Getting observations from kadi commands for 2026/Mar-09
2026-03-28 07:34:40,024 _get_tlm: Getting telemetry for 2026:071:12:41:57.120
2026-03-28 07:34:40,321 obss: Getting observations from kadi commands for 2026/Mar-12
2026-03-28 07:34:40,333 _get_tlm: Getting telemetry for 2026:074:04:09:29.826
2026-03-28 07:34:40,675 obss: Getting observations from kadi commands for 2026/Mar-15
2026-03-28 07:34:40,689 _get_tlm: Getting telemetry for 2026:076:19:36:34.978
2026-03-28 07:34:40,979 obss: Getting observations from kadi commands for 2026/Mar-17
2026-03-28 07:34:40,993 _get_tlm: Getting telemetry for 2026:079:11:03:52.694
2026-03-28 07:34:41,244 obss: Getting observations from kadi commands for 2026/Mar-20
2026-03-28 07:34:41,257 _get_tlm: Getting telemetry for 2026:082:02:32:10.520
2026-03-28 07:34:41,581 obss: Getting observations from kadi commands for 2026/Mar-23
2026-03-28 07:34:41,593 _get_tlm: Getting telemetry for 2026:084:18:01:03.087
2026-03-28 07:34:41,852 obss: Getting observations from kadi commands for 2026/Mar-25
2026-03-28 07:35:03,908 get_color_marker_for_perigee_plotly: Perigee 2026:042:10:36:04.628 color='rgb(229, 134, 6)' marker='circle'
2026-03-28 07:35:03,908 get_color_marker_for_perigee_plotly: Perigee 2026:045:02:03:56.696 color='rgb(93, 105, 177)' marker='square'
2026-03-28 07:35:03,908 get_color_marker_for_perigee_plotly: Perigee 2026:047:17:31:28.743 color='rgb(82, 188, 163)' marker='diamond'
2026-03-28 07:35:03,908 get_color_marker_for_perigee_plotly: Perigee 2026:050:08:58:37.782 color='rgb(153, 201, 69)' marker='triangle-up'
2026-03-28 07:35:03,909 get_color_marker_for_perigee_plotly: Perigee 2026:053:00:26:17.845 color='rgb(204, 97, 176)' marker='triangle-down'
2026-03-28 07:35:03,909 get_color_marker_for_perigee_plotly: Perigee 2026:055:15:54:54.637 color='rgb(36, 121, 108)' marker='triangle-left'
2026-03-28 07:35:03,909 get_color_marker_for_perigee_plotly: Perigee 2026:058:07:23:48.691 color='rgb(218, 165, 27)' marker='triangle-right'
2026-03-28 07:35:03,909 get_color_marker_for_perigee_plotly: Perigee 2026:060:22:52:08.597 color='rgb(47, 138, 196)' marker='circle'
2026-03-28 07:35:03,909 get_color_marker_for_perigee_plotly: Perigee 2026:063:14:19:39.343 color='rgb(118, 78, 159)' marker='square'
2026-03-28 07:35:03,910 get_color_marker_for_perigee_plotly: Perigee 2026:066:05:46:52.246 color='rgb(237, 100, 90)' marker='diamond'
2026-03-28 07:35:03,910 get_color_marker_for_perigee_plotly: Perigee 2026:068:21:14:15.444 color='rgb(165, 170, 153)' marker='triangle-up'
2026-03-28 07:35:03,910 get_color_marker_for_perigee_plotly: Perigee 2026:071:12:41:57.120 color='rgb(229, 134, 6)' marker='triangle-down'
2026-03-28 07:35:03,910 get_color_marker_for_perigee_plotly: Perigee 2026:074:04:09:29.826 color='rgb(93, 105, 177)' marker='triangle-left'
2026-03-28 07:35:03,910 get_color_marker_for_perigee_plotly: Perigee 2026:076:19:36:34.978 color='rgb(82, 188, 163)' marker='triangle-right'
2026-03-28 07:35:03,911 get_color_marker_for_perigee_plotly: Perigee 2026:079:11:03:52.694 color='rgb(153, 201, 69)' marker='circle'
2026-03-28 07:35:03,911 get_color_marker_for_perigee_plotly: Perigee 2026:082:02:32:10.520 color='rgb(204, 97, 176)' marker='square'
2026-03-28 07:35:03,911 get_color_marker_for_perigee_plotly: Perigee 2026:084:18:01:03.087 color='rgb(36, 121, 108)' marker='diamond'
2026-03-28 07:35:03,911 plot_mon_win_and_aokalstr_composite_plotly: plotting predictions
2026-03-28 07:35:05,132 clean_aca_images_cache: Cleaning ACA images cache to keep most recent 70 files
2026-03-28 08:35:08,814 main: Overriding starting time with the time of latest low-kalman event
2026-03-28 08:35:08,815 get_lowkals_new: Getting telemetry between 2026:086:01:32:20.886 and 2026:087:12:35:08.808
2026-03-28 08:35:09,098 get_lowkals_new: Finding intervals of low kalman stars
2026-03-28 08:35:09,121 get_lowkals_new: Last telemetry date: 2026:087:02:54:16.097
2026-03-28 08:35:09,123 main: Updating events file /proj/sot/ska3/aca-test/www/ASPECT/kalman_watch3/low_kalman_events.ecsv
2026-03-28 08:35:09,447 make_web_page: Writing HTML to /proj/sot/ska3/aca-test/www/ASPECT/kalman_watch3/index.html
2026:086:01:32:20.886 2026:087:12:35:08.808