Prev   Index   Next

kalman_watch3: Mon Jun 16 08:34:08 2025

File: /proj/sot/ska/data/kalman_watch3/logs/daily.0/kalman_watch.log

Task Status AgeMaxAge Last Date
kalman_watch3 NOT OK 0.091.0 Mon Jun 16 08:34:08 2025

Errors:

File contents:

2025-06-16 08:31:15,444 log_run_info: ******************************************
2025-06-16 08:31:15,444 log_run_info: Running: /proj/sot/ska3/flight/lib/python3.12/site-packages/kalman_watch/kalman_perigee_mon.py
2025-06-16 08:31:15,444 log_run_info: Version: 0.6.0
2025-06-16 08:31:15,444 log_run_info: Time: Mon Jun 16 08:31:15 2025
2025-06-16 08:31:15,444 log_run_info: User: aca
2025-06-16 08:31:15,444 log_run_info: Machine: kady.cfa.harvard.edu
2025-06-16 08:31:15,444 log_run_info: Processing args:
2025-06-16 08:31:15,444 log_run_info: {'data_dir': PosixPath('/proj/sot/ska3/flight/www/ASPECT/kalman_watch3'),
2025-06-16 08:31:15,444 log_run_info: 'emails': ['aca_alert@cfa.harvard.edu'],
2025-06-16 08:31:15,444 log_run_info: 'lookback': 14,
2025-06-16 08:31:15,444 log_run_info: 'make_html': True,
2025-06-16 08:31:15,444 log_run_info: 'rad_data': PosixPath('/proj/sot/ska3/flight/data/stk/radiation_data.fits'),
2025-06-16 08:31:15,444 log_run_info: 'stop': None}
2025-06-16 08:31:15,444 log_run_info: ******************************************
2025-06-16 08:31:15,446 read_kalman_stats: Reading kalman perigee data from /proj/sot/ska3/flight/www/ASPECT/kalman_watch3/perigees/kalman_perigees.ecsv
2025-06-16 08:31:15,479 get_evts_perigee: Getting perigee events between 2025:153:12:31:15.445 and 2025:167:12:31:15.445
2025-06-16 08:31:17,258 _get_tlm: Getting telemetry for 2025:164:03:28:31.994
2025-06-16 08:31:17,540 get_evts_perigee: No TLM found for perigee event at 2025:164:03:28:31.994, skipping
2025-06-16 08:31:17,541 _get_tlm: Getting telemetry for 2025:166:18:56:30.365
2025-06-16 08:31:17,744 get_evts_perigee: No TLM found for perigee event at 2025:166:18:56:30.365, skipping
2025-06-16 08:31:17,744 get_evts_perigee: Found 0 new perigee event(s)
2025-06-16 08:31:17,744 main: No new perigee events, exiting
2025-06-16 08:31:21,457 get_manvrs_perigee: Getting maneuvers from 2025:122:12:31:21.456 to 2025:166:12:31:21.457
2025-06-16 08:31:24,964 get_manvrs_perigee: Getting intervals of Earth blocks from 2025:122:12:31:21.456 to 2025:166:12:31:21.457
2025-06-16 08:31:31,450 get_manvrs_perigee: Excluding Earth block from 2025:124:10:32:26.269 to 2025:124:10:33:16.494
2025-06-16 08:31:31,450 get_manvrs_perigee: Excluding Earth block from 2025:129:20:53:30.775 to 2025:129:20:59:59.250
2025-06-16 08:31:31,450 get_manvrs_perigee: Excluding Earth block from 2025:137:17:03:25.645 to 2025:137:17:59:15.345
2025-06-16 08:31:31,450 get_manvrs_perigee: Excluding Earth block from 2025:140:08:29:12.085 to 2025:140:08:42:49.010
2025-06-16 08:31:31,450 get_manvrs_perigee: Excluding Earth block from 2025:141:21:42:07.794 to 2025:141:21:48:48.569
2025-06-16 08:31:31,450 get_manvrs_perigee: Excluding Earth block from 2025:142:23:53:40.625 to 2025:143:00:02:44.900
2025-06-16 08:31:31,451 get_manvrs_perigee: Excluding Earth block from 2025:151:20:09:58.150 to 2025:151:20:14:19.525
2025-06-16 08:31:31,451 get_manvrs_perigee: Excluding Earth block from 2025:156:04:01:43.825 to 2025:156:04:12:30.600
2025-06-16 08:31:31,451 get_manvrs_perigee: Excluding Earth block from 2025:163:00:14:26.464 to 2025:163:00:16:43.814
2025-06-16 08:31:31,518 get_mon_dataset: Getting MON data from 2025:124:10:18:30.894 to 2025:124:10:32:26.269
2025-06-16 08:31:31,518 get_aca_images_cached: Getting ACA images from 2025:124:10:18:30.894 to 2025:124:10:32:26.269
2025-06-16 08:31:31,712 get_ir_thresholds: Getting IR thresholds from 2023:100 to 2023:200
2025-06-16 08:33:44,278 get_mon_dataset: Getting MON data from 2025:124:10:33:16.494 to 2025:124:10:40:55.694
2025-06-16 08:33:44,278 get_aca_images_cached: Getting ACA images from 2025:124:10:33:16.494 to 2025:124:10:40:55.694
2025-06-16 08:33:44,467 get_mon_dataset: Getting MON data from 2025:127:02:10:52.234 to 2025:127:02:25:55.259
2025-06-16 08:33:44,468 get_aca_images_cached: Getting ACA images from 2025:127:02:10:52.234 to 2025:127:02:25:55.259
2025-06-16 08:33:44,631 get_mon_dataset: Getting MON data from 2025:127:04:05:43.310 to 2025:127:04:40:55.835
2025-06-16 08:33:44,631 get_aca_images_cached: Getting ACA images from 2025:127:04:05:43.310 to 2025:127:04:40:55.835
2025-06-16 08:33:44,836 get_mon_dataset: Getting MON data from 2025:129:17:50:08.424 to 2025:129:18:25:55.800
2025-06-16 08:33:44,836 get_aca_images_cached: Getting ACA images from 2025:129:17:50:08.424 to 2025:129:18:25:55.800
2025-06-16 08:33:45,266 get_mon_dataset: Getting MON data from 2025:129:18:55:35.200 to 2025:129:19:30:55.925
2025-06-16 08:33:45,266 get_aca_images_cached: Getting ACA images from 2025:129:18:55:35.200 to 2025:129:19:30:55.925
2025-06-16 08:33:45,458 get_mon_dataset: Getting MON data from 2025:135:02:29:09.580 to 2025:135:02:36:20.080
2025-06-16 08:33:45,459 get_aca_images_cached: Getting ACA images from 2025:135:02:29:09.580 to 2025:135:02:36:20.080
2025-06-16 08:33:45,485 get_mon_dataset: Not enough images for slot 0 found 0)
2025-06-16 08:33:45,486 get_mon_dataset: Not enough images for slot 1 found 0)
2025-06-16 08:33:45,487 get_mon_dataset: Not enough images for slot 2 found 0)
2025-06-16 08:33:45,488 get_mon_dataset: Not enough images for slot 3 found 0)
2025-06-16 08:33:45,489 get_mon_dataset: Not enough images for slot 4 found 0)
2025-06-16 08:33:45,490 get_mon_dataset: Not enough images for slot 5 found 0)
2025-06-16 08:33:45,491 get_mon_dataset: Not enough images for slot 6 found 0)
2025-06-16 08:33:45,492 get_mon_dataset: Not enough images for slot 7 found 0)
2025-06-16 08:33:45,493 get_mon_dataset: Getting MON data from 2025:137:17:01:00.095 to 2025:137:17:03:25.645
2025-06-16 08:33:45,493 get_aca_images_cached: Getting ACA images from 2025:137:17:01:00.095 to 2025:137:17:03:25.645
2025-06-16 08:33:45,638 get_mon_dataset: Getting MON data from 2025:137:17:59:15.345 to 2025:137:18:00:26.070
2025-06-16 08:33:45,638 get_aca_images_cached: Getting ACA images from 2025:137:17:59:15.345 to 2025:137:18:00:26.070
2025-06-16 08:33:45,780 get_mon_dataset: Getting MON data from 2025:137:18:24:12.870 to 2025:137:18:31:24.395
2025-06-16 08:33:45,781 get_aca_images_cached: Getting ACA images from 2025:137:18:24:12.870 to 2025:137:18:31:24.395
2025-06-16 08:33:45,807 get_mon_dataset: Not enough images for slot 0 found 0)
2025-06-16 08:33:45,808 get_mon_dataset: Not enough images for slot 1 found 0)
2025-06-16 08:33:45,809 get_mon_dataset: Not enough images for slot 2 found 0)
2025-06-16 08:33:45,810 get_mon_dataset: Not enough images for slot 3 found 0)
2025-06-16 08:33:45,811 get_mon_dataset: Not enough images for slot 4 found 0)
2025-06-16 08:33:45,812 get_mon_dataset: Not enough images for slot 5 found 0)
2025-06-16 08:33:45,813 get_mon_dataset: Not enough images for slot 6 found 0)
2025-06-16 08:33:45,814 get_mon_dataset: Not enough images for slot 7 found 0)
2025-06-16 08:33:45,815 get_mon_dataset: Getting MON data from 2025:140:08:26:03.485 to 2025:140:08:29:12.085
2025-06-16 08:33:45,815 get_aca_images_cached: Getting ACA images from 2025:140:08:26:03.485 to 2025:140:08:29:12.085
2025-06-16 08:33:45,961 get_mon_dataset: Getting MON data from 2025:140:08:42:49.010 to 2025:140:08:55:56.210
2025-06-16 08:33:45,961 get_aca_images_cached: Getting ACA images from 2025:140:08:42:49.010 to 2025:140:08:55:56.210
2025-06-16 08:33:46,237 get_mon_dataset: Getting MON data from 2025:142:23:46:22.950 to 2025:142:23:53:40.625
2025-06-16 08:33:46,238 get_aca_images_cached: Getting ACA images from 2025:142:23:46:22.950 to 2025:142:23:53:40.625
2025-06-16 08:33:46,394 get_mon_dataset: Getting MON data from 2025:143:00:02:44.900 to 2025:143:00:15:56.200
2025-06-16 08:33:46,394 get_aca_images_cached: Getting ACA images from 2025:143:00:02:44.900 to 2025:143:00:15:56.200
2025-06-16 08:33:46,557 get_mon_dataset: Getting MON data from 2025:143:01:10:00.325 to 2025:143:01:43:55.975
2025-06-16 08:33:46,558 get_aca_images_cached: Getting ACA images from 2025:143:01:10:00.325 to 2025:143:01:43:55.975
2025-06-16 08:33:46,765 get_mon_dataset: Getting MON data from 2025:145:14:31:50.390 to 2025:145:15:20:56.240
2025-06-16 08:33:46,765 get_aca_images_cached: Getting ACA images from 2025:145:14:31:50.390 to 2025:145:15:20:56.240
2025-06-16 08:33:46,991 get_mon_dataset: Getting MON data from 2025:148:06:51:18.405 to 2025:148:07:25:56.080
2025-06-16 08:33:46,991 get_aca_images_cached: Getting ACA images from 2025:148:06:51:18.405 to 2025:148:07:25:56.080
2025-06-16 08:33:47,428 get_mon_dataset: Getting MON data from 2025:150:21:31:20.920 to 2025:150:22:05:55.520
2025-06-16 08:33:47,428 get_aca_images_cached: Getting ACA images from 2025:150:21:31:20.920 to 2025:150:22:05:55.520
2025-06-16 08:33:47,681 get_mon_dataset: Getting MON data from 2025:153:11:41:02.009 to 2025:153:12:15:35.585
2025-06-16 08:33:47,681 get_aca_images_cached: Getting ACA images from 2025:153:11:41:02.009 to 2025:153:12:15:35.585
2025-06-16 08:33:47,911 get_mon_dataset: Getting MON data from 2025:153:14:21:18.560 to 2025:153:14:55:56.235
2025-06-16 08:33:47,911 get_aca_images_cached: Getting ACA images from 2025:153:14:21:18.560 to 2025:153:14:55:56.235
2025-06-16 08:33:48,359 get_mon_dataset: Getting MON data from 2025:156:03:35:35.575 to 2025:156:04:01:43.825
2025-06-16 08:33:48,359 get_aca_images_cached: Getting ACA images from 2025:156:03:35:35.575 to 2025:156:04:01:43.825
2025-06-16 08:33:48,563 get_mon_dataset: Getting MON data from 2025:156:04:12:30.600 to 2025:156:04:15:34.075
2025-06-16 08:33:48,563 get_aca_images_cached: Getting ACA images from 2025:156:04:12:30.600 to 2025:156:04:15:34.075
2025-06-16 08:33:48,725 get_mon_dataset: Getting MON data from 2025:156:06:32:01.775 to 2025:156:07:15:56.025
2025-06-16 08:33:48,726 get_aca_images_cached: Getting ACA images from 2025:156:06:32:01.775 to 2025:156:07:15:56.025
2025-06-16 08:33:48,941 get_mon_dataset: Getting MON data from 2025:161:10:13:51.005 to 2025:161:10:45:55.955
2025-06-16 08:33:48,941 get_aca_images_cached: Getting ACA images from 2025:161:10:13:51.005 to 2025:161:10:45:55.955
2025-06-16 08:33:49,143 get_mon_dataset: Getting MON data from 2025:161:13:26:51.455 to 2025:161:13:55:56.006
2025-06-16 08:33:49,143 get_aca_images_cached: Getting ACA images from 2025:161:13:26:51.455 to 2025:161:13:55:56.006
2025-06-16 08:33:49,339 get_kalman_drops_nman: processing NMAN data
2025-06-16 07:33:49,936 _get_tlm: Getting telemetry for 2025:124:11:26:34.651
2025-06-16 07:33:50,481 obss: Getting observations from kadi commands for 2025/May-04
2025-06-16 07:33:50,496 _get_tlm: Getting telemetry for 2025:127:02:54:50.666
2025-06-16 07:33:50,855 obss: Getting observations from kadi commands for 2025/May-07
2025-06-16 07:33:50,868 _get_tlm: Getting telemetry for 2025:129:18:22:33.747
2025-06-16 07:33:51,141 obss: Getting observations from kadi commands for 2025/May-09
2025-06-16 07:33:51,152 _get_tlm: Getting telemetry for 2025:132:09:50:06.481
2025-06-16 07:33:51,412 obss: Getting observations from kadi commands for 2025/May-12
2025-06-16 07:33:51,428 _get_tlm: Getting telemetry for 2025:135:01:17:53.877
2025-06-16 07:33:51,802 obss: Getting observations from kadi commands for 2025/May-15
2025-06-16 07:33:51,817 _get_tlm: Getting telemetry for 2025:137:16:45:53.362
2025-06-16 07:33:52,072 obss: Getting observations from kadi commands for 2025/May-17
2025-06-16 07:33:52,085 _get_tlm: Getting telemetry for 2025:140:08:13:27.724
2025-06-16 07:33:52,384 obss: Getting observations from kadi commands for 2025/May-20
2025-06-16 07:33:52,398 _get_tlm: Getting telemetry for 2025:142:23:40:40.549
2025-06-16 07:33:52,698 obss: Getting observations from kadi commands for 2025/May-22
2025-06-16 07:33:52,711 _get_tlm: Getting telemetry for 2025:145:15:08:42.626
2025-06-16 07:33:52,986 obss: Getting observations from kadi commands for 2025/May-25
2025-06-16 07:33:52,999 _get_tlm: Getting telemetry for 2025:148:06:37:59.536
2025-06-16 07:33:53,335 obss: Getting observations from kadi commands for 2025/May-28
2025-06-16 07:33:53,349 _get_tlm: Getting telemetry for 2025:150:22:07:27.321
2025-06-16 07:33:53,613 obss: Getting observations from kadi commands for 2025/May-30
2025-06-16 07:33:53,626 _get_tlm: Getting telemetry for 2025:153:13:36:16.847
2025-06-16 07:33:53,880 obss: Getting observations from kadi commands for 2025/Jun-02
2025-06-16 07:33:53,893 _get_tlm: Getting telemetry for 2025:156:05:04:26.747
2025-06-16 07:33:54,130 obss: Getting observations from kadi commands for 2025/Jun-05
2025-06-16 07:33:54,143 _get_tlm: Getting telemetry for 2025:158:20:32:17.859
2025-06-16 07:33:54,382 obss: Getting observations from kadi commands for 2025/Jun-07
2025-06-16 07:33:54,398 _get_tlm: Getting telemetry for 2025:161:12:00:17.914
2025-06-16 07:33:54,640 obss: Getting observations from kadi commands for 2025/Jun-10
2025-06-16 07:33:54,654 _get_tlm: Getting telemetry for 2025:164:03:28:31.994
2025-06-16 07:33:54,857 get_kalman_drops_npnt: No telemetry for perigee 2025:164:03:28:31.994
2025-06-16 07:34:02,782 get_color_marker_for_perigee_plotly: Perigee 2025:124:11:26:34.651 color='rgb(229, 134, 6)' marker='circle'
2025-06-16 07:34:02,782 get_color_marker_for_perigee_plotly: Perigee 2025:127:02:54:50.666 color='rgb(93, 105, 177)' marker='square'
2025-06-16 07:34:02,783 get_color_marker_for_perigee_plotly: Perigee 2025:129:18:22:33.747 color='rgb(82, 188, 163)' marker='diamond'
2025-06-16 07:34:02,783 get_color_marker_for_perigee_plotly: Perigee 2025:132:09:50:06.481 color='rgb(153, 201, 69)' marker='triangle-up'
2025-06-16 07:34:02,783 get_color_marker_for_perigee_plotly: Perigee 2025:135:01:17:53.877 color='rgb(204, 97, 176)' marker='triangle-down'
2025-06-16 07:34:02,783 get_color_marker_for_perigee_plotly: Perigee 2025:137:16:45:53.362 color='rgb(36, 121, 108)' marker='triangle-left'
2025-06-16 07:34:02,783 get_color_marker_for_perigee_plotly: Perigee 2025:140:08:13:27.724 color='rgb(218, 165, 27)' marker='triangle-right'
2025-06-16 07:34:02,784 get_color_marker_for_perigee_plotly: Perigee 2025:142:23:40:40.549 color='rgb(47, 138, 196)' marker='circle'
2025-06-16 07:34:02,784 get_color_marker_for_perigee_plotly: Perigee 2025:145:15:08:42.626 color='rgb(118, 78, 159)' marker='square'
2025-06-16 07:34:02,784 get_color_marker_for_perigee_plotly: Perigee 2025:148:06:37:59.536 color='rgb(237, 100, 90)' marker='diamond'
2025-06-16 07:34:02,784 get_color_marker_for_perigee_plotly: Perigee 2025:150:22:07:27.321 color='rgb(165, 170, 153)' marker='triangle-up'
2025-06-16 07:34:02,784 get_color_marker_for_perigee_plotly: Perigee 2025:153:13:36:16.847 color='rgb(229, 134, 6)' marker='triangle-down'
2025-06-16 07:34:02,785 get_color_marker_for_perigee_plotly: Perigee 2025:156:05:04:26.747 color='rgb(93, 105, 177)' marker='triangle-left'
2025-06-16 07:34:02,785 get_color_marker_for_perigee_plotly: Perigee 2025:158:20:32:17.859 color='rgb(82, 188, 163)' marker='triangle-right'
2025-06-16 07:34:02,785 get_color_marker_for_perigee_plotly: Perigee 2025:161:12:00:17.914 color='rgb(153, 201, 69)' marker='circle'
2025-06-16 07:34:02,785 plot_mon_win_and_aokalstr_composite_plotly: plotting predictions
2025-06-16 07:34:02,831 plot_mon_win_and_aokalstr_composite_plotly: Skipping prediction for 2025:164:03:28:31.994 because there is no observed data
2025-06-16 07:34:03,200 clean_aca_images_cache: Cleaning ACA images cache to keep most recent 70 files
2025-06-16 08:34:07,397 main: Overriding starting time with the time of latest low-kalman event
2025-06-16 08:34:07,397 get_lowkals_new: Getting telemetry between 2025:163:04:24:03.249 and 2025:167:12:34:07.390
2025-06-16 08:34:07,548 get_lowkals_new: WARNING: Not enough data to find low Kalman intervals
2025-06-16 08:34:07,552 main: Updating events file /proj/sot/ska3/flight/www/ASPECT/kalman_watch3/low_kalman_events.ecsv
2025-06-16 08:34:07,845 make_web_page: Writing HTML to /proj/sot/ska3/flight/www/ASPECT/kalman_watch3/index.html
2025:163:04:24:03.249 2025:167:12:34:07.390