Prev
Index
Next
kalman_watch3: Thu Jun 5 08:34:26 2025
File: /proj/sot/ska/data/kalman_watch3/logs/daily.0/kalman_watch.log
| Task | Status | Age | MaxAge | Last Date |
|---|
| kalman_watch3 |
NOT OK
|
0.09 | 1.0 |
Thu Jun 5 08:34:26 2025 |
Errors:
- Line 177: 2025-06-05 08:34:26,335 get_lowkals_new: WARNING: Fewer than two kalman stars at 2025:155:09:41:14.343 for 31.8 secs
File contents:
2025-06-05 08:31:18,046 log_run_info: ******************************************
2025-06-05 08:31:18,046 log_run_info: Running: /proj/sot/ska3/flight/lib/python3.12/site-packages/kalman_watch/kalman_perigee_mon.py
2025-06-05 08:31:18,046 log_run_info: Version: 0.6.0
2025-06-05 08:31:18,046 log_run_info: Time: Thu Jun 5 08:31:18 2025
2025-06-05 08:31:18,046 log_run_info: User: aca
2025-06-05 08:31:18,046 log_run_info: Machine: kady.cfa.harvard.edu
2025-06-05 08:31:18,046 log_run_info: Processing args:
2025-06-05 08:31:18,046 log_run_info: {'data_dir': PosixPath('/proj/sot/ska3/flight/www/ASPECT/kalman_watch3'),
2025-06-05 08:31:18,046 log_run_info: 'emails': ['aca_alert@cfa.harvard.edu'],
2025-06-05 08:31:18,046 log_run_info: 'lookback': 14,
2025-06-05 08:31:18,046 log_run_info: 'make_html': True,
2025-06-05 08:31:18,046 log_run_info: 'rad_data': PosixPath('/proj/sot/ska3/flight/data/stk/radiation_data.fits'),
2025-06-05 08:31:18,046 log_run_info: 'stop': None}
2025-06-05 08:31:18,046 log_run_info: ******************************************
2025-06-05 08:31:18,048 read_kalman_stats: Reading kalman perigee data from /proj/sot/ska3/flight/www/ASPECT/kalman_watch3/perigees/kalman_perigees.ecsv
2025-06-05 08:31:18,079 get_evts_perigee: Getting perigee events between 2025:142:12:31:18.047 and 2025:156:12:31:18.047
2025-06-05 08:31:21,747 _get_tlm: Getting telemetry for 2025:156:05:04:26.747
2025-06-05 08:31:22,145 get_evts_perigee: No TLM found for perigee event at 2025:156:05:04:26.747, skipping
2025-06-05 08:31:22,145 get_evts_perigee: Found 0 new perigee event(s)
2025-06-05 08:31:22,146 main: No new perigee events, exiting
2025-06-05 08:31:26,268 get_manvrs_perigee: Getting maneuvers from 2025:111:12:31:26.267 to 2025:155:12:31:26.268
2025-06-05 08:31:30,088 get_manvrs_perigee: Getting intervals of Earth blocks from 2025:111:12:31:26.267 to 2025:155:12:31:26.268
2025-06-05 08:31:37,906 get_manvrs_perigee: Excluding Earth block from 2025:112:18:38:55.328 to 2025:112:18:44:02.828
2025-06-05 08:31:37,906 get_manvrs_perigee: Excluding Earth block from 2025:113:16:21:55.158 to 2025:113:16:28:23.633
2025-06-05 08:31:37,906 get_manvrs_perigee: Excluding Earth block from 2025:119:07:28:13.065 to 2025:119:07:34:39.490
2025-06-05 08:31:37,906 get_manvrs_perigee: Excluding Earth block from 2025:120:06:05:40.596 to 2025:120:06:10:57.321
2025-06-05 08:31:37,906 get_manvrs_perigee: Excluding Earth block from 2025:121:08:36:28.602 to 2025:121:08:41:41.227
2025-06-05 08:31:37,906 get_manvrs_perigee: Excluding Earth block from 2025:124:10:32:26.269 to 2025:124:10:33:16.494
2025-06-05 08:31:37,906 get_manvrs_perigee: Excluding Earth block from 2025:129:20:53:30.775 to 2025:129:20:59:59.250
2025-06-05 08:31:37,906 get_manvrs_perigee: Excluding Earth block from 2025:137:17:03:25.645 to 2025:137:17:59:15.345
2025-06-05 08:31:37,906 get_manvrs_perigee: Excluding Earth block from 2025:140:08:29:12.085 to 2025:140:08:42:49.010
2025-06-05 08:31:37,906 get_manvrs_perigee: Excluding Earth block from 2025:141:21:42:07.794 to 2025:141:21:48:48.569
2025-06-05 08:31:37,906 get_manvrs_perigee: Excluding Earth block from 2025:142:23:53:40.625 to 2025:143:00:02:44.900
2025-06-05 08:31:37,906 get_manvrs_perigee: Excluding Earth block from 2025:151:20:09:58.150 to 2025:151:20:14:19.525
2025-06-05 08:31:37,975 get_mon_dataset: Getting MON data from 2025:113:20:11:44.484 to 2025:113:20:45:56.534
2025-06-05 08:31:37,975 get_aca_images_cached: Getting ACA images from 2025:113:20:11:44.484 to 2025:113:20:45:56.534
2025-06-05 08:31:38,227 get_ir_thresholds: Getting IR thresholds from 2023:100 to 2023:200
2025-06-05 08:33:50,824 get_mon_dataset: Getting MON data from 2025:113:21:21:44.934 to 2025:113:21:55:55.959
2025-06-05 08:33:50,824 get_aca_images_cached: Getting ACA images from 2025:113:21:21:44.934 to 2025:113:21:55:55.959
2025-06-05 08:33:51,074 get_mon_dataset: Getting MON data from 2025:113:23:00:47.885 to 2025:113:23:40:55.610
2025-06-05 08:33:51,074 get_aca_images_cached: Getting ACA images from 2025:113:23:00:47.885 to 2025:113:23:40:55.610
2025-06-05 08:33:51,421 get_mon_dataset: Getting MON data from 2025:116:11:09:04.299 to 2025:116:11:55:55.874
2025-06-05 08:33:51,421 get_aca_images_cached: Getting ACA images from 2025:116:11:09:04.299 to 2025:116:11:55:55.874
2025-06-05 08:33:51,680 get_mon_dataset: Getting MON data from 2025:116:12:51:45.574 to 2025:116:13:25:55.575
2025-06-05 08:33:51,680 get_aca_images_cached: Getting ACA images from 2025:116:12:51:45.574 to 2025:116:13:25:55.575
2025-06-05 08:33:52,131 get_mon_dataset: Getting MON data from 2025:116:14:00:03.525 to 2025:116:14:30:55.700
2025-06-05 08:33:52,131 get_aca_images_cached: Getting ACA images from 2025:116:14:00:03.525 to 2025:116:14:30:55.700
2025-06-05 08:33:52,369 get_mon_dataset: Getting MON data from 2025:119:02:15:54.014 to 2025:119:02:55:55.589
2025-06-05 08:33:52,369 get_aca_images_cached: Getting ACA images from 2025:119:02:15:54.014 to 2025:119:02:55:55.589
2025-06-05 08:33:52,654 get_mon_dataset: Getting MON data from 2025:119:03:50:00.739 to 2025:119:04:13:02.439
2025-06-05 08:33:52,655 get_aca_images_cached: Getting ACA images from 2025:119:03:50:00.739 to 2025:119:04:13:02.439
2025-06-05 08:33:52,924 get_mon_dataset: Getting MON data from 2025:119:05:07:06.565 to 2025:119:05:30:56.440
2025-06-05 08:33:52,924 get_aca_images_cached: Getting ACA images from 2025:119:05:07:06.565 to 2025:119:05:30:56.440
2025-06-05 08:33:53,145 get_mon_dataset: Getting MON data from 2025:119:05:50:41.340 to 2025:119:06:30:38.815
2025-06-05 08:33:53,145 get_aca_images_cached: Getting ACA images from 2025:119:05:50:41.340 to 2025:119:06:30:38.815
2025-06-05 08:33:53,366 get_mon_dataset: Getting MON data from 2025:121:19:09:09.479 to 2025:121:19:16:19.979
2025-06-05 08:33:53,366 get_aca_images_cached: Getting ACA images from 2025:121:19:09:09.479 to 2025:121:19:16:19.979
2025-06-05 08:33:53,410 get_mon_dataset: Not enough images for slot 0 found 0)
2025-06-05 08:33:53,411 get_mon_dataset: Not enough images for slot 1 found 0)
2025-06-05 08:33:53,412 get_mon_dataset: Not enough images for slot 2 found 0)
2025-06-05 08:33:53,413 get_mon_dataset: Not enough images for slot 3 found 0)
2025-06-05 08:33:53,414 get_mon_dataset: Not enough images for slot 4 found 0)
2025-06-05 08:33:53,415 get_mon_dataset: Not enough images for slot 5 found 0)
2025-06-05 08:33:53,416 get_mon_dataset: Not enough images for slot 6 found 0)
2025-06-05 08:33:53,417 get_mon_dataset: Not enough images for slot 7 found 0)
2025-06-05 08:33:53,418 get_mon_dataset: Getting MON data from 2025:121:20:22:11.355 to 2025:121:20:55:55.730
2025-06-05 08:33:53,418 get_aca_images_cached: Getting ACA images from 2025:121:20:22:11.355 to 2025:121:20:55:55.730
2025-06-05 08:33:53,680 get_mon_dataset: Getting MON data from 2025:124:10:18:30.894 to 2025:124:10:32:26.269
2025-06-05 08:33:53,680 get_aca_images_cached: Getting ACA images from 2025:124:10:18:30.894 to 2025:124:10:32:26.269
2025-06-05 08:33:53,875 get_mon_dataset: Getting MON data from 2025:124:10:33:16.494 to 2025:124:10:40:55.694
2025-06-05 08:33:53,875 get_aca_images_cached: Getting ACA images from 2025:124:10:33:16.494 to 2025:124:10:40:55.694
2025-06-05 08:33:54,046 get_mon_dataset: Getting MON data from 2025:127:02:10:52.234 to 2025:127:02:25:55.259
2025-06-05 08:33:54,046 get_aca_images_cached: Getting ACA images from 2025:127:02:10:52.234 to 2025:127:02:25:55.259
2025-06-05 08:33:54,242 get_mon_dataset: Getting MON data from 2025:127:04:05:43.310 to 2025:127:04:40:55.835
2025-06-05 08:33:54,242 get_aca_images_cached: Getting ACA images from 2025:127:04:05:43.310 to 2025:127:04:40:55.835
2025-06-05 08:33:54,726 get_mon_dataset: Getting MON data from 2025:129:17:50:08.424 to 2025:129:18:25:55.800
2025-06-05 08:33:54,726 get_aca_images_cached: Getting ACA images from 2025:129:17:50:08.424 to 2025:129:18:25:55.800
2025-06-05 08:33:54,959 get_mon_dataset: Getting MON data from 2025:129:18:55:35.200 to 2025:129:19:30:55.925
2025-06-05 08:33:54,959 get_aca_images_cached: Getting ACA images from 2025:129:18:55:35.200 to 2025:129:19:30:55.925
2025-06-05 08:33:55,202 get_mon_dataset: Getting MON data from 2025:135:02:29:09.580 to 2025:135:02:36:20.080
2025-06-05 08:33:55,202 get_aca_images_cached: Getting ACA images from 2025:135:02:29:09.580 to 2025:135:02:36:20.080
2025-06-05 08:33:55,284 get_mon_dataset: Not enough images for slot 0 found 0)
2025-06-05 08:33:55,285 get_mon_dataset: Not enough images for slot 1 found 0)
2025-06-05 08:33:55,286 get_mon_dataset: Not enough images for slot 2 found 0)
2025-06-05 08:33:55,287 get_mon_dataset: Not enough images for slot 3 found 0)
2025-06-05 08:33:55,288 get_mon_dataset: Not enough images for slot 4 found 0)
2025-06-05 08:33:55,289 get_mon_dataset: Not enough images for slot 5 found 0)
2025-06-05 08:33:55,290 get_mon_dataset: Not enough images for slot 6 found 0)
2025-06-05 08:33:55,291 get_mon_dataset: Not enough images for slot 7 found 0)
2025-06-05 08:33:55,291 get_mon_dataset: Getting MON data from 2025:137:17:01:00.095 to 2025:137:17:03:25.645
2025-06-05 08:33:55,292 get_aca_images_cached: Getting ACA images from 2025:137:17:01:00.095 to 2025:137:17:03:25.645
2025-06-05 08:33:55,533 get_mon_dataset: Getting MON data from 2025:137:17:59:15.345 to 2025:137:18:00:26.070
2025-06-05 08:33:55,533 get_aca_images_cached: Getting ACA images from 2025:137:17:59:15.345 to 2025:137:18:00:26.070
2025-06-05 08:33:55,686 get_mon_dataset: Getting MON data from 2025:137:18:24:12.870 to 2025:137:18:31:24.395
2025-06-05 08:33:55,687 get_aca_images_cached: Getting ACA images from 2025:137:18:24:12.870 to 2025:137:18:31:24.395
2025-06-05 08:33:55,730 get_mon_dataset: Not enough images for slot 0 found 0)
2025-06-05 08:33:55,731 get_mon_dataset: Not enough images for slot 1 found 0)
2025-06-05 08:33:55,732 get_mon_dataset: Not enough images for slot 2 found 0)
2025-06-05 08:33:55,733 get_mon_dataset: Not enough images for slot 3 found 0)
2025-06-05 08:33:55,734 get_mon_dataset: Not enough images for slot 4 found 0)
2025-06-05 08:33:55,735 get_mon_dataset: Not enough images for slot 5 found 0)
2025-06-05 08:33:55,736 get_mon_dataset: Not enough images for slot 6 found 0)
2025-06-05 08:33:55,737 get_mon_dataset: Not enough images for slot 7 found 0)
2025-06-05 08:33:55,738 get_mon_dataset: Getting MON data from 2025:140:08:26:03.485 to 2025:140:08:29:12.085
2025-06-05 08:33:55,738 get_aca_images_cached: Getting ACA images from 2025:140:08:26:03.485 to 2025:140:08:29:12.085
2025-06-05 08:33:55,919 get_mon_dataset: Getting MON data from 2025:140:08:42:49.010 to 2025:140:08:55:56.210
2025-06-05 08:33:55,920 get_aca_images_cached: Getting ACA images from 2025:140:08:42:49.010 to 2025:140:08:55:56.210
2025-06-05 08:33:56,126 get_mon_dataset: Getting MON data from 2025:142:23:46:22.950 to 2025:142:23:53:40.625
2025-06-05 08:33:56,126 get_aca_images_cached: Getting ACA images from 2025:142:23:46:22.950 to 2025:142:23:53:40.625
2025-06-05 08:33:56,310 get_mon_dataset: Getting MON data from 2025:143:00:02:44.900 to 2025:143:00:15:56.200
2025-06-05 08:33:56,310 get_aca_images_cached: Getting ACA images from 2025:143:00:02:44.900 to 2025:143:00:15:56.200
2025-06-05 08:33:56,521 get_mon_dataset: Getting MON data from 2025:143:01:10:00.325 to 2025:143:01:43:55.975
2025-06-05 08:33:56,521 get_aca_images_cached: Getting ACA images from 2025:143:01:10:00.325 to 2025:143:01:43:55.975
2025-06-05 08:33:56,854 get_mon_dataset: Getting MON data from 2025:145:14:31:50.390 to 2025:145:15:20:56.240
2025-06-05 08:33:56,854 get_aca_images_cached: Getting ACA images from 2025:145:14:31:50.390 to 2025:145:15:20:56.240
2025-06-05 08:33:57,329 get_mon_dataset: Getting MON data from 2025:148:06:51:18.405 to 2025:148:07:25:56.080
2025-06-05 08:33:57,329 get_aca_images_cached: Getting ACA images from 2025:148:06:51:18.405 to 2025:148:07:25:56.080
2025-06-05 08:33:57,569 get_mon_dataset: Getting MON data from 2025:150:21:31:20.920 to 2025:150:22:05:55.520
2025-06-05 08:33:57,570 get_aca_images_cached: Getting ACA images from 2025:150:21:31:20.920 to 2025:150:22:05:55.520
2025-06-05 08:33:58,270 get_mon_dataset: Getting MON data from 2025:153:11:41:02.009 to 2025:153:12:15:35.585
2025-06-05 08:33:58,270 get_aca_images_cached: Getting ACA images from 2025:153:11:41:02.009 to 2025:153:12:15:35.585
2025-06-05 08:33:58,477 get_mon_dataset: Getting MON data from 2025:153:14:21:18.560 to 2025:153:14:55:56.235
2025-06-05 08:33:58,477 get_aca_images_cached: Getting ACA images from 2025:153:14:21:18.560 to 2025:153:14:55:56.235
2025-06-05 08:33:58,950 get_kalman_drops_nman: processing NMAN data
2025-06-05 07:33:59,534 _get_tlm: Getting telemetry for 2025:113:21:33:28.816
2025-06-05 07:34:00,042 obss: Getting observations from kadi commands for 2025/Apr-23
2025-06-05 07:34:00,053 _get_tlm: Getting telemetry for 2025:116:13:00:22.011
2025-06-05 07:34:00,583 obss: Getting observations from kadi commands for 2025/Apr-26
2025-06-05 07:34:00,593 _get_tlm: Getting telemetry for 2025:119:04:28:33.439
2025-06-05 07:34:00,981 obss: Getting observations from kadi commands for 2025/Apr-29
2025-06-05 07:34:00,992 _get_tlm: Getting telemetry for 2025:121:19:57:38.832
2025-06-05 07:34:01,408 obss: Getting observations from kadi commands for 2025/May-01
2025-06-05 07:34:01,421 _get_tlm: Getting telemetry for 2025:124:11:26:34.651
2025-06-05 07:34:01,840 obss: Getting observations from kadi commands for 2025/May-04
2025-06-05 07:34:01,856 _get_tlm: Getting telemetry for 2025:127:02:54:50.666
2025-06-05 07:34:02,353 obss: Getting observations from kadi commands for 2025/May-07
2025-06-05 07:34:02,366 _get_tlm: Getting telemetry for 2025:129:18:22:33.747
2025-06-05 07:34:02,810 obss: Getting observations from kadi commands for 2025/May-09
2025-06-05 07:34:02,823 _get_tlm: Getting telemetry for 2025:132:09:50:06.481
2025-06-05 07:34:03,211 obss: Getting observations from kadi commands for 2025/May-12
2025-06-05 07:34:03,227 _get_tlm: Getting telemetry for 2025:135:01:17:53.877
2025-06-05 07:34:03,564 obss: Getting observations from kadi commands for 2025/May-15
2025-06-05 07:34:03,580 _get_tlm: Getting telemetry for 2025:137:16:45:53.362
2025-06-05 07:34:04,176 obss: Getting observations from kadi commands for 2025/May-17
2025-06-05 07:34:04,189 _get_tlm: Getting telemetry for 2025:140:08:13:27.724
2025-06-05 07:34:05,241 obss: Getting observations from kadi commands for 2025/May-20
2025-06-05 07:34:05,256 _get_tlm: Getting telemetry for 2025:142:23:40:40.549
2025-06-05 07:34:06,746 obss: Getting observations from kadi commands for 2025/May-22
2025-06-05 07:34:06,760 _get_tlm: Getting telemetry for 2025:145:15:08:42.626
2025-06-05 07:34:07,050 obss: Getting observations from kadi commands for 2025/May-25
2025-06-05 07:34:07,063 _get_tlm: Getting telemetry for 2025:148:06:37:59.536
2025-06-05 07:34:07,378 obss: Getting observations from kadi commands for 2025/May-28
2025-06-05 07:34:07,392 _get_tlm: Getting telemetry for 2025:150:22:07:27.321
2025-06-05 07:34:07,716 obss: Getting observations from kadi commands for 2025/May-30
2025-06-05 07:34:07,730 _get_tlm: Getting telemetry for 2025:153:13:36:16.847
2025-06-05 07:34:07,993 obss: Getting observations from kadi commands for 2025/Jun-02
2025-06-05 07:34:22,299 get_color_marker_for_perigee_plotly: Perigee 2025:113:21:33:28.816 color='rgb(229, 134, 6)' marker='circle'
2025-06-05 07:34:22,300 get_color_marker_for_perigee_plotly: Perigee 2025:116:13:00:22.011 color='rgb(93, 105, 177)' marker='square'
2025-06-05 07:34:22,300 get_color_marker_for_perigee_plotly: Perigee 2025:119:04:28:33.439 color='rgb(82, 188, 163)' marker='diamond'
2025-06-05 07:34:22,300 get_color_marker_for_perigee_plotly: Perigee 2025:121:19:57:38.832 color='rgb(153, 201, 69)' marker='triangle-up'
2025-06-05 07:34:22,300 get_color_marker_for_perigee_plotly: Perigee 2025:124:11:26:34.651 color='rgb(204, 97, 176)' marker='triangle-down'
2025-06-05 07:34:22,301 get_color_marker_for_perigee_plotly: Perigee 2025:127:02:54:50.666 color='rgb(36, 121, 108)' marker='triangle-left'
2025-06-05 07:34:22,301 get_color_marker_for_perigee_plotly: Perigee 2025:129:18:22:33.747 color='rgb(218, 165, 27)' marker='triangle-right'
2025-06-05 07:34:22,301 get_color_marker_for_perigee_plotly: Perigee 2025:132:09:50:06.481 color='rgb(47, 138, 196)' marker='circle'
2025-06-05 07:34:22,301 get_color_marker_for_perigee_plotly: Perigee 2025:135:01:17:53.877 color='rgb(118, 78, 159)' marker='square'
2025-06-05 07:34:22,301 get_color_marker_for_perigee_plotly: Perigee 2025:137:16:45:53.362 color='rgb(237, 100, 90)' marker='diamond'
2025-06-05 07:34:22,302 get_color_marker_for_perigee_plotly: Perigee 2025:140:08:13:27.724 color='rgb(165, 170, 153)' marker='triangle-up'
2025-06-05 07:34:22,302 get_color_marker_for_perigee_plotly: Perigee 2025:142:23:40:40.549 color='rgb(229, 134, 6)' marker='triangle-down'
2025-06-05 07:34:22,302 get_color_marker_for_perigee_plotly: Perigee 2025:145:15:08:42.626 color='rgb(93, 105, 177)' marker='triangle-left'
2025-06-05 07:34:22,302 get_color_marker_for_perigee_plotly: Perigee 2025:148:06:37:59.536 color='rgb(82, 188, 163)' marker='triangle-right'
2025-06-05 07:34:22,302 get_color_marker_for_perigee_plotly: Perigee 2025:150:22:07:27.321 color='rgb(153, 201, 69)' marker='circle'
2025-06-05 07:34:22,303 get_color_marker_for_perigee_plotly: Perigee 2025:153:13:36:16.847 color='rgb(204, 97, 176)' marker='square'
2025-06-05 07:34:22,303 plot_mon_win_and_aokalstr_composite_plotly: plotting predictions
2025-06-05 07:34:24,062 clean_aca_images_cache: Cleaning ACA images cache to keep most recent 70 files
2025-06-05 08:34:26,140 main: Overriding starting time with the time of latest low-kalman event
2025-06-05 08:34:26,140 get_lowkals_new: Getting telemetry between 2025:155:01:13:54.401 and 2025:156:12:34:26.133
2025-06-05 08:34:26,324 get_lowkals_new: Finding intervals of low kalman stars
2025-06-05 08:34:26,335 get_lowkals_new: WARNING: Fewer than two kalman stars at 2025:155:09:41:14.343 for 31.8 secs
2025-06-05 08:34:26,336 get_lowkals_new: Last telemetry date: 2025:155:12:44:20.806
2025-06-05 08:34:26,336 get_lowkals_new: Found 1 new low kalman events
2025-06-05 08:34:26,339 main: Updating events file /proj/sot/ska3/flight/www/ASPECT/kalman_watch3/low_kalman_events.ecsv
2025-06-05 08:34:26,552 make_web_page: Writing HTML to /proj/sot/ska3/flight/www/ASPECT/kalman_watch3/index.html
2025:155:01:13:54.401 2025:156:12:34:26.133