Prev
Index
Next
centroid_dashboard: Tue Jul 14 13:17:26 2026
File: /proj/sot/ska/data/ska_trend/centroid_dashboard/logs/daily.0/centroid_dashboard.log
| Task | Status | Age | MaxAge | Last Date |
|---|
| centroid_dashboard |
OK
|
0.89 | 1.0 |
Tue Jul 14 13:17:26 2026 |
No errors
File contents:
2026-07-14 13:16:09,838 main: Processing from 2026:188:17:16:09.837 to 2026:195:17:16:09.837
2026-07-14 13:16:16,062 get_observations: Found observation 30419 at 2026:188:19:04:16.363 with 1 manvrs
2026-07-14 13:16:16,062 get_observations: Found observation 29817 at 2026:189:06:05:09.700 with 1 manvrs
2026-07-14 13:16:16,063 get_observations: Found observation 30187 at 2026:189:09:30:17.467 with 1 manvrs
2026-07-14 13:16:16,063 get_observations: Found observation 32420 at 2026:189:18:28:41.042 with 1 manvrs
2026-07-14 13:16:16,063 get_observations: Found observation 31302 at 2026:189:21:49:50.434 with 1 manvrs
2026-07-14 13:16:16,064 get_observations: Found observation 32421 at 2026:190:03:07:37.864 with 1 manvrs
2026-07-14 13:16:16,064 get_observations: Found observation 41357 at 2026:190:06:22:48.987 with 1 manvrs
2026-07-14 13:16:16,065 get_observations: Found observation 41356 at 2026:190:08:03:53.972 with 1 manvrs
2026-07-14 13:16:16,065 get_observations: Found observation 41355 at 2026:190:11:25:40.252 with 1 manvrs
2026-07-14 13:16:16,065 get_observations: Found observation 41354 at 2026:190:13:10:03.721 with 1 manvrs
2026-07-14 13:16:16,066 get_observations: Found observation 41353 at 2026:190:14:54:27.036 with 1 manvrs
2026-07-14 13:16:16,066 get_observations: Found observation 41352 at 2026:190:16:55:40.252 with 1 manvrs
2026-07-14 13:16:16,067 get_observations: Found observation 30364 at 2026:190:18:32:41.726 with 1 manvrs
2026-07-14 13:16:16,067 get_observations: Found observation 31118 at 2026:190:23:46:02.653 with 2 manvrs
2026-07-14 13:16:16,067 get_observations: Found observation 32427 at 2026:191:03:36:02.214 with 2 manvrs
2026-07-14 13:16:16,068 get_observations: Found observation 32429 at 2026:191:08:49:19.873 with 2 manvrs
2026-07-14 13:16:16,068 get_observations: Found observation 30365 at 2026:191:12:39:16.148 with 2 manvrs
2026-07-14 13:16:16,069 get_observations: Found observation 32418 at 2026:191:17:29:37.177 with 1 manvrs
2026-07-14 13:16:16,069 get_observations: Found observation 32428 at 2026:191:20:56:38.211 with 1 manvrs
2026-07-14 13:16:16,069 get_observations: Found observation 30027 at 2026:192:01:30:45.375 with 1 manvrs
2026-07-14 13:16:16,070 get_observations: Found observation 41347 at 2026:192:17:27:39.801 with 1 manvrs
2026-07-14 13:16:16,071 get_observations: Found observation 32422 at 2026:192:18:18:17.075 with 1 manvrs
2026-07-14 13:16:16,071 get_observations: Found observation 41346 at 2026:192:21:39:59.000 with 1 manvrs
2026-07-14 13:16:16,071 get_observations: Found observation 41345 at 2026:192:23:24:57.779 with 1 manvrs
2026-07-14 13:16:16,072 get_observations: Found observation 41344 at 2026:193:03:15:40.252 with 1 manvrs
2026-07-14 13:16:16,072 get_observations: Found observation 41343 at 2026:193:04:48:45.075 with 1 manvrs
2026-07-14 13:16:16,073 get_observations: Found observation 41342 at 2026:193:06:35:40.252 with 1 manvrs
2026-07-14 13:16:16,073 get_observations: Found observation 41341 at 2026:193:08:45:40.252 with 1 manvrs
2026-07-14 13:16:16,073 get_observations: Found observation 30334 at 2026:193:10:25:17.676 with 1 manvrs
2026-07-14 13:16:16,074 get_observations: Found observation 32419 at 2026:193:15:15:55.981 with 1 manvrs
2026-07-14 13:16:16,074 get_observations: Found observation 32426 at 2026:193:18:43:14.290 with 1 manvrs
2026-07-14 13:16:16,075 get_observations: Found observation 31242 at 2026:193:23:33:45.937 with 1 manvrs
2026-07-14 13:16:16,075 get_observations: Found observation 30329 at 2026:194:03:29:18.149 with 1 manvrs
2026-07-14 13:16:16,075 get_observations: Found observation 32439 at 2026:194:09:09:55.716 with 1 manvrs
2026-07-14 13:16:16,076 get_observations: Found observation 32434 at 2026:194:13:22:13.283 with 1 manvrs
2026-07-14 13:16:16,076 get_observations: Found observation 30411 at 2026:194:18:48:10.912 with 1 manvrs
2026-07-14 13:16:16,077 get_observations: Found observation 32417 at 2026:194:22:17:09.889 with 1 manvrs
2026-07-14 13:16:16,077 get_observations: Found observation 31338 at 2026:195:03:03:37.562 with 1 manvrs
2026-07-14 13:16:16,077 get_observations: Found observation 29663 at 2026:195:06:42:48.870 with 1 manvrs
2026-07-14 13:16:16,078 get_observations: Found observation 41340 at 2026:195:13:49:14.994 with 1 manvrs
2026-07-14 13:16:16,078 get_observations: Found observation 41339 at 2026:195:15:25:50.776 with 1 manvrs
2026-07-14 13:16:16,079 get_observations: Found observation 41338 at 2026:195:16:53:06.096 with 1 manvrs
2026-07-14 13:16:16,079 main: Found 42 observations
2026-07-14 13:16:16,079 main: ********************************************************************************
2026-07-14 13:16:16,079 main: Processing observation 30419
2026-07-14 13:16:16,079 main: ********************************************************************************
2026-07-14 13:16:16,082 obs_link_from_info: Found prev obsid 32425
2026-07-14 13:16:16,085 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,087 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,090 process_obs: ObsID 30419 already processed, skipping
2026-07-14 13:16:16,090 main:
2026-07-14 13:16:16,090 main: ********************************************************************************
2026-07-14 13:16:16,090 main: Processing observation 29817
2026-07-14 13:16:16,090 main: ********************************************************************************
2026-07-14 13:16:16,091 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,092 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,096 process_obs: ObsID 29817 already processed, skipping
2026-07-14 13:16:16,096 main:
2026-07-14 13:16:16,096 main: ********************************************************************************
2026-07-14 13:16:16,096 main: Processing observation 30187
2026-07-14 13:16:16,096 main: ********************************************************************************
2026-07-14 13:16:16,097 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,098 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,101 process_obs: ObsID 30187 already processed, skipping
2026-07-14 13:16:16,101 main:
2026-07-14 13:16:16,101 main: ********************************************************************************
2026-07-14 13:16:16,101 main: Processing observation 32420
2026-07-14 13:16:16,101 main: ********************************************************************************
2026-07-14 13:16:16,102 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,103 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,106 process_obs: ObsID 32420 already processed, skipping
2026-07-14 13:16:16,107 main:
2026-07-14 13:16:16,107 main: ********************************************************************************
2026-07-14 13:16:16,107 main: Processing observation 31302
2026-07-14 13:16:16,107 main: ********************************************************************************
2026-07-14 13:16:16,108 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,109 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,112 process_obs: ObsID 31302 already processed, skipping
2026-07-14 13:16:16,112 main:
2026-07-14 13:16:16,112 main: ********************************************************************************
2026-07-14 13:16:16,112 main: Processing observation 32421
2026-07-14 13:16:16,112 main: ********************************************************************************
2026-07-14 13:16:16,113 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,114 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,118 process_obs: ObsID 32421 already processed, skipping
2026-07-14 13:16:16,118 main:
2026-07-14 13:16:16,118 main: ********************************************************************************
2026-07-14 13:16:16,118 main: Processing observation 41357
2026-07-14 13:16:16,118 main: ********************************************************************************
2026-07-14 13:16:16,119 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,120 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,123 process_obs: ObsID 41357 already processed, skipping
2026-07-14 13:16:16,123 main:
2026-07-14 13:16:16,123 main: ********************************************************************************
2026-07-14 13:16:16,123 main: Processing observation 41356
2026-07-14 13:16:16,123 main: ********************************************************************************
2026-07-14 13:16:16,124 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,125 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,128 process_obs: ObsID 41356 already processed, skipping
2026-07-14 13:16:16,128 main:
2026-07-14 13:16:16,128 main: ********************************************************************************
2026-07-14 13:16:16,129 main: Processing observation 41355
2026-07-14 13:16:16,129 main: ********************************************************************************
2026-07-14 13:16:16,129 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,130 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,134 process_obs: ObsID 41355 already processed, skipping
2026-07-14 13:16:16,134 main:
2026-07-14 13:16:16,134 main: ********************************************************************************
2026-07-14 13:16:16,134 main: Processing observation 41354
2026-07-14 13:16:16,134 main: ********************************************************************************
2026-07-14 13:16:16,135 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,136 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,139 process_obs: ObsID 41354 already processed, skipping
2026-07-14 13:16:16,139 main:
2026-07-14 13:16:16,139 main: ********************************************************************************
2026-07-14 13:16:16,139 main: Processing observation 41353
2026-07-14 13:16:16,139 main: ********************************************************************************
2026-07-14 13:16:16,140 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,141 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,144 process_obs: ObsID 41353 already processed, skipping
2026-07-14 13:16:16,144 main:
2026-07-14 13:16:16,145 main: ********************************************************************************
2026-07-14 13:16:16,145 main: Processing observation 41352
2026-07-14 13:16:16,145 main: ********************************************************************************
2026-07-14 13:16:16,145 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,147 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,150 process_obs: ObsID 41352 already processed, skipping
2026-07-14 13:16:16,150 main:
2026-07-14 13:16:16,150 main: ********************************************************************************
2026-07-14 13:16:16,150 main: Processing observation 30364
2026-07-14 13:16:16,150 main: ********************************************************************************
2026-07-14 13:16:16,151 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,152 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,155 process_obs: ObsID 30364 already processed, skipping
2026-07-14 13:16:16,155 main:
2026-07-14 13:16:16,155 main: ********************************************************************************
2026-07-14 13:16:16,155 main: Processing observation 31118
2026-07-14 13:16:16,155 main: ********************************************************************************
2026-07-14 13:16:16,156 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,157 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,160 process_obs: ObsID 31118 already processed, skipping
2026-07-14 13:16:16,160 main:
2026-07-14 13:16:16,160 main: ********************************************************************************
2026-07-14 13:16:16,160 main: Processing observation 32427
2026-07-14 13:16:16,160 main: ********************************************************************************
2026-07-14 13:16:16,161 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,162 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,165 process_obs: ObsID 32427 already processed, skipping
2026-07-14 13:16:16,165 main:
2026-07-14 13:16:16,165 main: ********************************************************************************
2026-07-14 13:16:16,165 main: Processing observation 32429
2026-07-14 13:16:16,165 main: ********************************************************************************
2026-07-14 13:16:16,166 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,167 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,171 process_obs: ObsID 32429 already processed, skipping
2026-07-14 13:16:16,171 main:
2026-07-14 13:16:16,171 main: ********************************************************************************
2026-07-14 13:16:16,171 main: Processing observation 30365
2026-07-14 13:16:16,171 main: ********************************************************************************
2026-07-14 13:16:16,172 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,173 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,176 process_obs: ObsID 30365 already processed, skipping
2026-07-14 13:16:16,176 main:
2026-07-14 13:16:16,176 main: ********************************************************************************
2026-07-14 13:16:16,176 main: Processing observation 32418
2026-07-14 13:16:16,176 main: ********************************************************************************
2026-07-14 13:16:16,177 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,178 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,181 process_obs: ObsID 32418 already processed, skipping
2026-07-14 13:16:16,181 main:
2026-07-14 13:16:16,181 main: ********************************************************************************
2026-07-14 13:16:16,181 main: Processing observation 32428
2026-07-14 13:16:16,181 main: ********************************************************************************
2026-07-14 13:16:16,182 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,183 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 13:16:16,186 process_obs: ObsID 32428 already processed, skipping
2026-07-14 13:16:16,186 main:
2026-07-14 13:16:16,186 main: ********************************************************************************
2026-07-14 13:16:16,186 main: Processing observation 30027
2026-07-14 13:16:16,186 main: ********************************************************************************
2026-07-14 13:16:16,187 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 13:16:16,188 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 12:16:21,690 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/30027/centroid_resids.pkl
2026-07-14 12:16:21,713 write_centroid_resids: Centroid residuals file exists, skipping
2026-07-14 12:16:21,714 plot_crs_time: Plot file exists, skipping
2026-07-14 12:16:21,714 plot_crs_scatter: Plot file exists, skipping
2026-07-14 12:16:23,947 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/30027/n_kalman_delta_roll.png
2026-07-14 12:16:29,396 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/30027/index.html
2026-07-14 12:16:29,429 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/30027/info.json
2026-07-14 12:16:29,488 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/30/30027/index.html
2026-07-14 12:16:29,516 main:
2026-07-14 12:16:29,516 main: ********************************************************************************
2026-07-14 12:16:29,516 main: Processing observation 41347
2026-07-14 12:16:29,516 main: ********************************************************************************
2026-07-14 12:16:29,540 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:29,568 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 12:16:29,684 process_obs: ObsID 41347 already processed, skipping
2026-07-14 12:16:29,684 main:
2026-07-14 12:16:29,684 main: ********************************************************************************
2026-07-14 12:16:29,684 main: Processing observation 32422
2026-07-14 12:16:29,684 main: ********************************************************************************
2026-07-14 12:16:29,700 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:29,729 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 12:16:29,739 process_obs: ObsID 32422 already processed, skipping
2026-07-14 12:16:29,739 main:
2026-07-14 12:16:29,740 main: ********************************************************************************
2026-07-14 12:16:29,740 main: Processing observation 41346
2026-07-14 12:16:29,740 main: ********************************************************************************
2026-07-14 12:16:29,741 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:29,742 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 12:16:29,745 process_obs: ObsID 41346 already processed, skipping
2026-07-14 12:16:29,745 main:
2026-07-14 12:16:29,745 main: ********************************************************************************
2026-07-14 12:16:29,745 main: Processing observation 41345
2026-07-14 12:16:29,745 main: ********************************************************************************
2026-07-14 12:16:29,746 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:29,747 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 12:16:29,751 process_obs: ObsID 41345 already processed, skipping
2026-07-14 12:16:29,751 main:
2026-07-14 12:16:29,751 main: ********************************************************************************
2026-07-14 12:16:29,751 main: Processing observation 41344
2026-07-14 12:16:29,751 main: ********************************************************************************
2026-07-14 12:16:29,752 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:29,753 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 12:16:29,756 process_obs: ObsID 41344 already processed, skipping
2026-07-14 12:16:29,756 main:
2026-07-14 12:16:29,756 main: ********************************************************************************
2026-07-14 12:16:29,756 main: Processing observation 41343
2026-07-14 12:16:29,756 main: ********************************************************************************
2026-07-14 12:16:29,757 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:29,758 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 12:16:29,762 process_obs: ObsID 41343 already processed, skipping
2026-07-14 12:16:29,762 main:
2026-07-14 12:16:29,762 main: ********************************************************************************
2026-07-14 12:16:29,762 main: Processing observation 41342
2026-07-14 12:16:29,762 main: ********************************************************************************
2026-07-14 12:16:29,763 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:29,764 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 12:16:29,768 process_obs: ObsID 41342 already processed, skipping
2026-07-14 12:16:29,768 main:
2026-07-14 12:16:29,768 main: ********************************************************************************
2026-07-14 12:16:29,768 main: Processing observation 41341
2026-07-14 12:16:29,768 main: ********************************************************************************
2026-07-14 12:16:29,769 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:29,770 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 12:16:29,773 process_obs: ObsID 41341 already processed, skipping
2026-07-14 12:16:29,773 main:
2026-07-14 12:16:29,773 main: ********************************************************************************
2026-07-14 12:16:29,773 main: Processing observation 30334
2026-07-14 12:16:29,773 main: ********************************************************************************
2026-07-14 12:16:29,774 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:29,775 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 12:16:29,778 process_obs: ObsID 30334 already processed, skipping
2026-07-14 12:16:29,778 main:
2026-07-14 12:16:29,778 main: ********************************************************************************
2026-07-14 12:16:29,778 main: Processing observation 32419
2026-07-14 12:16:29,778 main: ********************************************************************************
2026-07-14 12:16:29,779 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:29,780 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 12:16:29,784 process_obs: ObsID 32419 already processed, skipping
2026-07-14 12:16:29,784 main:
2026-07-14 12:16:29,784 main: ********************************************************************************
2026-07-14 12:16:29,784 main: Processing observation 32426
2026-07-14 12:16:29,784 main: ********************************************************************************
2026-07-14 12:16:29,785 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:29,786 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL0626A/last.html
2026-07-14 12:16:29,789 process_obs: ObsID 32426 already processed, skipping
2026-07-14 12:16:29,789 main:
2026-07-14 12:16:29,789 main: ********************************************************************************
2026-07-14 12:16:29,789 main: Processing observation 31242
2026-07-14 12:16:29,789 main: ********************************************************************************
2026-07-14 12:16:29,790 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:29,791 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/last.html
2026-07-14 12:16:32,348 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-14 12:16:33,067 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/31242/centroid_resids.pkl with 5 slots
2026-07-14 12:16:33,130 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/31242/centroid_resids_time.png
2026-07-14 12:16:33,669 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/31242/centroid_resids_scatter.png
2026-07-14 12:16:34,682 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/31242/n_kalman_delta_roll.png
2026-07-14 12:16:37,322 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/31242/index.html
2026-07-14 12:16:37,323 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/31242/info.json
2026-07-14 12:16:37,326 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/31/31242/index.html
2026-07-14 12:16:37,327 main:
2026-07-14 12:16:37,327 main: ********************************************************************************
2026-07-14 12:16:37,327 main: Processing observation 30329
2026-07-14 12:16:37,327 main: ********************************************************************************
2026-07-14 12:16:37,328 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:37,329 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/last.html
2026-07-14 12:16:39,232 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-14 12:16:39,547 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/30329/centroid_resids.pkl with 5 slots
2026-07-14 12:16:39,782 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/30329/centroid_resids_time.png
2026-07-14 12:16:40,082 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/30329/centroid_resids_scatter.png
2026-07-14 12:16:40,882 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/30329/n_kalman_delta_roll.png
2026-07-14 12:16:41,020 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/30329/index.html
2026-07-14 12:16:41,022 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/30329/info.json
2026-07-14 12:16:41,024 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/30/30329/index.html
2026-07-14 12:16:41,025 main:
2026-07-14 12:16:41,025 main: ********************************************************************************
2026-07-14 12:16:41,025 main: Processing observation 32439
2026-07-14 12:16:41,025 main: ********************************************************************************
2026-07-14 12:16:41,026 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:41,027 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/last.html
2026-07-14 12:16:43,087 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-14 12:16:43,371 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32439/centroid_resids.pkl with 5 slots
2026-07-14 12:16:43,420 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32439/centroid_resids_time.png
2026-07-14 12:16:43,670 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32439/centroid_resids_scatter.png
2026-07-14 12:16:44,546 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32439/n_kalman_delta_roll.png
2026-07-14 12:16:44,694 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32439/index.html
2026-07-14 12:16:44,696 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32439/info.json
2026-07-14 12:16:44,698 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32439/index.html
2026-07-14 12:16:44,699 main:
2026-07-14 12:16:44,699 main: ********************************************************************************
2026-07-14 12:16:44,699 main: Processing observation 32434
2026-07-14 12:16:44,699 main: ********************************************************************************
2026-07-14 12:16:44,700 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:44,701 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/last.html
2026-07-14 12:16:46,753 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-14 12:16:47,053 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32434/centroid_resids.pkl with 5 slots
2026-07-14 12:16:47,105 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32434/centroid_resids_time.png
2026-07-14 12:16:47,403 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32434/centroid_resids_scatter.png
2026-07-14 12:16:48,490 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32434/n_kalman_delta_roll.png
2026-07-14 12:16:48,627 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32434/index.html
2026-07-14 12:16:48,629 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32434/info.json
2026-07-14 12:16:48,631 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32434/index.html
2026-07-14 12:16:48,632 main:
2026-07-14 12:16:48,632 main: ********************************************************************************
2026-07-14 12:16:48,632 main: Processing observation 30411
2026-07-14 12:16:48,632 main: ********************************************************************************
2026-07-14 12:16:48,633 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:48,634 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/last.html
2026-07-14 12:16:50,880 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-14 12:16:51,176 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/30411/centroid_resids.pkl with 5 slots
2026-07-14 12:16:51,223 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/30411/centroid_resids_time.png
2026-07-14 12:16:51,448 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/30411/centroid_resids_scatter.png
2026-07-14 12:16:52,105 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/30411/n_kalman_delta_roll.png
2026-07-14 12:16:52,244 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/30411/index.html
2026-07-14 12:16:52,245 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/30411/info.json
2026-07-14 12:16:52,248 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/30/30411/index.html
2026-07-14 12:16:52,248 main:
2026-07-14 12:16:52,248 main: ********************************************************************************
2026-07-14 12:16:52,249 main: Processing observation 32417
2026-07-14 12:16:52,249 main: ********************************************************************************
2026-07-14 12:16:52,250 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:52,251 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/last.html
2026-07-14 12:16:54,115 __init__: ObsID 32417 has no maneuver event in telemetry, skipping
2026-07-14 12:16:54,116 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32417/index.html
2026-07-14 12:16:54,120 info: could not get attribute kalman_start for obsid 32417
2026-07-14 12:16:54,120 info: could not get attribute kalman_stop for obsid 32417
2026-07-14 12:16:54,120 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/32417/info.json
2026-07-14 12:16:54,121 main:
2026-07-14 12:16:54,121 main: ********************************************************************************
2026-07-14 12:16:54,121 main: Processing observation 31338
2026-07-14 12:16:54,121 main: ********************************************************************************
2026-07-14 12:16:54,122 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:54,123 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/last.html
2026-07-14 12:16:56,453 __init__: ObsID 31338 has no maneuver event in telemetry, skipping
2026-07-14 12:16:56,454 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/31338/index.html
2026-07-14 12:16:56,459 info: could not get attribute kalman_start for obsid 31338
2026-07-14 12:16:56,459 info: could not get attribute kalman_stop for obsid 31338
2026-07-14 12:16:56,459 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/31338/info.json
2026-07-14 12:16:56,460 main:
2026-07-14 12:16:56,460 main: ********************************************************************************
2026-07-14 12:16:56,460 main: Processing observation 29663
2026-07-14 12:16:56,460 main: ********************************************************************************
2026-07-14 12:16:56,462 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:56,463 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/last.html
2026-07-14 12:16:58,425 __init__: ObsID 29663 has no maneuver event in telemetry, skipping
2026-07-14 12:16:58,426 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/29663/index.html
2026-07-14 12:16:58,430 info: could not get attribute kalman_start for obsid 29663
2026-07-14 12:16:58,430 info: could not get attribute kalman_stop for obsid 29663
2026-07-14 12:16:58,430 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/29663/info.json
2026-07-14 12:16:58,431 main:
2026-07-14 12:16:58,431 main: ********************************************************************************
2026-07-14 12:16:58,431 main: Processing observation 41340
2026-07-14 12:16:58,431 main: ********************************************************************************
2026-07-14 12:16:58,432 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:16:58,434 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/last.html
2026-07-14 12:17:00,194 __init__: ObsID 41340 has no maneuver event in telemetry, skipping
2026-07-14 12:17:00,195 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/41340/index.html
2026-07-14 12:17:00,199 info: could not get attribute kalman_start for obsid 41340
2026-07-14 12:17:00,199 info: could not get attribute kalman_stop for obsid 41340
2026-07-14 12:17:00,199 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/41340/info.json
2026-07-14 12:17:00,200 main:
2026-07-14 12:17:00,200 main: ********************************************************************************
2026-07-14 12:17:00,200 main: Processing observation 41339
2026-07-14 12:17:00,200 main: ********************************************************************************
2026-07-14 12:17:00,203 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:17:00,205 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/last.html
2026-07-14 12:17:03,245 __init__: ObsID 41339 has no maneuver event in telemetry, skipping
2026-07-14 12:17:03,245 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/41339/index.html
2026-07-14 12:17:03,250 info: could not get attribute kalman_start for obsid 41339
2026-07-14 12:17:03,250 info: could not get attribute kalman_stop for obsid 41339
2026-07-14 12:17:03,250 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/41339/info.json
2026-07-14 12:17:03,254 main:
2026-07-14 12:17:03,254 main: ********************************************************************************
2026-07-14 12:17:03,254 main: Processing observation 41338
2026-07-14 12:17:03,254 main: ********************************************************************************
2026-07-14 12:17:03,255 obs_link_from_info: No /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/41338/info.json file found, obs=None
2026-07-14 12:17:03,259 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-14 12:17:03,286 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/last.html
2026-07-14 12:17:07,556 __init__: ObsID 41338 has no maneuver event in telemetry, skipping
2026-07-14 12:17:07,557 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/41338/index.html
2026-07-14 12:17:07,563 info: could not get attribute kalman_start for obsid 41338
2026-07-14 12:17:07,563 info: could not get attribute kalman_stop for obsid 41338
2026-07-14 12:17:07,563 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUL1326A/41338/info.json
2026-07-14 12:17:07,564 main:
2026-07-14 13:17:09,098 run_check: Starting centroid dashboard continuity check
2026-07-14 13:17:09,098 collect_observations: Scanning observations in /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports
2026-07-14 13:17:09,099 collect_observations: Processing year 2020
2026-07-14 13:17:10,849 collect_observations: Processing year 2021
2026-07-14 13:17:13,349 collect_observations: Processing year 2022
2026-07-14 13:17:16,467 collect_observations: Processing year 2023
2026-07-14 13:17:19,375 collect_observations: Processing year 2024
2026-07-14 13:17:22,126 collect_observations: Processing year 2025
2026-07-14 13:17:25,001 collect_observations: Processing year 2026
2026-07-14 13:17:26,266 collect_observations: Collected 14208 observations from 4-digit years
2026-07-14 13:17:26,269 sort_observations_by_date: Sorted observations by date
2026-07-14 13:17:26,270 check_continuity: Starting continuity check from most recent: ObsID 41338 (JUL1326A) at 2026:195:16:20:16.524
2026-07-14 13:17:26,345 check_continuity: Reached chain start at ObsID 47337 (MAR3020A) at 2020:092:05:29:19.712
2026-07-14 13:17:26,345 check_continuity: Chain complete: visited 14208 observations out of 14208 total
2026-07-14 13:17:26,346 run_check: ✓ Observation chain continuity check PASSED
2026-07-14 13:17:26,346 run_check: Summary: 14208 observations from 4-digit years, all properly linked