Prev
Index
Next
centroid_dashboard: Wed Jul 1 13:17:43 2026
File: /proj/sot/ska/data/ska_trend/centroid_dashboard/logs/daily.0/centroid_dashboard.log
| Task | Status | Age | MaxAge | Last Date |
|---|
| centroid_dashboard |
OK
|
0.89 | 1.0 |
Wed Jul 1 13:17:43 2026 |
No errors
File contents:
2026-07-01 13:16:10,575 main: Processing from 2026:175:17:16:10.574 to 2026:182:17:16:10.574
2026-07-01 13:16:17,770 get_observations: Found observation 31405 at 2026:175:17:44:45.401 with 1 manvrs
2026-07-01 13:16:17,771 get_observations: Found observation 31406 at 2026:176:00:49:55.401 with 1 manvrs
2026-07-01 13:16:17,771 get_observations: Found observation 31611 at 2026:176:08:04:17.279 with 1 manvrs
2026-07-01 13:16:17,771 get_observations: Found observation 30512 at 2026:176:10:14:45.557 with 1 manvrs
2026-07-01 13:16:17,772 get_observations: Found observation 41390 at 2026:176:23:41:47.248 with 1 manvrs
2026-07-01 13:16:17,772 get_observations: Found observation 41389 at 2026:177:00:55:40.252 with 1 manvrs
2026-07-01 13:16:17,773 get_observations: Found observation 41388 at 2026:177:03:25:40.252 with 1 manvrs
2026-07-01 13:16:17,773 get_observations: Found observation 41387 at 2026:177:07:25:40.252 with 1 manvrs
2026-07-01 13:16:17,773 get_observations: Found observation 41386 at 2026:177:09:55:40.252 with 1 manvrs
2026-07-01 13:16:17,774 get_observations: Found observation 31483 at 2026:177:13:31:23.009 with 1 manvrs
2026-07-01 13:16:17,774 get_observations: Found observation 31384 at 2026:177:19:05:05.114 with 1 manvrs
2026-07-01 13:16:17,775 get_observations: Found observation 31385 at 2026:177:19:53:51.437 with 1 manvrs
2026-07-01 13:16:17,775 get_observations: Found observation 31596 at 2026:177:21:07:14.768 with 1 manvrs
2026-07-01 13:16:17,775 get_observations: Found observation 29829 at 2026:178:02:27:51.034 with 1 manvrs
2026-07-01 13:16:17,776 get_observations: Found observation 31597 at 2026:178:07:35:09.169 with 1 manvrs
2026-07-01 13:16:17,776 get_observations: Found observation 31408 at 2026:178:12:04:53.765 with 1 manvrs
2026-07-01 13:16:17,777 get_observations: Found observation 32335 at 2026:178:16:58:23.394 with 1 manvrs
2026-07-01 13:16:17,777 get_observations: Found observation 32403 at 2026:179:01:40:03.076 with 1 manvrs
2026-07-01 13:16:17,777 get_observations: Found observation 32404 at 2026:179:05:08:22.265 with 1 manvrs
2026-07-01 13:16:17,778 get_observations: Found observation 32406 at 2026:179:11:14:33.099 with 1 manvrs
2026-07-01 13:16:17,778 get_observations: Found observation 41385 at 2026:179:16:14:34.593 with 1 manvrs
2026-07-01 13:16:17,779 get_observations: Found observation 41384 at 2026:179:17:48:26.289 with 1 manvrs
2026-07-01 13:16:17,779 get_observations: Found observation 41383 at 2026:179:19:28:40.940 with 1 manvrs
2026-07-01 13:16:17,780 get_observations: Found observation 41382 at 2026:179:20:56:58.149 with 1 manvrs
2026-07-01 13:16:17,780 get_observations: Found observation 41381 at 2026:180:01:55:40.252 with 1 manvrs
2026-07-01 13:16:17,780 get_observations: Found observation 41380 at 2026:180:03:25:40.252 with 1 manvrs
2026-07-01 13:16:17,781 get_observations: Found observation 29996 at 2026:180:04:22:57.002 with 1 manvrs
2026-07-01 13:16:17,781 get_observations: Found observation 30123 at 2026:180:10:49:24.483 with 2 manvrs
2026-07-01 13:16:17,782 get_observations: Found observation 30118 at 2026:180:15:46:31.682 with 2 manvrs
2026-07-01 13:16:17,782 get_observations: Found observation 32410 at 2026:181:00:40:55.419 with 1 manvrs
2026-07-01 13:16:17,782 get_observations: Found observation 30261 at 2026:181:06:38:29.029 with 1 manvrs
2026-07-01 13:16:17,783 get_observations: Found observation 32411 at 2026:181:12:52:41.524 with 1 manvrs
2026-07-01 13:16:17,783 get_observations: Found observation 32409 at 2026:181:17:55:53.879 with 2 manvrs
2026-07-01 13:16:17,784 get_observations: Found observation 31876 at 2026:181:23:56:29.103 with 1 manvrs
2026-07-01 13:16:17,784 get_observations: Found observation 41376 at 2026:182:08:29:16.505 with 1 manvrs
2026-07-01 13:16:17,784 get_observations: Found observation 41375 at 2026:182:10:13:24.295 with 1 manvrs
2026-07-01 13:16:17,785 get_observations: Found observation 41374 at 2026:182:12:45:40.252 with 1 manvrs
2026-07-01 13:16:17,785 get_observations: Found observation 41373 at 2026:182:16:25:40.252 with 1 manvrs
2026-07-01 13:16:17,786 main: Found 38 observations
2026-07-01 13:16:17,786 main: ********************************************************************************
2026-07-01 13:16:17,786 main: Processing observation 31405
2026-07-01 13:16:17,786 main: ********************************************************************************
2026-07-01 13:16:17,790 obs_link_from_info: Found prev obsid 31403
2026-07-01 13:16:17,793 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 13:16:17,794 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 13:16:17,797 process_obs: ObsID 31405 already processed, skipping
2026-07-01 13:16:17,797 main:
2026-07-01 13:16:17,797 main: ********************************************************************************
2026-07-01 13:16:17,797 main: Processing observation 31406
2026-07-01 13:16:17,797 main: ********************************************************************************
2026-07-01 13:16:17,798 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 13:16:17,799 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 13:16:17,803 process_obs: ObsID 31406 already processed, skipping
2026-07-01 13:16:17,803 main:
2026-07-01 13:16:17,803 main: ********************************************************************************
2026-07-01 13:16:17,803 main: Processing observation 31611
2026-07-01 13:16:17,803 main: ********************************************************************************
2026-07-01 13:16:17,804 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 13:16:17,805 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 13:16:17,808 process_obs: ObsID 31611 already processed, skipping
2026-07-01 13:16:17,808 main:
2026-07-01 13:16:17,808 main: ********************************************************************************
2026-07-01 13:16:17,808 main: Processing observation 30512
2026-07-01 13:16:17,808 main: ********************************************************************************
2026-07-01 13:16:17,809 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 13:16:17,810 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:20,161 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/30512/centroid_resids.pkl
2026-07-01 12:16:20,208 write_centroid_resids: Centroid residuals file exists, skipping
2026-07-01 12:16:20,208 plot_crs_time: Plot file exists, skipping
2026-07-01 12:16:20,208 plot_crs_scatter: Plot file exists, skipping
2026-07-01 12:16:21,492 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/30512/n_kalman_delta_roll.png
2026-07-01 12:16:26,285 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/30512/index.html
2026-07-01 12:16:26,289 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/30512/info.json
2026-07-01 12:16:26,308 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/30/30512/index.html
2026-07-01 12:16:26,313 main:
2026-07-01 12:16:26,313 main: ********************************************************************************
2026-07-01 12:16:26,313 main: Processing observation 41390
2026-07-01 12:16:26,313 main: ********************************************************************************
2026-07-01 12:16:26,319 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,324 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:26,338 process_obs: ObsID 41390 already processed, skipping
2026-07-01 12:16:26,339 main:
2026-07-01 12:16:26,339 main: ********************************************************************************
2026-07-01 12:16:26,339 main: Processing observation 41389
2026-07-01 12:16:26,339 main: ********************************************************************************
2026-07-01 12:16:26,344 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,348 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:26,363 process_obs: ObsID 41389 already processed, skipping
2026-07-01 12:16:26,363 main:
2026-07-01 12:16:26,363 main: ********************************************************************************
2026-07-01 12:16:26,363 main: Processing observation 41388
2026-07-01 12:16:26,363 main: ********************************************************************************
2026-07-01 12:16:26,369 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,374 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:26,390 process_obs: ObsID 41388 already processed, skipping
2026-07-01 12:16:26,390 main:
2026-07-01 12:16:26,390 main: ********************************************************************************
2026-07-01 12:16:26,390 main: Processing observation 41387
2026-07-01 12:16:26,390 main: ********************************************************************************
2026-07-01 12:16:26,396 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,400 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:26,415 process_obs: ObsID 41387 already processed, skipping
2026-07-01 12:16:26,416 main:
2026-07-01 12:16:26,416 main: ********************************************************************************
2026-07-01 12:16:26,416 main: Processing observation 41386
2026-07-01 12:16:26,416 main: ********************************************************************************
2026-07-01 12:16:26,422 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,427 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:26,444 process_obs: ObsID 41386 already processed, skipping
2026-07-01 12:16:26,444 main:
2026-07-01 12:16:26,444 main: ********************************************************************************
2026-07-01 12:16:26,444 main: Processing observation 31483
2026-07-01 12:16:26,444 main: ********************************************************************************
2026-07-01 12:16:26,449 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,453 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:26,465 process_obs: ObsID 31483 already processed, skipping
2026-07-01 12:16:26,465 main:
2026-07-01 12:16:26,465 main: ********************************************************************************
2026-07-01 12:16:26,465 main: Processing observation 31384
2026-07-01 12:16:26,465 main: ********************************************************************************
2026-07-01 12:16:26,471 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,475 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:26,490 process_obs: ObsID 31384 already processed, skipping
2026-07-01 12:16:26,490 main:
2026-07-01 12:16:26,490 main: ********************************************************************************
2026-07-01 12:16:26,490 main: Processing observation 31385
2026-07-01 12:16:26,490 main: ********************************************************************************
2026-07-01 12:16:26,497 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,502 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:26,519 process_obs: ObsID 31385 already processed, skipping
2026-07-01 12:16:26,519 main:
2026-07-01 12:16:26,519 main: ********************************************************************************
2026-07-01 12:16:26,519 main: Processing observation 31596
2026-07-01 12:16:26,519 main: ********************************************************************************
2026-07-01 12:16:26,522 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,526 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:26,541 process_obs: ObsID 31596 already processed, skipping
2026-07-01 12:16:26,541 main:
2026-07-01 12:16:26,541 main: ********************************************************************************
2026-07-01 12:16:26,541 main: Processing observation 29829
2026-07-01 12:16:26,541 main: ********************************************************************************
2026-07-01 12:16:26,546 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,550 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:26,568 process_obs: ObsID 29829 already processed, skipping
2026-07-01 12:16:26,568 main:
2026-07-01 12:16:26,568 main: ********************************************************************************
2026-07-01 12:16:26,568 main: Processing observation 31597
2026-07-01 12:16:26,568 main: ********************************************************************************
2026-07-01 12:16:26,573 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,578 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:26,591 process_obs: ObsID 31597 already processed, skipping
2026-07-01 12:16:26,591 main:
2026-07-01 12:16:26,591 main: ********************************************************************************
2026-07-01 12:16:26,591 main: Processing observation 31408
2026-07-01 12:16:26,591 main: ********************************************************************************
2026-07-01 12:16:26,596 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,601 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:26,618 process_obs: ObsID 31408 already processed, skipping
2026-07-01 12:16:26,619 main:
2026-07-01 12:16:26,619 main: ********************************************************************************
2026-07-01 12:16:26,619 main: Processing observation 32335
2026-07-01 12:16:26,619 main: ********************************************************************************
2026-07-01 12:16:26,624 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:26,629 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:33,748 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32335/centroid_resids.pkl
2026-07-01 12:16:33,756 write_centroid_resids: Centroid residuals file exists, skipping
2026-07-01 12:16:33,756 plot_crs_time: Plot file exists, skipping
2026-07-01 12:16:33,756 plot_crs_scatter: Plot file exists, skipping
2026-07-01 12:16:34,588 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32335/n_kalman_delta_roll.png
2026-07-01 12:16:39,102 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32335/index.html
2026-07-01 12:16:39,106 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32335/info.json
2026-07-01 12:16:39,121 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32335/index.html
2026-07-01 12:16:39,122 main:
2026-07-01 12:16:39,122 main: ********************************************************************************
2026-07-01 12:16:39,122 main: Processing observation 32403
2026-07-01 12:16:39,122 main: ********************************************************************************
2026-07-01 12:16:39,123 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:39,125 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:39,129 process_obs: ObsID 32403 already processed, skipping
2026-07-01 12:16:39,129 main:
2026-07-01 12:16:39,129 main: ********************************************************************************
2026-07-01 12:16:39,129 main: Processing observation 32404
2026-07-01 12:16:39,129 main: ********************************************************************************
2026-07-01 12:16:39,130 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:39,131 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:39,136 process_obs: ObsID 32404 already processed, skipping
2026-07-01 12:16:39,136 main:
2026-07-01 12:16:39,136 main: ********************************************************************************
2026-07-01 12:16:39,136 main: Processing observation 32406
2026-07-01 12:16:39,136 main: ********************************************************************************
2026-07-01 12:16:39,139 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:39,140 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:39,147 process_obs: ObsID 32406 already processed, skipping
2026-07-01 12:16:39,147 main:
2026-07-01 12:16:39,147 main: ********************************************************************************
2026-07-01 12:16:39,147 main: Processing observation 41385
2026-07-01 12:16:39,147 main: ********************************************************************************
2026-07-01 12:16:39,148 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:39,150 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-01 12:16:39,155 process_obs: ObsID 41385 already processed, skipping
2026-07-01 12:16:39,155 main:
2026-07-01 12:16:39,155 main: ********************************************************************************
2026-07-01 12:16:39,155 main: Processing observation 41384
2026-07-01 12:16:39,155 main: ********************************************************************************
2026-07-01 12:16:39,157 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:39,159 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:16:39,163 process_obs: ObsID 41384 already processed, skipping
2026-07-01 12:16:39,163 main:
2026-07-01 12:16:39,163 main: ********************************************************************************
2026-07-01 12:16:39,163 main: Processing observation 41383
2026-07-01 12:16:39,163 main: ********************************************************************************
2026-07-01 12:16:39,164 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:39,168 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:16:39,174 process_obs: ObsID 41383 already processed, skipping
2026-07-01 12:16:39,174 main:
2026-07-01 12:16:39,174 main: ********************************************************************************
2026-07-01 12:16:39,174 main: Processing observation 41382
2026-07-01 12:16:39,174 main: ********************************************************************************
2026-07-01 12:16:39,175 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:39,176 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:16:39,180 process_obs: ObsID 41382 already processed, skipping
2026-07-01 12:16:39,180 main:
2026-07-01 12:16:39,180 main: ********************************************************************************
2026-07-01 12:16:39,180 main: Processing observation 41381
2026-07-01 12:16:39,180 main: ********************************************************************************
2026-07-01 12:16:39,181 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:39,183 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:16:39,191 process_obs: ObsID 41381 already processed, skipping
2026-07-01 12:16:39,191 main:
2026-07-01 12:16:39,191 main: ********************************************************************************
2026-07-01 12:16:39,191 main: Processing observation 41380
2026-07-01 12:16:39,191 main: ********************************************************************************
2026-07-01 12:16:39,193 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:39,196 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:16:39,201 process_obs: ObsID 41380 already processed, skipping
2026-07-01 12:16:39,201 main:
2026-07-01 12:16:39,201 main: ********************************************************************************
2026-07-01 12:16:39,201 main: Processing observation 29996
2026-07-01 12:16:39,201 main: ********************************************************************************
2026-07-01 12:16:39,203 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:39,204 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:16:43,696 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29996/centroid_resids.pkl
2026-07-01 12:16:43,707 write_centroid_resids: Centroid residuals file exists, skipping
2026-07-01 12:16:43,707 plot_crs_time: Plot file exists, skipping
2026-07-01 12:16:43,707 plot_crs_scatter: Plot file exists, skipping
2026-07-01 12:16:44,442 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29996/n_kalman_delta_roll.png
2026-07-01 12:16:44,744 get_aberration_correction: More than one entry for 29996 in /proj/sot/ska3/aca/data/mpcrit1/mplogs/2026/JUN2926/oflsa/output/JUN2926A_ManErr.txt. Skipping aber correction
2026-07-01 12:16:49,136 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29996/index.html
2026-07-01 12:16:49,138 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29996/info.json
2026-07-01 12:16:49,151 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/29/29996/index.html
2026-07-01 12:16:49,152 main:
2026-07-01 12:16:49,152 main: ********************************************************************************
2026-07-01 12:16:49,152 main: Processing observation 30123
2026-07-01 12:16:49,152 main: ********************************************************************************
2026-07-01 12:16:49,153 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:49,155 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:16:50,955 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30123/centroid_resids.pkl
2026-07-01 12:16:50,959 write_centroid_resids: Centroid residuals file exists, skipping
2026-07-01 12:16:50,959 plot_crs_time: Plot file exists, skipping
2026-07-01 12:16:50,960 plot_crs_scatter: Plot file exists, skipping
2026-07-01 12:16:51,524 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30123/n_kalman_delta_roll.png
2026-07-01 12:16:51,659 get_aberration_correction: More than one entry for 30123 in /proj/sot/ska3/aca/data/mpcrit1/mplogs/2026/JUN2926/oflsa/output/JUN2926A_ManErr.txt. Skipping aber correction
2026-07-01 12:16:51,670 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30123/index.html
2026-07-01 12:16:51,671 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30123/info.json
2026-07-01 12:16:51,689 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/30/30123/index.html
2026-07-01 12:16:51,690 main:
2026-07-01 12:16:51,690 main: ********************************************************************************
2026-07-01 12:16:51,690 main: Processing observation 30118
2026-07-01 12:16:51,690 main: ********************************************************************************
2026-07-01 12:16:51,691 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:51,692 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:16:53,498 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30118/centroid_resids.pkl
2026-07-01 12:16:53,506 write_centroid_resids: Centroid residuals file exists, skipping
2026-07-01 12:16:53,506 plot_crs_time: Plot file exists, skipping
2026-07-01 12:16:53,506 plot_crs_scatter: Plot file exists, skipping
2026-07-01 12:16:54,200 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30118/n_kalman_delta_roll.png
2026-07-01 12:16:54,517 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30118/index.html
2026-07-01 12:16:54,519 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30118/info.json
2026-07-01 12:16:54,575 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/30/30118/index.html
2026-07-01 12:16:54,576 main:
2026-07-01 12:16:54,576 main: ********************************************************************************
2026-07-01 12:16:54,576 main: Processing observation 32410
2026-07-01 12:16:54,576 main: ********************************************************************************
2026-07-01 12:16:54,577 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:54,578 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:16:56,378 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-01 12:16:56,759 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32410/centroid_resids.pkl with 5 slots
2026-07-01 12:16:56,811 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32410/centroid_resids_time.png
2026-07-01 12:16:57,120 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32410/centroid_resids_scatter.png
2026-07-01 12:16:57,841 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32410/n_kalman_delta_roll.png
2026-07-01 12:16:57,980 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32410/index.html
2026-07-01 12:16:57,982 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32410/info.json
2026-07-01 12:16:57,985 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32410/index.html
2026-07-01 12:16:57,986 main:
2026-07-01 12:16:57,986 main: ********************************************************************************
2026-07-01 12:16:57,986 main: Processing observation 30261
2026-07-01 12:16:57,986 main: ********************************************************************************
2026-07-01 12:16:57,987 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:16:57,988 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:17:00,093 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-01 12:17:00,412 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30261/centroid_resids.pkl with 5 slots
2026-07-01 12:17:00,469 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30261/centroid_resids_time.png
2026-07-01 12:17:00,781 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30261/centroid_resids_scatter.png
2026-07-01 12:17:01,609 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30261/n_kalman_delta_roll.png
2026-07-01 12:17:01,785 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30261/index.html
2026-07-01 12:17:01,787 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30261/info.json
2026-07-01 12:17:01,790 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/30/30261/index.html
2026-07-01 12:17:01,791 main:
2026-07-01 12:17:01,791 main: ********************************************************************************
2026-07-01 12:17:01,791 main: Processing observation 32411
2026-07-01 12:17:01,791 main: ********************************************************************************
2026-07-01 12:17:01,792 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:17:01,794 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:17:04,461 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-01 12:17:04,737 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32411/centroid_resids.pkl with 5 slots
2026-07-01 12:17:04,785 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32411/centroid_resids_time.png
2026-07-01 12:17:05,056 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32411/centroid_resids_scatter.png
2026-07-01 12:17:05,728 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32411/n_kalman_delta_roll.png
2026-07-01 12:17:05,877 get_aberration_correction: More than one entry for 32411 in /proj/sot/ska3/aca/data/mpcrit1/mplogs/2026/JUN2926/oflsa/output/JUN2926A_ManErr.txt. Skipping aber correction
2026-07-01 12:17:05,893 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32411/index.html
2026-07-01 12:17:05,902 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32411/info.json
2026-07-01 12:17:05,904 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32411/index.html
2026-07-01 12:17:05,905 main:
2026-07-01 12:17:05,905 main: ********************************************************************************
2026-07-01 12:17:05,905 main: Processing observation 32409
2026-07-01 12:17:05,905 main: ********************************************************************************
2026-07-01 12:17:05,909 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:17:05,910 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:17:10,000 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-01 12:17:10,331 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32409/centroid_resids.pkl with 5 slots
2026-07-01 12:17:10,396 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32409/centroid_resids_time.png
2026-07-01 12:17:10,691 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32409/centroid_resids_scatter.png
2026-07-01 12:17:11,438 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32409/n_kalman_delta_roll.png
2026-07-01 12:17:11,586 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32409/index.html
2026-07-01 12:17:11,588 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32409/info.json
2026-07-01 12:17:11,590 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32409/index.html
2026-07-01 12:17:11,591 main:
2026-07-01 12:17:11,591 main: ********************************************************************************
2026-07-01 12:17:11,591 main: Processing observation 31876
2026-07-01 12:17:11,591 main: ********************************************************************************
2026-07-01 12:17:11,592 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:17:11,593 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:17:13,533 __init__: ObsID 31876 has insufficient telemetry, skipping
2026-07-01 12:17:13,533 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31876/index.html
2026-07-01 12:17:13,538 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31876/info.json
2026-07-01 12:17:13,539 main:
2026-07-01 12:17:13,539 main: ********************************************************************************
2026-07-01 12:17:13,539 main: Processing observation 41376
2026-07-01 12:17:13,539 main: ********************************************************************************
2026-07-01 12:17:13,540 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:17:13,541 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:17:15,644 __init__: ObsID 41376 has insufficient telemetry, skipping
2026-07-01 12:17:15,645 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41376/index.html
2026-07-01 12:17:15,651 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41376/info.json
2026-07-01 12:17:15,652 main:
2026-07-01 12:17:15,652 main: ********************************************************************************
2026-07-01 12:17:15,652 main: Processing observation 41375
2026-07-01 12:17:15,652 main: ********************************************************************************
2026-07-01 12:17:15,653 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:17:15,654 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:17:17,711 __init__: ObsID 41375 has no maneuver event in telemetry, skipping
2026-07-01 12:17:17,711 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41375/index.html
2026-07-01 12:17:17,716 info: could not get attribute kalman_start for obsid 41375
2026-07-01 12:17:17,716 info: could not get attribute kalman_stop for obsid 41375
2026-07-01 12:17:17,716 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41375/info.json
2026-07-01 12:17:17,717 main:
2026-07-01 12:17:17,717 main: ********************************************************************************
2026-07-01 12:17:17,717 main: Processing observation 41374
2026-07-01 12:17:17,717 main: ********************************************************************************
2026-07-01 12:17:17,718 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:17:17,719 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:17:19,729 __init__: ObsID 41374 has no maneuver event in telemetry, skipping
2026-07-01 12:17:19,730 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41374/index.html
2026-07-01 12:17:19,735 info: could not get attribute kalman_start for obsid 41374
2026-07-01 12:17:19,735 info: could not get attribute kalman_stop for obsid 41374
2026-07-01 12:17:19,735 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41374/info.json
2026-07-01 12:17:19,736 main:
2026-07-01 12:17:19,736 main: ********************************************************************************
2026-07-01 12:17:19,736 main: Processing observation 41373
2026-07-01 12:17:19,736 main: ********************************************************************************
2026-07-01 12:17:19,736 obs_link_from_info: No /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41373/info.json file found, obs=None
2026-07-01 12:17:19,737 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-01 12:17:19,738 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-01 12:17:21,672 __init__: ObsID 41373 has no maneuver event in telemetry, skipping
2026-07-01 12:17:21,672 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41373/index.html
2026-07-01 12:17:21,677 info: could not get attribute kalman_start for obsid 41373
2026-07-01 12:17:21,677 info: could not get attribute kalman_stop for obsid 41373
2026-07-01 12:17:21,677 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41373/info.json
2026-07-01 12:17:21,678 main:
2026-07-01 13:17:22,990 run_check: Starting centroid dashboard continuity check
2026-07-01 13:17:22,990 collect_observations: Scanning observations in /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports
2026-07-01 13:17:23,007 collect_observations: Processing year 2020
2026-07-01 13:17:25,188 collect_observations: Processing year 2021
2026-07-01 13:17:28,110 collect_observations: Processing year 2022
2026-07-01 13:17:31,599 collect_observations: Processing year 2023
2026-07-01 13:17:34,964 collect_observations: Processing year 2024
2026-07-01 13:17:38,231 collect_observations: Processing year 2025
2026-07-01 13:17:41,882 collect_observations: Processing year 2026
2026-07-01 13:17:43,405 collect_observations: Collected 14128 observations from 4-digit years
2026-07-01 13:17:43,409 sort_observations_by_date: Sorted observations by date
2026-07-01 13:17:43,409 check_continuity: Starting continuity check from most recent: ObsID 41373 (JUN2926A) at 2026:182:15:51:46.338
2026-07-01 13:17:43,428 check_continuity: Reached chain start at ObsID 47337 (MAR3020A) at 2020:092:05:29:19.712
2026-07-01 13:17:43,428 check_continuity: Chain complete: visited 14128 observations out of 14128 total
2026-07-01 13:17:43,429 run_check: ✓ Observation chain continuity check PASSED
2026-07-01 13:17:43,429 run_check: Summary: 14128 observations from 4-digit years, all properly linked