Prev
Index
Next
centroid_dashboard: Fri Jul 3 13:17:31 2026
File: /proj/sot/ska/data/ska_trend/centroid_dashboard/logs/daily.0/centroid_dashboard.log
| Task | Status | Age | MaxAge | Last Date |
|---|
| centroid_dashboard |
OK
|
0.89 | 1.0 |
Fri Jul 3 13:17:31 2026 |
No errors
File contents:
2026-07-03 13:16:12,326 main: Processing from 2026:177:17:16:12.325 to 2026:184:17:16:12.325
2026-07-03 13:16:20,341 get_observations: Found observation 31384 at 2026:177:19:05:05.114 with 1 manvrs
2026-07-03 13:16:20,341 get_observations: Found observation 31385 at 2026:177:19:53:51.437 with 1 manvrs
2026-07-03 13:16:20,341 get_observations: Found observation 31596 at 2026:177:21:07:14.768 with 1 manvrs
2026-07-03 13:16:20,342 get_observations: Found observation 29829 at 2026:178:02:27:51.034 with 1 manvrs
2026-07-03 13:16:20,342 get_observations: Found observation 31597 at 2026:178:07:35:09.169 with 1 manvrs
2026-07-03 13:16:20,343 get_observations: Found observation 31408 at 2026:178:12:04:53.765 with 1 manvrs
2026-07-03 13:16:20,343 get_observations: Found observation 32335 at 2026:178:16:58:23.394 with 1 manvrs
2026-07-03 13:16:20,343 get_observations: Found observation 32403 at 2026:179:01:40:03.076 with 1 manvrs
2026-07-03 13:16:20,344 get_observations: Found observation 32404 at 2026:179:05:08:22.265 with 1 manvrs
2026-07-03 13:16:20,344 get_observations: Found observation 32406 at 2026:179:11:14:33.099 with 1 manvrs
2026-07-03 13:16:20,345 get_observations: Found observation 41385 at 2026:179:16:14:34.593 with 1 manvrs
2026-07-03 13:16:20,345 get_observations: Found observation 41384 at 2026:179:17:48:26.289 with 1 manvrs
2026-07-03 13:16:20,345 get_observations: Found observation 41383 at 2026:179:19:28:40.940 with 1 manvrs
2026-07-03 13:16:20,346 get_observations: Found observation 41382 at 2026:179:20:56:58.149 with 1 manvrs
2026-07-03 13:16:20,346 get_observations: Found observation 41381 at 2026:180:01:55:40.252 with 1 manvrs
2026-07-03 13:16:20,347 get_observations: Found observation 41380 at 2026:180:03:25:40.252 with 1 manvrs
2026-07-03 13:16:20,347 get_observations: Found observation 29996 at 2026:180:04:22:57.002 with 1 manvrs
2026-07-03 13:16:20,347 get_observations: Found observation 30123 at 2026:180:10:49:24.483 with 2 manvrs
2026-07-03 13:16:20,348 get_observations: Found observation 30118 at 2026:180:15:46:31.682 with 2 manvrs
2026-07-03 13:16:20,348 get_observations: Found observation 32410 at 2026:181:00:40:55.419 with 1 manvrs
2026-07-03 13:16:20,349 get_observations: Found observation 30261 at 2026:181:06:38:29.029 with 1 manvrs
2026-07-03 13:16:20,349 get_observations: Found observation 32411 at 2026:181:12:52:41.524 with 1 manvrs
2026-07-03 13:16:20,350 get_observations: Found observation 32409 at 2026:181:17:55:53.879 with 2 manvrs
2026-07-03 13:16:20,350 get_observations: Found observation 31876 at 2026:181:23:56:29.103 with 1 manvrs
2026-07-03 13:16:20,350 get_observations: Found observation 41376 at 2026:182:08:29:16.505 with 1 manvrs
2026-07-03 13:16:20,351 get_observations: Found observation 41375 at 2026:182:10:13:24.295 with 1 manvrs
2026-07-03 13:16:20,351 get_observations: Found observation 41374 at 2026:182:12:45:40.252 with 1 manvrs
2026-07-03 13:16:20,352 get_observations: Found observation 41373 at 2026:182:16:25:40.252 with 1 manvrs
2026-07-03 13:16:20,352 get_observations: Found observation 41372 at 2026:182:17:25:16.463 with 1 manvrs
2026-07-03 13:16:20,352 get_observations: Found observation 41371 at 2026:182:18:24:12.321 with 1 manvrs
2026-07-03 13:16:20,353 get_observations: Found observation 31410 at 2026:182:20:01:18.416 with 1 manvrs
2026-07-03 13:16:20,353 get_observations: Found observation 32401 at 2026:183:02:02:08.857 with 1 manvrs
2026-07-03 13:16:20,354 get_observations: Found observation 29858 at 2026:183:06:41:48.633 with 1 manvrs
2026-07-03 13:16:20,354 get_observations: Found observation 31340 at 2026:183:15:02:39.456 with 1 manvrs
2026-07-03 13:16:20,354 get_observations: Found observation 32400 at 2026:183:18:27:05.225 with 1 manvrs
2026-07-03 13:16:20,355 get_observations: Found observation 32414 at 2026:183:23:14:50.486 with 1 manvrs
2026-07-03 13:16:20,355 get_observations: Found observation 32416 at 2026:184:02:39:15.786 with 1 manvrs
2026-07-03 13:16:20,356 get_observations: Found observation 30020 at 2026:184:07:19:53.604 with 1 manvrs
2026-07-03 13:16:20,356 get_observations: Found observation 29932 at 2026:184:16:45:10.211 with 1 manvrs
2026-07-03 13:16:20,357 main: Found 39 observations
2026-07-03 13:16:20,357 main: ********************************************************************************
2026-07-03 13:16:20,357 main: Processing observation 31384
2026-07-03 13:16:20,357 main: ********************************************************************************
2026-07-03 13:16:20,361 obs_link_from_info: Found prev obsid 31483
2026-07-03 13:16:20,364 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,365 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-03 13:16:20,368 process_obs: ObsID 31384 already processed, skipping
2026-07-03 13:16:20,368 main:
2026-07-03 13:16:20,368 main: ********************************************************************************
2026-07-03 13:16:20,368 main: Processing observation 31385
2026-07-03 13:16:20,368 main: ********************************************************************************
2026-07-03 13:16:20,369 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,370 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-03 13:16:20,373 process_obs: ObsID 31385 already processed, skipping
2026-07-03 13:16:20,374 main:
2026-07-03 13:16:20,374 main: ********************************************************************************
2026-07-03 13:16:20,374 main: Processing observation 31596
2026-07-03 13:16:20,374 main: ********************************************************************************
2026-07-03 13:16:20,375 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,376 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-03 13:16:20,379 process_obs: ObsID 31596 already processed, skipping
2026-07-03 13:16:20,379 main:
2026-07-03 13:16:20,379 main: ********************************************************************************
2026-07-03 13:16:20,379 main: Processing observation 29829
2026-07-03 13:16:20,379 main: ********************************************************************************
2026-07-03 13:16:20,380 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,381 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-03 13:16:20,384 process_obs: ObsID 29829 already processed, skipping
2026-07-03 13:16:20,384 main:
2026-07-03 13:16:20,384 main: ********************************************************************************
2026-07-03 13:16:20,385 main: Processing observation 31597
2026-07-03 13:16:20,385 main: ********************************************************************************
2026-07-03 13:16:20,385 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,386 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-03 13:16:20,390 process_obs: ObsID 31597 already processed, skipping
2026-07-03 13:16:20,390 main:
2026-07-03 13:16:20,390 main: ********************************************************************************
2026-07-03 13:16:20,390 main: Processing observation 31408
2026-07-03 13:16:20,390 main: ********************************************************************************
2026-07-03 13:16:20,391 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,392 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-03 13:16:20,395 process_obs: ObsID 31408 already processed, skipping
2026-07-03 13:16:20,395 main:
2026-07-03 13:16:20,395 main: ********************************************************************************
2026-07-03 13:16:20,395 main: Processing observation 32335
2026-07-03 13:16:20,395 main: ********************************************************************************
2026-07-03 13:16:20,396 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,397 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-03 13:16:20,401 process_obs: ObsID 32335 already processed, skipping
2026-07-03 13:16:20,401 main:
2026-07-03 13:16:20,401 main: ********************************************************************************
2026-07-03 13:16:20,401 main: Processing observation 32403
2026-07-03 13:16:20,401 main: ********************************************************************************
2026-07-03 13:16:20,402 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,403 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-03 13:16:20,406 process_obs: ObsID 32403 already processed, skipping
2026-07-03 13:16:20,406 main:
2026-07-03 13:16:20,406 main: ********************************************************************************
2026-07-03 13:16:20,406 main: Processing observation 32404
2026-07-03 13:16:20,406 main: ********************************************************************************
2026-07-03 13:16:20,407 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,408 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-03 13:16:20,411 process_obs: ObsID 32404 already processed, skipping
2026-07-03 13:16:20,411 main:
2026-07-03 13:16:20,411 main: ********************************************************************************
2026-07-03 13:16:20,412 main: Processing observation 32406
2026-07-03 13:16:20,412 main: ********************************************************************************
2026-07-03 13:16:20,412 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,413 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-03 13:16:20,417 process_obs: ObsID 32406 already processed, skipping
2026-07-03 13:16:20,417 main:
2026-07-03 13:16:20,417 main: ********************************************************************************
2026-07-03 13:16:20,417 main: Processing observation 41385
2026-07-03 13:16:20,417 main: ********************************************************************************
2026-07-03 13:16:20,418 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,419 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-07-03 13:16:20,422 process_obs: ObsID 41385 already processed, skipping
2026-07-03 13:16:20,422 main:
2026-07-03 13:16:20,422 main: ********************************************************************************
2026-07-03 13:16:20,422 main: Processing observation 41384
2026-07-03 13:16:20,422 main: ********************************************************************************
2026-07-03 13:16:20,424 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,425 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 13:16:20,428 process_obs: ObsID 41384 already processed, skipping
2026-07-03 13:16:20,428 main:
2026-07-03 13:16:20,428 main: ********************************************************************************
2026-07-03 13:16:20,428 main: Processing observation 41383
2026-07-03 13:16:20,428 main: ********************************************************************************
2026-07-03 13:16:20,429 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,430 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 13:16:20,433 process_obs: ObsID 41383 already processed, skipping
2026-07-03 13:16:20,433 main:
2026-07-03 13:16:20,433 main: ********************************************************************************
2026-07-03 13:16:20,433 main: Processing observation 41382
2026-07-03 13:16:20,433 main: ********************************************************************************
2026-07-03 13:16:20,434 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,435 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 13:16:20,438 process_obs: ObsID 41382 already processed, skipping
2026-07-03 13:16:20,438 main:
2026-07-03 13:16:20,439 main: ********************************************************************************
2026-07-03 13:16:20,439 main: Processing observation 41381
2026-07-03 13:16:20,439 main: ********************************************************************************
2026-07-03 13:16:20,439 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,441 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 13:16:20,444 process_obs: ObsID 41381 already processed, skipping
2026-07-03 13:16:20,444 main:
2026-07-03 13:16:20,444 main: ********************************************************************************
2026-07-03 13:16:20,444 main: Processing observation 41380
2026-07-03 13:16:20,444 main: ********************************************************************************
2026-07-03 13:16:20,445 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,446 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 13:16:20,449 process_obs: ObsID 41380 already processed, skipping
2026-07-03 13:16:20,449 main:
2026-07-03 13:16:20,449 main: ********************************************************************************
2026-07-03 13:16:20,449 main: Processing observation 29996
2026-07-03 13:16:20,449 main: ********************************************************************************
2026-07-03 13:16:20,450 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,451 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 13:16:20,454 process_obs: ObsID 29996 already processed, skipping
2026-07-03 13:16:20,454 main:
2026-07-03 13:16:20,454 main: ********************************************************************************
2026-07-03 13:16:20,454 main: Processing observation 30123
2026-07-03 13:16:20,454 main: ********************************************************************************
2026-07-03 13:16:20,455 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,456 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 13:16:20,460 process_obs: ObsID 30123 already processed, skipping
2026-07-03 13:16:20,460 main:
2026-07-03 13:16:20,460 main: ********************************************************************************
2026-07-03 13:16:20,460 main: Processing observation 30118
2026-07-03 13:16:20,460 main: ********************************************************************************
2026-07-03 13:16:20,461 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,462 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 13:16:20,465 process_obs: ObsID 30118 already processed, skipping
2026-07-03 13:16:20,465 main:
2026-07-03 13:16:20,465 main: ********************************************************************************
2026-07-03 13:16:20,465 main: Processing observation 32410
2026-07-03 13:16:20,465 main: ********************************************************************************
2026-07-03 13:16:20,466 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,467 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 13:16:20,470 process_obs: ObsID 32410 already processed, skipping
2026-07-03 13:16:20,470 main:
2026-07-03 13:16:20,470 main: ********************************************************************************
2026-07-03 13:16:20,470 main: Processing observation 30261
2026-07-03 13:16:20,470 main: ********************************************************************************
2026-07-03 13:16:20,471 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,472 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 13:16:20,475 process_obs: ObsID 30261 already processed, skipping
2026-07-03 13:16:20,476 main:
2026-07-03 13:16:20,476 main: ********************************************************************************
2026-07-03 13:16:20,476 main: Processing observation 32411
2026-07-03 13:16:20,476 main: ********************************************************************************
2026-07-03 13:16:20,476 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,477 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 13:16:20,481 process_obs: ObsID 32411 already processed, skipping
2026-07-03 13:16:20,481 main:
2026-07-03 13:16:20,481 main: ********************************************************************************
2026-07-03 13:16:20,481 main: Processing observation 32409
2026-07-03 13:16:20,481 main: ********************************************************************************
2026-07-03 13:16:20,482 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,483 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 13:16:20,486 process_obs: ObsID 32409 already processed, skipping
2026-07-03 13:16:20,486 main:
2026-07-03 13:16:20,486 main: ********************************************************************************
2026-07-03 13:16:20,486 main: Processing observation 31876
2026-07-03 13:16:20,486 main: ********************************************************************************
2026-07-03 13:16:20,487 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 13:16:20,488 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:16:23,040 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31876/centroid_resids.pkl
2026-07-03 12:16:23,048 write_centroid_resids: Centroid residuals file exists, skipping
2026-07-03 12:16:23,048 plot_crs_time: Plot file exists, skipping
2026-07-03 12:16:23,048 plot_crs_scatter: Plot file exists, skipping
2026-07-03 12:16:24,294 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31876/n_kalman_delta_roll.png
2026-07-03 12:16:34,210 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31876/index.html
2026-07-03 12:16:34,211 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31876/info.json
2026-07-03 12:16:34,224 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/31/31876/index.html
2026-07-03 12:16:34,225 main:
2026-07-03 12:16:34,225 main: ********************************************************************************
2026-07-03 12:16:34,225 main: Processing observation 41376
2026-07-03 12:16:34,225 main: ********************************************************************************
2026-07-03 12:16:34,226 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:16:34,227 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:16:34,230 process_obs: ObsID 41376 already processed, skipping
2026-07-03 12:16:34,230 main:
2026-07-03 12:16:34,230 main: ********************************************************************************
2026-07-03 12:16:34,230 main: Processing observation 41375
2026-07-03 12:16:34,230 main: ********************************************************************************
2026-07-03 12:16:34,231 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:16:34,233 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:16:34,236 process_obs: ObsID 41375 already processed, skipping
2026-07-03 12:16:34,236 main:
2026-07-03 12:16:34,236 main: ********************************************************************************
2026-07-03 12:16:34,236 main: Processing observation 41374
2026-07-03 12:16:34,236 main: ********************************************************************************
2026-07-03 12:16:34,237 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:16:34,238 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:16:34,242 process_obs: ObsID 41374 already processed, skipping
2026-07-03 12:16:34,242 main:
2026-07-03 12:16:34,242 main: ********************************************************************************
2026-07-03 12:16:34,242 main: Processing observation 41373
2026-07-03 12:16:34,242 main: ********************************************************************************
2026-07-03 12:16:34,243 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:16:34,244 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:16:34,247 process_obs: ObsID 41373 already processed, skipping
2026-07-03 12:16:34,247 main:
2026-07-03 12:16:34,247 main: ********************************************************************************
2026-07-03 12:16:34,247 main: Processing observation 41372
2026-07-03 12:16:34,247 main: ********************************************************************************
2026-07-03 12:16:34,248 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:16:34,249 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:16:34,252 process_obs: ObsID 41372 already processed, skipping
2026-07-03 12:16:34,252 main:
2026-07-03 12:16:34,252 main: ********************************************************************************
2026-07-03 12:16:34,252 main: Processing observation 41371
2026-07-03 12:16:34,252 main: ********************************************************************************
2026-07-03 12:16:34,254 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:16:34,255 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:16:34,258 process_obs: ObsID 41371 already processed, skipping
2026-07-03 12:16:34,258 main:
2026-07-03 12:16:34,258 main: ********************************************************************************
2026-07-03 12:16:34,258 main: Processing observation 31410
2026-07-03 12:16:34,258 main: ********************************************************************************
2026-07-03 12:16:34,259 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:16:34,260 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:16:38,378 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31410/centroid_resids.pkl
2026-07-03 12:16:38,384 write_centroid_resids: Centroid residuals file exists, skipping
2026-07-03 12:16:38,384 plot_crs_time: Plot file exists, skipping
2026-07-03 12:16:38,384 plot_crs_scatter: Plot file exists, skipping
2026-07-03 12:16:39,062 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31410/n_kalman_delta_roll.png
2026-07-03 12:16:43,642 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31410/index.html
2026-07-03 12:16:43,645 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31410/info.json
2026-07-03 12:16:43,661 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/31/31410/index.html
2026-07-03 12:16:43,662 main:
2026-07-03 12:16:43,662 main: ********************************************************************************
2026-07-03 12:16:43,662 main: Processing observation 32401
2026-07-03 12:16:43,662 main: ********************************************************************************
2026-07-03 12:16:43,664 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:16:43,666 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:16:50,313 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-03 12:16:50,651 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32401/centroid_resids.pkl with 5 slots
2026-07-03 12:16:50,707 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32401/centroid_resids_time.png
2026-07-03 12:16:51,003 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32401/centroid_resids_scatter.png
2026-07-03 12:16:51,658 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32401/n_kalman_delta_roll.png
2026-07-03 12:16:51,795 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32401/index.html
2026-07-03 12:16:51,796 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32401/info.json
2026-07-03 12:16:51,799 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32401/index.html
2026-07-03 12:16:51,800 main:
2026-07-03 12:16:51,800 main: ********************************************************************************
2026-07-03 12:16:51,800 main: Processing observation 29858
2026-07-03 12:16:51,800 main: ********************************************************************************
2026-07-03 12:16:51,801 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:16:51,802 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:16:53,598 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-03 12:16:53,862 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29858/centroid_resids.pkl with 4 slots
2026-07-03 12:16:54,093 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29858/centroid_resids_time.png
2026-07-03 12:16:54,340 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29858/centroid_resids_scatter.png
2026-07-03 12:16:55,415 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29858/n_kalman_delta_roll.png
2026-07-03 12:16:55,563 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29858/index.html
2026-07-03 12:16:55,565 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29858/info.json
2026-07-03 12:16:55,568 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/29/29858/index.html
2026-07-03 12:16:55,569 main:
2026-07-03 12:16:55,569 main: ********************************************************************************
2026-07-03 12:16:55,569 main: Processing observation 31340
2026-07-03 12:16:55,569 main: ********************************************************************************
2026-07-03 12:16:55,570 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:16:55,572 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:16:57,370 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-03 12:16:57,643 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31340/centroid_resids.pkl with 5 slots
2026-07-03 12:16:57,691 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31340/centroid_resids_time.png
2026-07-03 12:16:57,930 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31340/centroid_resids_scatter.png
2026-07-03 12:16:58,586 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31340/n_kalman_delta_roll.png
2026-07-03 12:16:58,728 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31340/index.html
2026-07-03 12:16:58,729 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/31340/info.json
2026-07-03 12:16:58,732 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/31/31340/index.html
2026-07-03 12:16:58,733 main:
2026-07-03 12:16:58,733 main: ********************************************************************************
2026-07-03 12:16:58,733 main: Processing observation 32400
2026-07-03 12:16:58,733 main: ********************************************************************************
2026-07-03 12:16:58,734 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:16:58,736 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:17:00,677 get_centroid_resids: Computing centroid residuals from telemetry
2026-07-03 12:17:01,016 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32400/centroid_resids.pkl with 5 slots
2026-07-03 12:17:01,067 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32400/centroid_resids_time.png
2026-07-03 12:17:01,345 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32400/centroid_resids_scatter.png
2026-07-03 12:17:02,219 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32400/n_kalman_delta_roll.png
2026-07-03 12:17:02,404 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32400/index.html
2026-07-03 12:17:02,406 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32400/info.json
2026-07-03 12:17:02,411 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32400/index.html
2026-07-03 12:17:02,413 main:
2026-07-03 12:17:02,414 main: ********************************************************************************
2026-07-03 12:17:02,414 main: Processing observation 32414
2026-07-03 12:17:02,414 main: ********************************************************************************
2026-07-03 12:17:02,416 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:17:02,418 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:17:04,666 __init__: ObsID 32414 has insufficient telemetry, skipping
2026-07-03 12:17:04,667 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32414/index.html
2026-07-03 12:17:04,672 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32414/info.json
2026-07-03 12:17:04,673 main:
2026-07-03 12:17:04,673 main: ********************************************************************************
2026-07-03 12:17:04,673 main: Processing observation 32416
2026-07-03 12:17:04,673 main: ********************************************************************************
2026-07-03 12:17:04,674 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:17:04,675 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:17:07,645 __init__: ObsID 32416 has insufficient telemetry, skipping
2026-07-03 12:17:07,646 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32416/index.html
2026-07-03 12:17:07,653 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/32416/info.json
2026-07-03 12:17:07,655 main:
2026-07-03 12:17:07,656 main: ********************************************************************************
2026-07-03 12:17:07,656 main: Processing observation 30020
2026-07-03 12:17:07,656 main: ********************************************************************************
2026-07-03 12:17:07,660 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:17:07,663 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:17:10,496 __init__: ObsID 30020 has no maneuver event in telemetry, skipping
2026-07-03 12:17:10,497 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30020/index.html
2026-07-03 12:17:10,501 info: could not get attribute kalman_start for obsid 30020
2026-07-03 12:17:10,501 info: could not get attribute kalman_stop for obsid 30020
2026-07-03 12:17:10,501 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30020/info.json
2026-07-03 12:17:10,502 main:
2026-07-03 12:17:10,502 main: ********************************************************************************
2026-07-03 12:17:10,502 main: Processing observation 29932
2026-07-03 12:17:10,502 main: ********************************************************************************
2026-07-03 12:17:10,503 obs_link_from_info: No /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29932/info.json file found, obs=None
2026-07-03 12:17:10,504 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-07-03 12:17:10,505 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-07-03 12:17:12,256 __init__: ObsID 29932 has no maneuver event in telemetry, skipping
2026-07-03 12:17:12,256 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29932/index.html
2026-07-03 12:17:12,261 info: could not get attribute kalman_start for obsid 29932
2026-07-03 12:17:12,261 info: could not get attribute kalman_stop for obsid 29932
2026-07-03 12:17:12,261 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29932/info.json
2026-07-03 12:17:12,262 main:
2026-07-03 13:17:13,710 run_check: Starting centroid dashboard continuity check
2026-07-03 13:17:13,710 collect_observations: Scanning observations in /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports
2026-07-03 13:17:13,711 collect_observations: Processing year 2020
2026-07-03 13:17:15,525 collect_observations: Processing year 2021
2026-07-03 13:17:18,038 collect_observations: Processing year 2022
2026-07-03 13:17:21,053 collect_observations: Processing year 2023
2026-07-03 13:17:23,997 collect_observations: Processing year 2024
2026-07-03 13:17:26,825 collect_observations: Processing year 2025
2026-07-03 13:17:29,910 collect_observations: Processing year 2026
2026-07-03 13:17:31,428 collect_observations: Collected 14139 observations from 4-digit years
2026-07-03 13:17:31,432 sort_observations_by_date: Sorted observations by date
2026-07-03 13:17:31,432 check_continuity: Starting continuity check from most recent: ObsID 29932 (JUN2926A) at 2026:184:16:34:19.352
2026-07-03 13:17:31,451 check_continuity: Reached chain start at ObsID 47337 (MAR3020A) at 2020:092:05:29:19.712
2026-07-03 13:17:31,451 check_continuity: Chain complete: visited 14139 observations out of 14139 total
2026-07-03 13:17:31,452 run_check: ✓ Observation chain continuity check PASSED
2026-07-03 13:17:31,452 run_check: Summary: 14139 observations from 4-digit years, all properly linked