Prev
Index
Next
centroid_dashboard: Thu May 29 13:17:52 2025
File: /proj/sot/ska/data/ska_trend/centroid_dashboard/logs/daily.0/centroid_dashboard.log
| Task | Status | Age | MaxAge | Last Date |
|---|
| centroid_dashboard |
OK
|
0.89 | 1.0 |
Thu May 29 13:17:52 2025 |
No errors
File contents:
2025-05-29 13:16:44,909 main: Processing from 2025:142:17:16:44.909 to 2025:149:17:16:44.909
2025-05-29 13:16:53,447 get_observations: Found observation 42392 at 2025:142:18:05:16.760 with 1 manvrs
2025-05-29 13:16:53,448 get_observations: Found observation 42391 at 2025:142:19:33:15.097 with 1 manvrs
2025-05-29 13:16:53,448 get_observations: Found observation 42390 at 2025:142:20:40:07.985 with 1 manvrs
2025-05-29 13:16:53,449 get_observations: Found observation 42389 at 2025:142:21:29:14.264 with 1 manvrs
2025-05-29 13:16:53,449 get_observations: Found observation 42388 at 2025:143:00:15:40.252 with 1 manvrs
2025-05-29 13:16:53,450 get_observations: Found observation 42387 at 2025:143:01:43:40.430 with 1 manvrs
2025-05-29 13:16:53,450 get_observations: Found observation 28530 at 2025:143:03:11:15.898 with 1 manvrs
2025-05-29 13:16:53,451 get_observations: Found observation 28767 at 2025:143:10:28:17.488 with 1 manvrs
2025-05-29 13:16:53,451 get_observations: Found observation 30940 at 2025:143:14:14:06.463 with 1 manvrs
2025-05-29 13:16:53,451 get_observations: Found observation 30047 at 2025:143:17:45:18.392 with 1 manvrs
2025-05-29 13:16:53,452 get_observations: Found observation 30931 at 2025:143:22:25:53.320 with 1 manvrs
2025-05-29 13:16:53,452 get_observations: Found observation 30941 at 2025:144:01:43:02.998 with 1 manvrs
2025-05-29 13:16:53,453 get_observations: Found observation 30932 at 2025:144:05:39:57.436 with 1 manvrs
2025-05-29 13:16:53,453 get_observations: Found observation 29586 at 2025:144:09:29:00.709 with 1 manvrs
2025-05-29 13:16:53,454 get_observations: Found observation 30930 at 2025:144:15:17:24.000 with 1 manvrs
2025-05-29 13:16:53,455 get_observations: Found observation 30928 at 2025:144:18:44:09.345 with 1 manvrs
2025-05-29 13:16:53,455 get_observations: Found observation 30933 at 2025:144:22:06:39.019 with 1 manvrs
2025-05-29 13:16:53,456 get_observations: Found observation 30165 at 2025:145:01:40:14.447 with 1 manvrs
2025-05-29 13:16:53,456 get_observations: Found observation 42386 at 2025:145:09:59:29.737 with 1 manvrs
2025-05-29 13:16:53,457 get_observations: Found observation 42385 at 2025:145:11:09:51.258 with 1 manvrs
2025-05-29 13:16:53,457 get_observations: Found observation 42384 at 2025:145:12:15:34.555 with 1 manvrs
2025-05-29 13:16:53,457 get_observations: Found observation 42383 at 2025:145:15:20:40.252 with 1 manvrs
2025-05-29 13:16:53,458 get_observations: Found observation 30942 at 2025:145:18:51:56.025 with 1 manvrs
2025-05-29 13:16:53,458 get_observations: Found observation 29606 at 2025:145:23:02:07.005 with 1 manvrs
2025-05-29 13:16:53,459 get_observations: Found observation 29788 at 2025:146:01:51:56.205 with 1 manvrs
2025-05-29 13:16:53,459 get_observations: Found observation 30953 at 2025:146:08:53:36.052 with 1 manvrs
2025-05-29 13:16:53,460 get_observations: Found observation 28841 at 2025:146:12:18:53.672 with 1 manvrs
2025-05-29 13:16:53,460 get_observations: Found observation 28304 at 2025:146:17:10:24.380 with 1 manvrs
2025-05-29 13:16:53,461 get_observations: Found observation 30946 at 2025:146:23:41:55.785 with 1 manvrs
2025-05-29 13:16:53,461 get_observations: Found observation 30945 at 2025:147:04:41:52.184 with 1 manvrs
2025-05-29 13:16:53,461 get_observations: Found observation 28858 at 2025:147:10:21:33.037 with 1 manvrs
2025-05-29 13:16:53,462 get_observations: Found observation 29585 at 2025:147:14:47:31.521 with 1 manvrs
2025-05-29 13:16:53,462 get_observations: Found observation 29587 at 2025:147:19:06:01.522 with 1 manvrs
2025-05-29 13:16:53,463 get_observations: Found observation 42382 at 2025:148:01:01:55.689 with 1 manvrs
2025-05-29 13:16:53,463 get_observations: Found observation 42381 at 2025:148:04:55:40.252 with 1 manvrs
2025-05-29 13:16:53,464 get_observations: Found observation 42380 at 2025:148:07:25:40.252 with 1 manvrs
2025-05-29 13:16:53,464 get_observations: Found observation 29809 at 2025:148:11:01:32.912 with 1 manvrs
2025-05-29 13:16:53,464 get_observations: Found observation 28250 at 2025:148:15:39:02.541 with 1 manvrs
2025-05-29 13:16:53,465 get_observations: Found observation 28812 at 2025:148:17:37:13.285 with 1 manvrs
2025-05-29 13:16:53,465 get_observations: Found observation 28129 at 2025:148:19:21:22.311 with 1 manvrs
2025-05-29 13:16:53,466 get_observations: Found observation 29791 at 2025:149:00:17:49.266 with 1 manvrs
2025-05-29 13:16:53,466 get_observations: Found observation 30944 at 2025:149:04:03:08.248 with 1 manvrs
2025-05-29 13:16:53,467 get_observations: Found observation 28851 at 2025:149:08:12:59.688 with 1 manvrs
2025-05-29 13:16:53,467 get_observations: Found observation 29729 at 2025:149:17:13:42.718 with 1 manvrs
2025-05-29 13:16:53,468 main: Found 44 observations
2025-05-29 13:16:53,468 main: ********************************************************************************
2025-05-29 13:16:53,468 main: Processing observation 42392
2025-05-29 13:16:53,468 main: ********************************************************************************
2025-05-29 13:16:53,487 obs_link_from_info: Found prev obsid 28534
2025-05-29 13:16:53,490 process_obs: ObsID 42392 already processed, skipping
2025-05-29 13:16:53,490 main:
2025-05-29 13:16:53,490 main: ********************************************************************************
2025-05-29 13:16:53,490 main: Processing observation 42391
2025-05-29 13:16:53,490 main: ********************************************************************************
2025-05-29 13:16:53,494 process_obs: ObsID 42391 already processed, skipping
2025-05-29 13:16:53,494 main:
2025-05-29 13:16:53,494 main: ********************************************************************************
2025-05-29 13:16:53,494 main: Processing observation 42390
2025-05-29 13:16:53,494 main: ********************************************************************************
2025-05-29 13:16:53,498 process_obs: ObsID 42390 already processed, skipping
2025-05-29 13:16:53,498 main:
2025-05-29 13:16:53,498 main: ********************************************************************************
2025-05-29 13:16:53,498 main: Processing observation 42389
2025-05-29 13:16:53,498 main: ********************************************************************************
2025-05-29 13:16:53,502 process_obs: ObsID 42389 already processed, skipping
2025-05-29 13:16:53,502 main:
2025-05-29 13:16:53,502 main: ********************************************************************************
2025-05-29 13:16:53,502 main: Processing observation 42388
2025-05-29 13:16:53,502 main: ********************************************************************************
2025-05-29 13:16:53,505 process_obs: ObsID 42388 already processed, skipping
2025-05-29 13:16:53,505 main:
2025-05-29 13:16:53,505 main: ********************************************************************************
2025-05-29 13:16:53,505 main: Processing observation 42387
2025-05-29 13:16:53,505 main: ********************************************************************************
2025-05-29 13:16:53,509 process_obs: ObsID 42387 already processed, skipping
2025-05-29 13:16:53,509 main:
2025-05-29 13:16:53,509 main: ********************************************************************************
2025-05-29 13:16:53,509 main: Processing observation 28530
2025-05-29 13:16:53,509 main: ********************************************************************************
2025-05-29 13:16:53,514 process_obs: ObsID 28530 already processed, skipping
2025-05-29 13:16:53,514 main:
2025-05-29 13:16:53,514 main: ********************************************************************************
2025-05-29 13:16:53,514 main: Processing observation 28767
2025-05-29 13:16:53,514 main: ********************************************************************************
2025-05-29 13:16:53,517 process_obs: ObsID 28767 already processed, skipping
2025-05-29 13:16:53,518 main:
2025-05-29 13:16:53,518 main: ********************************************************************************
2025-05-29 13:16:53,518 main: Processing observation 30940
2025-05-29 13:16:53,518 main: ********************************************************************************
2025-05-29 13:16:53,522 process_obs: ObsID 30940 already processed, skipping
2025-05-29 13:16:53,522 main:
2025-05-29 13:16:53,522 main: ********************************************************************************
2025-05-29 13:16:53,522 main: Processing observation 30047
2025-05-29 13:16:53,522 main: ********************************************************************************
2025-05-29 13:16:53,525 process_obs: ObsID 30047 already processed, skipping
2025-05-29 13:16:53,525 main:
2025-05-29 13:16:53,525 main: ********************************************************************************
2025-05-29 13:16:53,526 main: Processing observation 30931
2025-05-29 13:16:53,526 main: ********************************************************************************
2025-05-29 13:16:53,529 process_obs: ObsID 30931 already processed, skipping
2025-05-29 13:16:53,529 main:
2025-05-29 13:16:53,529 main: ********************************************************************************
2025-05-29 13:16:53,529 main: Processing observation 30941
2025-05-29 13:16:53,529 main: ********************************************************************************
2025-05-29 13:16:53,533 process_obs: ObsID 30941 already processed, skipping
2025-05-29 13:16:53,533 main:
2025-05-29 13:16:53,533 main: ********************************************************************************
2025-05-29 13:16:53,533 main: Processing observation 30932
2025-05-29 13:16:53,533 main: ********************************************************************************
2025-05-29 13:16:53,536 process_obs: ObsID 30932 already processed, skipping
2025-05-29 13:16:53,537 main:
2025-05-29 13:16:53,537 main: ********************************************************************************
2025-05-29 13:16:53,537 main: Processing observation 29586
2025-05-29 13:16:53,537 main: ********************************************************************************
2025-05-29 13:16:53,541 process_obs: ObsID 29586 already processed, skipping
2025-05-29 13:16:53,541 main:
2025-05-29 13:16:53,541 main: ********************************************************************************
2025-05-29 13:16:53,541 main: Processing observation 30930
2025-05-29 13:16:53,541 main: ********************************************************************************
2025-05-29 12:16:59,045 get_centroid_resids_from_file: Reading centroid residuals from /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY1925A/30930/centroid_resids.pkl
2025-05-29 12:16:59,584 write_centroid_resids: Centroid residuals file exists, skipping
2025-05-29 12:16:59,584 plot_crs_time: Plot file exists, skipping
2025-05-29 12:16:59,584 plot_crs_scatter: Plot file exists, skipping
2025-05-29 12:16:59,622 plot_n_kalman_delta_roll: No OBC - GND deltas available, not plotting
2025-05-29 12:17:01,237 plot_n_kalman_delta_roll: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY1925A/30930/n_kalman_delta_roll.png
2025-05-29 12:17:05,502 write_index_html: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY1925A/30930/index.html
2025-05-29 12:17:05,508 write_info_json: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY1925A/30930/info.json
2025-05-29 12:17:05,523 make_obsid_dir_links: Removing existing file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/30/30930/index.html
2025-05-29 12:17:05,532 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY1925A/30930/index.html to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/30/30930/index.html
2025-05-29 12:17:05,534 make_obsid_dir_links: Removing existing file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/30/30930/centroid_resids.pkl
2025-05-29 12:17:05,535 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY1925A/30930/centroid_resids.pkl to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/30/30930/centroid_resids.pkl
2025-05-29 12:17:05,537 make_obsid_dir_links: Removing existing file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/30/30930/centroid_resids_time.png
2025-05-29 12:17:05,538 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY1925A/30930/centroid_resids_time.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/30/30930/centroid_resids_time.png
2025-05-29 12:17:05,540 make_obsid_dir_links: Removing existing file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/30/30930/centroid_resids_scatter.png
2025-05-29 12:17:05,541 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY1925A/30930/centroid_resids_scatter.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/30/30930/centroid_resids_scatter.png
2025-05-29 12:17:05,543 make_obsid_dir_links: Removing existing file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/30/30930/n_kalman_delta_roll.png
2025-05-29 12:17:05,544 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY1925A/30930/n_kalman_delta_roll.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/30/30930/n_kalman_delta_roll.png
2025-05-29 12:17:05,547 make_obsid_dir_links: Removing existing file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/30/30930/info.json
2025-05-29 12:17:05,548 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY1925A/30930/info.json to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/30/30930/info.json
2025-05-29 12:17:05,549 main:
2025-05-29 12:17:05,549 main: ********************************************************************************
2025-05-29 12:17:05,549 main: Processing observation 30928
2025-05-29 12:17:05,549 main: ********************************************************************************
2025-05-29 12:17:05,565 process_obs: ObsID 30928 already processed, skipping
2025-05-29 12:17:05,565 main:
2025-05-29 12:17:05,565 main: ********************************************************************************
2025-05-29 12:17:05,565 main: Processing observation 30933
2025-05-29 12:17:05,565 main: ********************************************************************************
2025-05-29 12:17:05,584 process_obs: ObsID 30933 already processed, skipping
2025-05-29 12:17:05,585 main:
2025-05-29 12:17:05,585 main: ********************************************************************************
2025-05-29 12:17:05,585 main: Processing observation 30165
2025-05-29 12:17:05,585 main: ********************************************************************************
2025-05-29 12:17:05,595 process_obs: ObsID 30165 already processed, skipping
2025-05-29 12:17:05,595 main:
2025-05-29 12:17:05,595 main: ********************************************************************************
2025-05-29 12:17:05,595 main: Processing observation 42386
2025-05-29 12:17:05,595 main: ********************************************************************************
2025-05-29 12:17:05,610 process_obs: ObsID 42386 already processed, skipping
2025-05-29 12:17:05,610 main:
2025-05-29 12:17:05,610 main: ********************************************************************************
2025-05-29 12:17:05,610 main: Processing observation 42385
2025-05-29 12:17:05,610 main: ********************************************************************************
2025-05-29 12:17:05,625 process_obs: ObsID 42385 already processed, skipping
2025-05-29 12:17:05,625 main:
2025-05-29 12:17:05,625 main: ********************************************************************************
2025-05-29 12:17:05,625 main: Processing observation 42384
2025-05-29 12:17:05,625 main: ********************************************************************************
2025-05-29 12:17:05,638 process_obs: ObsID 42384 already processed, skipping
2025-05-29 12:17:05,638 main:
2025-05-29 12:17:05,638 main: ********************************************************************************
2025-05-29 12:17:05,638 main: Processing observation 42383
2025-05-29 12:17:05,638 main: ********************************************************************************
2025-05-29 12:17:05,652 process_obs: ObsID 42383 already processed, skipping
2025-05-29 12:17:05,652 main:
2025-05-29 12:17:05,652 main: ********************************************************************************
2025-05-29 12:17:05,652 main: Processing observation 30942
2025-05-29 12:17:05,652 main: ********************************************************************************
2025-05-29 12:17:05,675 process_obs: ObsID 30942 already processed, skipping
2025-05-29 12:17:05,675 main:
2025-05-29 12:17:05,675 main: ********************************************************************************
2025-05-29 12:17:05,675 main: Processing observation 29606
2025-05-29 12:17:05,675 main: ********************************************************************************
2025-05-29 12:17:05,706 process_obs: ObsID 29606 already processed, skipping
2025-05-29 12:17:05,706 main:
2025-05-29 12:17:05,706 main: ********************************************************************************
2025-05-29 12:17:05,706 main: Processing observation 29788
2025-05-29 12:17:05,706 main: ********************************************************************************
2025-05-29 12:17:05,728 process_obs: ObsID 29788 already processed, skipping
2025-05-29 12:17:05,728 main:
2025-05-29 12:17:05,728 main: ********************************************************************************
2025-05-29 12:17:05,728 main: Processing observation 30953
2025-05-29 12:17:05,728 main: ********************************************************************************
2025-05-29 12:17:05,762 process_obs: ObsID 30953 already processed, skipping
2025-05-29 12:17:05,763 main:
2025-05-29 12:17:05,763 main: ********************************************************************************
2025-05-29 12:17:05,763 main: Processing observation 28841
2025-05-29 12:17:05,763 main: ********************************************************************************
2025-05-29 12:17:05,794 process_obs: ObsID 28841 already processed, skipping
2025-05-29 12:17:05,794 main:
2025-05-29 12:17:05,794 main: ********************************************************************************
2025-05-29 12:17:05,794 main: Processing observation 28304
2025-05-29 12:17:05,794 main: ********************************************************************************
2025-05-29 12:17:05,824 process_obs: ObsID 28304 already processed, skipping
2025-05-29 12:17:05,824 main:
2025-05-29 12:17:05,824 main: ********************************************************************************
2025-05-29 12:17:05,824 main: Processing observation 30946
2025-05-29 12:17:05,824 main: ********************************************************************************
2025-05-29 12:17:05,849 process_obs: ObsID 30946 already processed, skipping
2025-05-29 12:17:05,849 main:
2025-05-29 12:17:05,849 main: ********************************************************************************
2025-05-29 12:17:05,849 main: Processing observation 30945
2025-05-29 12:17:05,849 main: ********************************************************************************
2025-05-29 12:17:05,876 process_obs: ObsID 30945 already processed, skipping
2025-05-29 12:17:05,876 main:
2025-05-29 12:17:05,876 main: ********************************************************************************
2025-05-29 12:17:05,876 main: Processing observation 28858
2025-05-29 12:17:05,876 main: ********************************************************************************
2025-05-29 12:17:05,893 process_obs: ObsID 28858 already processed, skipping
2025-05-29 12:17:05,893 main:
2025-05-29 12:17:05,893 main: ********************************************************************************
2025-05-29 12:17:05,893 main: Processing observation 29585
2025-05-29 12:17:05,893 main: ********************************************************************************
2025-05-29 12:17:05,926 process_obs: ObsID 29585 already processed, skipping
2025-05-29 12:17:05,926 main:
2025-05-29 12:17:05,926 main: ********************************************************************************
2025-05-29 12:17:05,926 main: Processing observation 29587
2025-05-29 12:17:05,926 main: ********************************************************************************
2025-05-29 12:17:05,953 process_obs: ObsID 29587 already processed, skipping
2025-05-29 12:17:05,954 main:
2025-05-29 12:17:05,954 main: ********************************************************************************
2025-05-29 12:17:05,954 main: Processing observation 42382
2025-05-29 12:17:05,954 main: ********************************************************************************
2025-05-29 12:17:10,462 write_centroid_resids: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/centroid_resids.pkl with 8 slots
2025-05-29 12:17:10,716 plot_crs_time: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/centroid_resids_time.png
2025-05-29 12:17:12,519 plot_crs_scatter: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/centroid_resids_scatter.png
2025-05-29 12:17:12,663 plot_n_kalman_delta_roll: No OBC - GND deltas available, not plotting
2025-05-29 12:17:12,831 plot_n_kalman_delta_roll: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/n_kalman_delta_roll.png
2025-05-29 12:17:16,150 write_index_html: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/index.html
2025-05-29 12:17:16,152 write_info_json: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/info.json
2025-05-29 12:17:16,155 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/index.html to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42382/index.html
2025-05-29 12:17:16,155 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/centroid_resids.pkl to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42382/centroid_resids.pkl
2025-05-29 12:17:16,156 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/centroid_resids_time.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42382/centroid_resids_time.png
2025-05-29 12:17:16,157 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/centroid_resids_scatter.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42382/centroid_resids_scatter.png
2025-05-29 12:17:16,157 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/n_kalman_delta_roll.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42382/n_kalman_delta_roll.png
2025-05-29 12:17:16,158 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/kalman_plot_done to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42382/kalman_plot_done
2025-05-29 12:17:16,159 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42382/info.json to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42382/info.json
2025-05-29 12:17:16,159 main:
2025-05-29 12:17:16,159 main: ********************************************************************************
2025-05-29 12:17:16,159 main: Processing observation 42381
2025-05-29 12:17:16,159 main: ********************************************************************************
2025-05-29 12:17:19,911 write_centroid_resids: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/centroid_resids.pkl with 8 slots
2025-05-29 12:17:19,988 plot_crs_time: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/centroid_resids_time.png
2025-05-29 12:17:20,389 plot_crs_scatter: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/centroid_resids_scatter.png
2025-05-29 12:17:20,513 plot_n_kalman_delta_roll: No OBC - GND deltas available, not plotting
2025-05-29 12:17:20,659 plot_n_kalman_delta_roll: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/n_kalman_delta_roll.png
2025-05-29 12:17:20,856 write_index_html: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/index.html
2025-05-29 12:17:20,858 write_info_json: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/info.json
2025-05-29 12:17:20,861 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/index.html to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42381/index.html
2025-05-29 12:17:20,861 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/centroid_resids.pkl to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42381/centroid_resids.pkl
2025-05-29 12:17:20,862 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/centroid_resids_time.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42381/centroid_resids_time.png
2025-05-29 12:17:20,863 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/centroid_resids_scatter.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42381/centroid_resids_scatter.png
2025-05-29 12:17:20,863 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/n_kalman_delta_roll.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42381/n_kalman_delta_roll.png
2025-05-29 12:17:20,864 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/kalman_plot_done to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42381/kalman_plot_done
2025-05-29 12:17:20,864 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42381/info.json to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42381/info.json
2025-05-29 12:17:20,865 main:
2025-05-29 12:17:20,865 main: ********************************************************************************
2025-05-29 12:17:20,865 main: Processing observation 42380
2025-05-29 12:17:20,865 main: ********************************************************************************
2025-05-29 12:17:24,068 write_centroid_resids: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/centroid_resids.pkl with 8 slots
2025-05-29 12:17:24,143 plot_crs_time: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/centroid_resids_time.png
2025-05-29 12:17:24,494 plot_crs_scatter: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/centroid_resids_scatter.png
2025-05-29 12:17:24,628 plot_n_kalman_delta_roll: No OBC - GND deltas available, not plotting
2025-05-29 12:17:24,780 plot_n_kalman_delta_roll: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/n_kalman_delta_roll.png
2025-05-29 12:17:24,951 write_index_html: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/index.html
2025-05-29 12:17:24,952 write_info_json: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/info.json
2025-05-29 12:17:24,955 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/index.html to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42380/index.html
2025-05-29 12:17:24,956 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/centroid_resids.pkl to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42380/centroid_resids.pkl
2025-05-29 12:17:24,956 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/centroid_resids_time.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42380/centroid_resids_time.png
2025-05-29 12:17:24,957 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/centroid_resids_scatter.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42380/centroid_resids_scatter.png
2025-05-29 12:17:24,957 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/n_kalman_delta_roll.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42380/n_kalman_delta_roll.png
2025-05-29 12:17:24,958 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/kalman_plot_done to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42380/kalman_plot_done
2025-05-29 12:17:24,959 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/42380/info.json to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/42/42380/info.json
2025-05-29 12:17:24,961 main:
2025-05-29 12:17:24,961 main: ********************************************************************************
2025-05-29 12:17:24,961 main: Processing observation 29809
2025-05-29 12:17:24,961 main: ********************************************************************************
2025-05-29 12:17:28,141 write_centroid_resids: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/centroid_resids.pkl with 5 slots
2025-05-29 12:17:28,190 plot_crs_time: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/centroid_resids_time.png
2025-05-29 12:17:28,681 plot_crs_scatter: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/centroid_resids_scatter.png
2025-05-29 12:17:29,572 plot_n_kalman_delta_roll: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/n_kalman_delta_roll.png
2025-05-29 12:17:29,747 write_index_html: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/index.html
2025-05-29 12:17:29,749 write_info_json: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/info.json
2025-05-29 12:17:29,771 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/index.html to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/29/29809/index.html
2025-05-29 12:17:29,771 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/centroid_resids.pkl to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/29/29809/centroid_resids.pkl
2025-05-29 12:17:29,773 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/centroid_resids_time.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/29/29809/centroid_resids_time.png
2025-05-29 12:17:29,773 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/centroid_resids_scatter.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/29/29809/centroid_resids_scatter.png
2025-05-29 12:17:29,774 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/n_kalman_delta_roll.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/29/29809/n_kalman_delta_roll.png
2025-05-29 12:17:29,774 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/kalman_plot_done to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/29/29809/kalman_plot_done
2025-05-29 12:17:29,775 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29809/info.json to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/29/29809/info.json
2025-05-29 12:17:29,775 main:
2025-05-29 12:17:29,776 main: ********************************************************************************
2025-05-29 12:17:29,776 main: Processing observation 28250
2025-05-29 12:17:29,776 main: ********************************************************************************
2025-05-29 12:17:32,792 write_centroid_resids: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/centroid_resids.pkl with 5 slots
2025-05-29 12:17:32,836 plot_crs_time: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/centroid_resids_time.png
2025-05-29 12:17:33,073 plot_crs_scatter: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/centroid_resids_scatter.png
2025-05-29 12:17:33,636 plot_n_kalman_delta_roll: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/n_kalman_delta_roll.png
2025-05-29 12:17:33,792 write_index_html: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/index.html
2025-05-29 12:17:33,794 write_info_json: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/info.json
2025-05-29 12:17:33,812 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/index.html to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28250/index.html
2025-05-29 12:17:33,812 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/centroid_resids.pkl to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28250/centroid_resids.pkl
2025-05-29 12:17:33,813 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/centroid_resids_time.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28250/centroid_resids_time.png
2025-05-29 12:17:33,814 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/centroid_resids_scatter.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28250/centroid_resids_scatter.png
2025-05-29 12:17:33,814 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/n_kalman_delta_roll.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28250/n_kalman_delta_roll.png
2025-05-29 12:17:33,815 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/kalman_plot_done to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28250/kalman_plot_done
2025-05-29 12:17:33,815 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28250/info.json to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28250/info.json
2025-05-29 12:17:33,816 main:
2025-05-29 12:17:33,816 main: ********************************************************************************
2025-05-29 12:17:33,816 main: Processing observation 28812
2025-05-29 12:17:33,816 main: ********************************************************************************
2025-05-29 12:17:36,619 write_centroid_resids: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/centroid_resids.pkl with 5 slots
2025-05-29 12:17:36,665 plot_crs_time: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/centroid_resids_time.png
2025-05-29 12:17:36,911 plot_crs_scatter: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/centroid_resids_scatter.png
2025-05-29 12:17:37,567 plot_n_kalman_delta_roll: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/n_kalman_delta_roll.png
2025-05-29 12:17:37,779 write_index_html: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/index.html
2025-05-29 12:17:37,782 write_info_json: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/info.json
2025-05-29 12:17:37,786 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/centroid_resids.pkl to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28812/centroid_resids.pkl
2025-05-29 12:17:37,787 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/centroid_resids_time.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28812/centroid_resids_time.png
2025-05-29 12:17:37,788 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/centroid_resids_scatter.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28812/centroid_resids_scatter.png
2025-05-29 12:17:37,790 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/n_kalman_delta_roll.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28812/n_kalman_delta_roll.png
2025-05-29 12:17:37,790 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/kalman_plot_done to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28812/kalman_plot_done
2025-05-29 12:17:37,791 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/index.html to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28812/index.html
2025-05-29 12:17:37,792 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28812/info.json to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28812/info.json
2025-05-29 12:17:37,793 main:
2025-05-29 12:17:37,793 main: ********************************************************************************
2025-05-29 12:17:37,793 main: Processing observation 28129
2025-05-29 12:17:37,793 main: ********************************************************************************
2025-05-29 12:17:40,608 write_centroid_resids: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28129/centroid_resids.pkl with 5 slots
2025-05-29 12:17:40,664 plot_crs_time: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28129/centroid_resids_time.png
2025-05-29 12:17:41,388 plot_crs_scatter: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28129/centroid_resids_scatter.png
2025-05-29 12:17:41,512 plot_n_kalman_delta_roll: No OBC - GND deltas available, not plotting
2025-05-29 12:17:41,696 plot_n_kalman_delta_roll: Writing plot file /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28129/n_kalman_delta_roll.png
2025-05-29 12:17:41,859 write_index_html: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28129/index.html
2025-05-29 12:17:41,861 write_info_json: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28129/info.json
2025-05-29 12:17:41,863 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28129/centroid_resids.pkl to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28129/centroid_resids.pkl
2025-05-29 12:17:41,864 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28129/centroid_resids_time.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28129/centroid_resids_time.png
2025-05-29 12:17:41,865 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28129/centroid_resids_scatter.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28129/centroid_resids_scatter.png
2025-05-29 12:17:41,866 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28129/n_kalman_delta_roll.png to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28129/n_kalman_delta_roll.png
2025-05-29 12:17:41,867 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28129/index.html to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28129/index.html
2025-05-29 12:17:41,867 make_obsid_dir_links: Linking /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28129/info.json to /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/28/28129/info.json
2025-05-29 12:17:41,868 main:
2025-05-29 12:17:41,868 main: ********************************************************************************
2025-05-29 12:17:41,868 main: Processing observation 29791
2025-05-29 12:17:41,868 main: ********************************************************************************
2025-05-29 12:17:44,459 __init__: ObsID 29791 has insufficient telemetry, skipping
2025-05-29 12:17:44,471 write_index_html: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29791/index.html
2025-05-29 12:17:44,473 main:
2025-05-29 12:17:44,473 main: ********************************************************************************
2025-05-29 12:17:44,473 main: Processing observation 30944
2025-05-29 12:17:44,473 main: ********************************************************************************
2025-05-29 12:17:47,067 __init__: ObsID 30944 has insufficient telemetry, skipping
2025-05-29 12:17:47,080 write_index_html: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/30944/index.html
2025-05-29 12:17:47,081 main:
2025-05-29 12:17:47,081 main: ********************************************************************************
2025-05-29 12:17:47,081 main: Processing observation 28851
2025-05-29 12:17:47,081 main: ********************************************************************************
2025-05-29 12:17:49,633 __init__: ObsID 28851 has no maneuver event in telemetry, skipping
2025-05-29 12:17:49,646 write_index_html: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/28851/index.html
2025-05-29 12:17:49,647 main:
2025-05-29 12:17:49,647 main: ********************************************************************************
2025-05-29 12:17:49,647 main: Processing observation 29729
2025-05-29 12:17:49,647 main: ********************************************************************************
2025-05-29 12:17:49,648 obs_link_from_info: No /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29729/info.json file found, obs=None
2025-05-29 12:17:52,022 __init__: ObsID 29729 has no maneuver event in telemetry, skipping
2025-05-29 12:17:52,035 write_index_html: Writing /export/tom/centroid_reports/www/ASPECT_ICXC/centroid_reports/2025/MAY2625B/29729/index.html
2025-05-29 12:17:52,038 main: