Prev Index Next

mica: Thu Apr 17 09:31:31 2025

File: /proj/sot/ska/data/mica/logs/daily.0/mica_archive.log

Task	Status	Age	MaxAge	Last Date
mica	NOT OK	0.05	2.0	Thu Apr 17 09:31:31 2025

Errors:

Line 190: <<2025-Apr-17 08:07>> 2025-04-17 07:07:13,947 ERROR get_observed_att_errors: 3
Line 190: <<2025-Apr-17 08:07>> 2025-04-17 07:07:13,947 ERROR get_observed_att_errors: 3
Line 191: <<2025-Apr-17 08:07>> 2025-04-17 07:07:13,947 Skipping obsid 30887 ERROR: 'dr'
Line 191: <<2025-Apr-17 08:07>> 2025-04-17 07:07:13,947 Skipping obsid 30887 ERROR: 'dr'
Line 222: <<2025-Apr-17 08:08>> 2025-04-17 07:08:30,716 ERROR get_observed_att_errors: 3
Line 222: <<2025-Apr-17 08:08>> 2025-04-17 07:08:30,716 ERROR get_observed_att_errors: 3
Line 223: <<2025-Apr-17 08:08>> 2025-04-17 07:08:30,716 Skipping obsid 30889 ERROR: 'dr'
Line 223: <<2025-Apr-17 08:08>> 2025-04-17 07:08:30,716 Skipping obsid 30889 ERROR: 'dr'
Line 230: <<2025-Apr-17 08:09>> 2025-04-17 07:09:42,596 ERROR get_observed_att_errors: 3
Line 230: <<2025-Apr-17 08:09>> 2025-04-17 07:09:42,596 ERROR get_observed_att_errors: 3
Line 231: <<2025-Apr-17 08:09>> 2025-04-17 07:09:42,596 Skipping obsid 30454 ERROR: 'dr'
Line 231: <<2025-Apr-17 08:09>> 2025-04-17 07:09:42,596 Skipping obsid 30454 ERROR: 'dr'
Line 251: <<2025-Apr-17 08:10>> 2025-04-17 07:10:25,447 ERROR get_observed_att_errors: 3
Line 251: <<2025-Apr-17 08:10>> 2025-04-17 07:10:25,447 ERROR get_observed_att_errors: 3
Line 252: <<2025-Apr-17 08:10>> 2025-04-17 07:10:25,447 Skipping obsid 30905 ERROR: 'dr'
Line 252: <<2025-Apr-17 08:10>> 2025-04-17 07:10:25,447 Skipping obsid 30905 ERROR: 'dr'
Line 256: <<2025-Apr-17 08:11>> 2025-04-17 07:11:01,931 ERROR get_observed_att_errors: 3
Line 256: <<2025-Apr-17 08:11>> 2025-04-17 07:11:01,931 ERROR get_observed_att_errors: 3
Line 257: <<2025-Apr-17 08:11>> 2025-04-17 07:11:01,931 Skipping obsid 28115 ERROR: 'dr'
Line 257: <<2025-Apr-17 08:11>> 2025-04-17 07:11:01,931 Skipping obsid 28115 ERROR: 'dr'
Line 271: <<2025-Apr-17 08:11>> 2025-04-17 07:11:38,310 ERROR get_observed_att_errors: 3
Line 271: <<2025-Apr-17 08:11>> 2025-04-17 07:11:38,310 ERROR get_observed_att_errors: 3
Line 272: <<2025-Apr-17 08:11>> 2025-04-17 07:11:38,310 Skipping obsid 42483 ERROR: 'dr'
Line 272: <<2025-Apr-17 08:11>> 2025-04-17 07:11:38,310 Skipping obsid 42483 ERROR: 'dr'
Line 277: <<2025-Apr-17 08:12>> 2025-04-17 07:12:14,915 ERROR get_observed_att_errors: 3
Line 277: <<2025-Apr-17 08:12>> 2025-04-17 07:12:14,915 ERROR get_observed_att_errors: 3
Line 278: <<2025-Apr-17 08:12>> 2025-04-17 07:12:14,915 Skipping obsid 28322 ERROR: 'dr'
Line 278: <<2025-Apr-17 08:12>> 2025-04-17 07:12:14,915 Skipping obsid 28322 ERROR: 'dr'
Line 283: <<2025-Apr-17 08:12>> 2025-04-17 07:12:51,419 ERROR get_observed_att_errors: 3
Line 283: <<2025-Apr-17 08:12>> 2025-04-17 07:12:51,419 ERROR get_observed_att_errors: 3
Line 284: <<2025-Apr-17 08:12>> 2025-04-17 07:12:51,419 Skipping obsid 29782 ERROR: 'dr'
Line 284: <<2025-Apr-17 08:12>> 2025-04-17 07:12:51,419 Skipping obsid 29782 ERROR: 'dr'
Line 289: <<2025-Apr-17 08:13>> 2025-04-17 07:13:27,930 ERROR get_observed_att_errors: 3
Line 289: <<2025-Apr-17 08:13>> 2025-04-17 07:13:27,930 ERROR get_observed_att_errors: 3
Line 290: <<2025-Apr-17 08:13>> 2025-04-17 07:13:27,930 Skipping obsid 42482 ERROR: 'dr'
Line 290: <<2025-Apr-17 08:13>> 2025-04-17 07:13:27,930 Skipping obsid 42482 ERROR: 'dr'
Line 295: <<2025-Apr-17 08:14>> 2025-04-17 07:14:04,204 ERROR get_observed_att_errors: 3
Line 295: <<2025-Apr-17 08:14>> 2025-04-17 07:14:04,204 ERROR get_observed_att_errors: 3
Line 296: <<2025-Apr-17 08:14>> 2025-04-17 07:14:04,204 Skipping obsid 30902 ERROR: 'dr'
Line 296: <<2025-Apr-17 08:14>> 2025-04-17 07:14:04,204 Skipping obsid 30902 ERROR: 'dr'
Line 372: <<2025-Apr-17 09:24>> Skipping obsid 30905: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 372: <<2025-Apr-17 09:24>> Skipping obsid 30905: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 376: <<2025-Apr-17 09:25>> Skipping obsid 28115: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 376: <<2025-Apr-17 09:25>> Skipping obsid 28115: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 380: <<2025-Apr-17 09:26>> Skipping obsid 42488: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 380: <<2025-Apr-17 09:26>> Skipping obsid 42488: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 384: <<2025-Apr-17 09:26>> Skipping obsid 42487: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 384: <<2025-Apr-17 09:26>> Skipping obsid 42487: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 388: <<2025-Apr-17 09:27>> Skipping obsid 42486: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 388: <<2025-Apr-17 09:27>> Skipping obsid 42486: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 392: <<2025-Apr-17 09:28>> Skipping obsid 42485: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 392: <<2025-Apr-17 09:28>> Skipping obsid 42485: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 396: <<2025-Apr-17 09:28>> Skipping obsid 42484: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 396: <<2025-Apr-17 09:28>> Skipping obsid 42484: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 400: <<2025-Apr-17 09:29>> Skipping obsid 42483: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 400: <<2025-Apr-17 09:29>> Skipping obsid 42483: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 404: <<2025-Apr-17 09:30>> Skipping obsid 28322: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 404: <<2025-Apr-17 09:30>> Skipping obsid 28322: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 684: <<2025-Apr-17 09:31>> Traceback (most recent call last):
Line 692: <<2025-Apr-17 09:31>> TimeoutError: timed out
Line 692: <<2025-Apr-17 09:31>> TimeoutError: timed out
Line 694: <<2025-Apr-17 09:31>> The above exception was the direct cause of the following exception:
Line 696: <<2025-Apr-17 09:31>> Traceback (most recent call last):
Line 710: <<2025-Apr-17 09:31>> raise ConnectTimeoutError(
Line 710: <<2025-Apr-17 09:31>> raise ConnectTimeoutError(
Line 711: <<2025-Apr-17 09:31>> urllib3.exceptions.ConnectTimeoutError: (, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)')
Line 711: <<2025-Apr-17 09:31>> urllib3.exceptions.ConnectTimeoutError: (, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)')
Line 711: <<2025-Apr-17 09:31>> urllib3.exceptions.ConnectTimeoutError: (, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)')
Line 713: <<2025-Apr-17 09:31>> The above exception was the direct cause of the following exception:
Line 715: <<2025-Apr-17 09:31>> Traceback (most recent call last):
Line 723: <<2025-Apr-17 09:31>> raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
Line 723: <<2025-Apr-17 09:31>> raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
Line 725: <<2025-Apr-17 09:31>> urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 725: <<2025-Apr-17 09:31>> urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 725: <<2025-Apr-17 09:31>> urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 727: <<2025-Apr-17 09:31>> During handling of the above exception, another exception occurred:
Line 729: <<2025-Apr-17 09:31>> Traceback (most recent call last):
Line 767: <<2025-Apr-17 09:31>> requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 767: <<2025-Apr-17 09:31>> requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
Line 767: <<2025-Apr-17 09:31>> requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))

File contents:

############################################################
/fido.real/conda/envs/ska3-mica/share/mica/update_obspar.py
############################################################
<<2025-Apr-17 08:02>> using 114200 for last_id
<<2025-Apr-17 08:02>> running get_arch for obsid 30905 run on Apr 16 2025 1:25AM
<<2025-Apr-17 08:02>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/30/30905_v01
<<2025-Apr-17 08:02>> retrieving data for 30905 in /fido.real/conda/envs/ska3-mica/data/mica/archive/tempobs/tmpltdoqol8
<<2025-Apr-17 08:02>> linking 30905_v01 -> /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/30/30905
<<2025-Apr-17 08:02>> updating archfiles default rev to 1 for 30905
<<2025-Apr-17 08:02>> running get_arch for obsid 28115 run on Apr 16 2025 1:25AM
<<2025-Apr-17 08:02>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/28/28115_v01
<<2025-Apr-17 08:02>> retrieving data for 28115 in /fido.real/conda/envs/ska3-mica/data/mica/archive/tempobs/tmp7_jx0_rd
<<2025-Apr-17 08:02>> linking 28115_v01 -> /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/28/28115
<<2025-Apr-17 08:02>> updating archfiles default rev to 1 for 28115
<<2025-Apr-17 08:02>> running get_arch for obsid 42488 run on Apr 16 2025 2:00AM
<<2025-Apr-17 08:02>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/42/42488_v01
<<2025-Apr-17 08:02>> retrieving data for 42488 in /fido.real/conda/envs/ska3-mica/data/mica/archive/tempobs/tmpc_46bkb9
<<2025-Apr-17 08:02>> linking 42488_v01 -> /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/42/42488
<<2025-Apr-17 08:02>> updating archfiles default rev to 1 for 42488
<<2025-Apr-17 08:02>> running get_arch for obsid 42487 run on Apr 16 2025 2:00AM
<<2025-Apr-17 08:02>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/42/42487_v01
<<2025-Apr-17 08:02>> retrieving data for 42487 in /fido.real/conda/envs/ska3-mica/data/mica/archive/tempobs/tmpxwp0ytj3
<<2025-Apr-17 08:02>> linking 42487_v01 -> /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/42/42487
<<2025-Apr-17 08:02>> updating archfiles default rev to 1 for 42487
<<2025-Apr-17 08:02>> running get_arch for obsid 42484 run on Apr 16 2025 2:00AM
<<2025-Apr-17 08:02>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/42/42484_v01
<<2025-Apr-17 08:02>> retrieving data for 42484 in /fido.real/conda/envs/ska3-mica/data/mica/archive/tempobs/tmpkfmwe7sd
<<2025-Apr-17 08:02>> linking 42484_v01 -> /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/42/42484
<<2025-Apr-17 08:02>> updating archfiles default rev to 1 for 42484
<<2025-Apr-17 08:02>> running get_arch for obsid 42486 run on Apr 16 2025 2:00AM
<<2025-Apr-17 08:02>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/42/42486_v01
<<2025-Apr-17 08:02>> retrieving data for 42486 in /fido.real/conda/envs/ska3-mica/data/mica/archive/tempobs/tmpa5vlmiyt
<<2025-Apr-17 08:02>> linking 42486_v01 -> /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/42/42486
<<2025-Apr-17 08:02>> updating archfiles default rev to 1 for 42486
<<2025-Apr-17 08:02>> running get_arch for obsid 42485 run on Apr 16 2025 2:00AM
<<2025-Apr-17 08:02>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/42/42485_v01
<<2025-Apr-17 08:02>> retrieving data for 42485 in /fido.real/conda/envs/ska3-mica/data/mica/archive/tempobs/tmpu3u0isuh
<<2025-Apr-17 08:02>> linking 42485_v01 -> /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/42/42485
<<2025-Apr-17 08:02>> updating archfiles default rev to 1 for 42485
<<2025-Apr-17 08:02>> running get_arch for obsid 42483 run on Apr 16 2025 12:51PM
<<2025-Apr-17 08:02>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/42/42483_v01
<<2025-Apr-17 08:02>> retrieving data for 42483 in /fido.real/conda/envs/ska3-mica/data/mica/archive/tempobs/tmp66on77_f
<<2025-Apr-17 08:02>> linking 42483_v01 -> /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/42/42483
<<2025-Apr-17 08:02>> updating archfiles default rev to 1 for 42483
<<2025-Apr-17 08:02>> running get_arch for obsid 28322 run on Apr 16 2025 12:52PM
<<2025-Apr-17 08:02>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/28/28322_v01
<<2025-Apr-17 08:02>> retrieving data for 28322 in /fido.real/conda/envs/ska3-mica/data/mica/archive/tempobs/tmp0nlr7y7k
<<2025-Apr-17 08:02>> linking 28322_v01 -> /fido.real/conda/envs/ska3-mica/data/mica/archive/obspar/28/28322
<<2025-Apr-17 08:02>> updating archfiles default rev to 1 for 28322
<<2025-Apr-17 08:02>> Checking for updates to obsids with provisional data
<<2025-Apr-17 08:03>> checking database status for obsid 346 obi 0 ver 4,
<<2025-Apr-17 08:03>> checking database status for obsid 960 obi 0 ver 5,
<<2025-Apr-17 08:03>> checking database status for obsid 52 obi 0 ver 7,
<<2025-Apr-17 08:03>> checking database status for obsid 211 obi 1 ver 4,
<<2025-Apr-17 08:03>> checking database status for obsid 212 obi 1 ver 4,
<<2025-Apr-17 08:03>> checking database status for obsid 66 obi 0 ver 4,
<<2025-Apr-17 08:03>> checking database status for obsid 1463 obi 0 ver 5,
<<2025-Apr-17 08:03>> checking database status for obsid 1309 obi 0 ver 6,
<<2025-Apr-17 08:03>> checking database status for obsid 1508 obi 0 ver 4,
<<2025-Apr-17 08:03>> checking database status for obsid 1098 obi 0 ver 6,
<<2025-Apr-17 08:03>> checking database status for obsid 1099 obi 1 ver 5,
<<2025-Apr-17 08:03>> checking database status for obsid 2292 obi 0 ver 4,
<<2025-Apr-17 08:03>> checking database status for obsid 2288 obi 0 ver 5,
<<2025-Apr-17 08:03>> checking database status for obsid 7928 obi 0 ver 3,
<<2025-Apr-17 08:03>> checking database status for obsid 9379 obi 0 ver 4,
<<2025-Apr-17 08:03>> checking database status for obsid 12181 obi 0 ver 3,
<<2025-Apr-17 08:03>> checking database status for obsid 13142 obi 0 ver 3,
<<2025-Apr-17 08:03>> checking database status for obsid 13141 obi 0 ver 3,
<<2025-Apr-17 08:03>> checking database status for obsid 13455 obi 0 ver 4,
<<2025-Apr-17 08:03>> checking database status for obsid 15703 obi 0 ver 2,
<<2025-Apr-17 08:03>> checking database status for obsid 15070 obi 0 ver 3,
<<2025-Apr-17 08:03>> checking database status for obsid 19421 obi 0 ver 2,
<<2025-Apr-17 08:03>> checking database status for obsid 19436 obi 0 ver 2,
<<2025-Apr-17 08:03>> checking database status for obsid 20325 obi 0 ver 2,
<<2025-Apr-17 08:03>> checking database status for obsid 47063 obi 0 ver 1,
<<2025-Apr-17 08:03>> checking database status for obsid 27599 obi 0 ver 2,

############################################################
/fido.real/conda/envs/ska3-mica/share/mica/update_aca_l0.py
############################################################
<<2025-Apr-17 08:03>> Fetching new CDA list from 2025:105:09:34:00.000
<<2025-Apr-17 08:03>> URL for fetch https://icxc.harvard.edu/dbtm/CDA/cgi/aspect_fetch.cgi?tstart=04-15-2025&pattern=acaimgc%%25&submit=Search
<<2025-Apr-17 08:04>> Updating /fido.real/conda/envs/ska3-mica/data/mica/archive/aca0/cda_aca0.h5 with 602 new rows
<<2025-Apr-17 08:04>> Fetching ACA L0 Data from 2025:105:21:13:20.000 to 2025:108:02:46:40.000
<<2025-Apr-17 08:04>> Ingested 180 files
<<2025-Apr-17 08:04>> Checking for missing files from 2025:105:09:34:00.000
<<2025-Apr-17 08:04>> No missing files

############################################################
/fido.real/conda/envs/ska3-mica/share/mica/update_asp_l1.py
############################################################
<<2025-Apr-17 08:04>> using 71357 for last_id
<<2025-Apr-17 08:04>> running get_arch for obsid 28115 run on Apr 16 2025 1:30AM
<<2025-Apr-17 08:04>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/asp1/28/28115_v01
<<2025-Apr-17 08:04>> retrieving data for 28115 in /fido.real/conda/envs/ska3-mica/data/mica/archive/temp/tmp8ofypbog
<<2025-Apr-17 08:05>> linking 28115_v01 -> /fido.real/conda/envs/ska3-mica/data/mica/archive/asp1/28/28115
<<2025-Apr-17 08:05>> updating archfiles default rev to 1 for 28115
<<2025-Apr-17 08:05>> Adding asp1 processing for obs 28115
<<2025-Apr-17 08:05>> processing /fido.real/conda/envs/ska3-mica/data/mica/archive/asp1/28/28115_v01/pcadf861115927N001_asol1.fits.gz
<<2025-Apr-17 08:05>> Updated table for 28115
<<2025-Apr-17 08:05>> running get_arch for obsid 30905 run on Apr 16 2025 1:33AM
<<2025-Apr-17 08:05>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/asp1/30/30905_v01
<<2025-Apr-17 08:05>> retrieving data for 30905 in /fido.real/conda/envs/ska3-mica/data/mica/archive/temp/tmpus6dj7a8
<<2025-Apr-17 08:05>> linking 30905_v01 -> /fido.real/conda/envs/ska3-mica/data/mica/archive/asp1/30/30905
<<2025-Apr-17 08:05>> updating archfiles default rev to 1 for 30905
<<2025-Apr-17 08:05>> Adding asp1 processing for obs 30905
<<2025-Apr-17 08:05>> processing /fido.real/conda/envs/ska3-mica/data/mica/archive/asp1/30/30905_v01/pcadf861099793N001_asol1.fits.gz
<<2025-Apr-17 08:05>> Updated table for 30905
<<2025-Apr-17 08:05>> running get_arch for obsid 28322 run on Apr 16 2025 12:56PM
<<2025-Apr-17 08:05>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/asp1/28/28322_v01
<<2025-Apr-17 08:05>> retrieving data for 28322 in /fido.real/conda/envs/ska3-mica/data/mica/archive/temp/tmpn7ddyxa6
<<2025-Apr-17 08:05>> linking 28322_v01 -> /fido.real/conda/envs/ska3-mica/data/mica/archive/asp1/28/28322
<<2025-Apr-17 08:05>> updating archfiles default rev to 1 for 28322
<<2025-Apr-17 08:05>> Adding asp1 processing for obs 28322
<<2025-Apr-17 08:05>> processing /fido.real/conda/envs/ska3-mica/data/mica/archive/asp1/28/28322_v01/pcadf861161240N001_asol1.fits.gz
<<2025-Apr-17 08:05>> Updated table for 28322
<<2025-Apr-17 08:05>> Checking for updates to obsids with provisional data
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 2593 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 2586 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 2587 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 3366 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 3367 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 3678 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 3679 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 5046 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 5047 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 5048 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 5049 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 5961 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 5962 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 5963 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 5964 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 6447 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 6448 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 6445 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 6446 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 37 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 36 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 1856 'bad_obsid' table
<<2025-Apr-17 08:05>> Skipping obsid asp_l1; in 1855 'bad_obsid' table
<<2025-Apr-17 08:05>> checking database status for obsid 1169 obi 1 ver 4,
<<2025-Apr-17 08:05>> Skipping 1169, obsid 'discarded'
<<2025-Apr-17 08:05>> checking database status for obsid 15703 obi 0 ver 1,
<<2025-Apr-17 08:05>> Skipping 15703, obsid 'discarded'
<<2025-Apr-17 08:05>> checking database status for obsid 22668 obi 1 ver 1,
<<2025-Apr-17 08:05>> Adding asp1 processing for obs 22668
<<2025-Apr-17 08:05>> processing /fido.real/conda/envs/ska3-mica/data/mica/archive/asp1/22/22668_v01/pcadf798846709N001_asol1.fits.gz
<<2025-Apr-17 08:05>> Resetting vv_complete to 0 for obsid 22668 rev 1
<<2025-Apr-17 08:05>> Updated table for 22668

############################################################
/fido.real/conda/envs/ska3-mica/share/mica/update_starchecks.py
############################################################
<<2025-Apr-17 08:05>> Getting new starcheck.txt files from /data/mpcrit1/mplogs
<<2025-Apr-17 08:06>> Attempting ingest of /data/mpcrit1/mplogs/2025/APR1425/oflsa/starcheck.txt

############################################################
/fido.real/conda/envs/ska3-mica/share/mica/update_centroid_reports.py
############################################################
<<2025-Apr-17 08:06>> 2025-04-17 07:06:29,563 Reading /proj/sot/ska/www/ASPECT_ICXC/centroid_reports/guide_metrics_obsid.dat
<<2025-Apr-17 08:06>> 2025-04-17 07:06:29,862 Reading /proj/sot/ska/www/ASPECT_ICXC/centroid_reports/guide_metrics_slot.dat
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,910 Obsid=42504
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,912 Skipping obsid 42504: already processed
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,912 Obsid=42503
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,912 Skipping obsid 42503: already processed
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,912 Obsid=42502
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,913 Skipping obsid 42502: already processed
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,913 Obsid=42501
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,913 Skipping obsid 42501: already processed
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,913 Obsid=42500
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,914 Skipping obsid 42500: already processed
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,914 Obsid=28138
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,915 Skipping obsid 28138: already processed
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,915 Obsid=29107
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,916 Skipping obsid 29107: already processed
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,916 Obsid=30523
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,917 Skipping obsid 30523: already processed
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,917 Obsid=28533
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,918 Skipping obsid 28533: already processed
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,918 Obsid=29810
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,919 Skipping obsid 29810: already processed
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,919 Obsid=29764
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,919 Skipping obsid 29764: already processed
<<2025-Apr-17 08:06>> 2025-04-17 07:06:30,919 Obsid=42499
<<2025-Apr-17 08:06>> 2025-04-17 07:06:36,793 Running get_observed_att_errors for 42499
<<2025-Apr-17 08:06>> 2025-04-17 07:06:37,096 Skipping obsid 42499 missing data: No dwell for obsid=42499
<<2025-Apr-17 08:06>> 2025-04-17 07:06:37,096 Obsid=30887
<<2025-Apr-17 08:06>> 2025-04-17 07:06:43,022 Running get_observed_att_errors for 30887
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,947 No ground aspect solution for 30887. Using obc aspect solution for reference
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,947 ERROR get_observed_att_errors: 3
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,947 Skipping obsid 30887 ERROR: 'dr'
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,947 Obsid=28737
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,948 Skipping obsid 28737: already processed
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,948 Obsid=29766
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,948 Skipping obsid 29766: already processed
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,949 Obsid=30159
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,949 Skipping obsid 30159: already processed
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,949 Obsid=28521
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,950 Skipping obsid 28521: already processed
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,950 Obsid=42498
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,950 Skipping obsid 42498: already processed
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,950 Obsid=42497
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,951 Skipping obsid 42497: already processed
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,951 Obsid=42496
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,952 Skipping obsid 42496: already processed
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,952 Obsid=42495
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,952 Skipping obsid 42495: already processed
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,952 Obsid=42494
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,953 Skipping obsid 42494: already processed
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,953 Obsid=42493
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,953 Skipping obsid 42493: already processed
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,953 Obsid=30508
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,954 Skipping obsid 30508: already processed
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,954 Obsid=30888
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,955 Skipping obsid 30888: already processed
<<2025-Apr-17 08:07>> 2025-04-17 07:07:13,955 Obsid=42492
<<2025-Apr-17 08:07>> 2025-04-17 07:07:38,070 Running get_observed_att_errors for 42492
<<2025-Apr-17 08:07>> 2025-04-17 07:07:38,927 Skipping obsid 42492 missing data: No dwell for obsid=42492
<<2025-Apr-17 08:07>> 2025-04-17 07:07:38,928 Obsid=30889
<<2025-Apr-17 08:07>> 2025-04-17 07:07:59,050 Running get_observed_att_errors for 30889
<<2025-Apr-17 08:08>> 2025-04-17 07:08:30,716 No ground aspect solution for 30889. Using obc aspect solution for reference
<<2025-Apr-17 08:08>> 2025-04-17 07:08:30,716 ERROR get_observed_att_errors: 3
<<2025-Apr-17 08:08>> 2025-04-17 07:08:30,716 Skipping obsid 30889 ERROR: 'dr'
<<2025-Apr-17 08:08>> 2025-04-17 07:08:30,716 Obsid=42491
<<2025-Apr-17 08:08>> 2025-04-17 07:08:52,983 Running get_observed_att_errors for 42491
<<2025-Apr-17 08:08>> 2025-04-17 07:08:54,566 Skipping obsid 42491 missing data: No dwell for obsid=42491
<<2025-Apr-17 08:08>> 2025-04-17 07:08:54,566 Obsid=30454
<<2025-Apr-17 08:09>> 2025-04-17 07:09:11,326 Running get_observed_att_errors for 30454
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,596 No ground aspect solution for 30454. Using obc aspect solution for reference
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,596 ERROR get_observed_att_errors: 3
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,596 Skipping obsid 30454 ERROR: 'dr'
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,596 Obsid=42490
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,598 Skipping obsid 42490: already processed
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,598 Obsid=30085
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,599 Skipping obsid 30085: already processed
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,599 Obsid=42489
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,599 Skipping obsid 42489: already processed
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,599 Obsid=30781
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,600 Skipping obsid 30781: already processed
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,600 Obsid=29928
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,601 Skipping obsid 29928: already processed
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,601 Obsid=27993
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,602 Skipping obsid 27993: already processed
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,602 Obsid=30179
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,603 Skipping obsid 30179: already processed
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,603 Obsid=30084
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,603 Skipping obsid 30084: already processed
<<2025-Apr-17 08:09>> 2025-04-17 07:09:42,603 Obsid=30905
<<2025-Apr-17 08:09>> 2025-04-17 07:09:54,327 Running get_observed_att_errors for 30905
<<2025-Apr-17 08:10>> 2025-04-17 07:10:25,447 No ground aspect solution for 30905. Using obc aspect solution for reference
<<2025-Apr-17 08:10>> 2025-04-17 07:10:25,447 ERROR get_observed_att_errors: 3
<<2025-Apr-17 08:10>> 2025-04-17 07:10:25,447 Skipping obsid 30905 ERROR: 'dr'
<<2025-Apr-17 08:10>> 2025-04-17 07:10:25,447 Obsid=28115
<<2025-Apr-17 08:10>> 2025-04-17 07:10:31,086 Running get_observed_att_errors for 28115
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,931 No ground aspect solution for 28115. Using obc aspect solution for reference
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,931 ERROR get_observed_att_errors: 3
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,931 Skipping obsid 28115 ERROR: 'dr'
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,931 Obsid=42488
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,932 Skipping obsid 42488: already processed
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,932 Obsid=42487
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,933 Skipping obsid 42487: already processed
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,933 Obsid=42486
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,933 Skipping obsid 42486: already processed
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,933 Obsid=42485
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,934 Skipping obsid 42485: already processed
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,934 Obsid=42484
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,934 Skipping obsid 42484: already processed
<<2025-Apr-17 08:11>> 2025-04-17 07:11:01,934 Obsid=42483
<<2025-Apr-17 08:11>> 2025-04-17 07:11:07,607 Running get_observed_att_errors for 42483
<<2025-Apr-17 08:11>> 2025-04-17 07:11:38,310 No ground aspect solution for 42483. Using obc aspect solution for reference
<<2025-Apr-17 08:11>> 2025-04-17 07:11:38,310 ERROR get_observed_att_errors: 3
<<2025-Apr-17 08:11>> 2025-04-17 07:11:38,310 Skipping obsid 42483 ERROR: 'dr'
<<2025-Apr-17 08:11>> 2025-04-17 07:11:38,310 Obsid=28322
<<2025-Apr-17 08:11>> 2025-04-17 07:11:38,315 Creating directory /proj/sot/ska/www/ASPECT_ICXC/centroid_reports/28/28322
<<2025-Apr-17 08:11>> 2025-04-17 07:11:43,955 Running get_observed_att_errors for 28322
<<2025-Apr-17 08:12>> 2025-04-17 07:12:14,914 No ground aspect solution for 28322. Using obc aspect solution for reference
<<2025-Apr-17 08:12>> 2025-04-17 07:12:14,915 ERROR get_observed_att_errors: 3
<<2025-Apr-17 08:12>> 2025-04-17 07:12:14,915 Skipping obsid 28322 ERROR: 'dr'
<<2025-Apr-17 08:12>> 2025-04-17 07:12:14,915 Obsid=29782
<<2025-Apr-17 08:12>> 2025-04-17 07:12:14,917 Creating directory /proj/sot/ska/www/ASPECT_ICXC/centroid_reports/29/29782
<<2025-Apr-17 08:12>> 2025-04-17 07:12:20,569 Running get_observed_att_errors for 29782
<<2025-Apr-17 08:12>> 2025-04-17 07:12:51,419 No ground aspect solution for 29782. Using obc aspect solution for reference
<<2025-Apr-17 08:12>> 2025-04-17 07:12:51,419 ERROR get_observed_att_errors: 3
<<2025-Apr-17 08:12>> 2025-04-17 07:12:51,419 Skipping obsid 29782 ERROR: 'dr'
<<2025-Apr-17 08:12>> 2025-04-17 07:12:51,419 Obsid=42482
<<2025-Apr-17 08:12>> 2025-04-17 07:12:51,424 Creating directory /proj/sot/ska/www/ASPECT_ICXC/centroid_reports/42/42482
<<2025-Apr-17 08:12>> 2025-04-17 07:12:57,091 Running get_observed_att_errors for 42482
<<2025-Apr-17 08:13>> 2025-04-17 07:13:27,930 No ground aspect solution for 42482. Using obc aspect solution for reference
<<2025-Apr-17 08:13>> 2025-04-17 07:13:27,930 ERROR get_observed_att_errors: 3
<<2025-Apr-17 08:13>> 2025-04-17 07:13:27,930 Skipping obsid 42482 ERROR: 'dr'
<<2025-Apr-17 08:13>> 2025-04-17 07:13:27,930 Obsid=30902
<<2025-Apr-17 08:13>> 2025-04-17 07:13:27,932 Creating directory /proj/sot/ska/www/ASPECT_ICXC/centroid_reports/30/30902
<<2025-Apr-17 08:13>> 2025-04-17 07:13:33,290 Running get_observed_att_errors for 30902
<<2025-Apr-17 08:14>> 2025-04-17 07:14:04,204 No ground aspect solution for 30902. Using obc aspect solution for reference
<<2025-Apr-17 08:14>> 2025-04-17 07:14:04,204 ERROR get_observed_att_errors: 3
<<2025-Apr-17 08:14>> 2025-04-17 07:14:04,204 Skipping obsid 30902 ERROR: 'dr'
<<2025-Apr-17 08:14>> 2025-04-17 07:14:04,204 Obsid=28234
<<2025-Apr-17 08:14>> 2025-04-17 07:14:04,209 Creating directory /proj/sot/ska/www/ASPECT_ICXC/centroid_reports/28/28234
<<2025-Apr-17 08:14>> 2025-04-17 07:14:09,282 Skipping obsid 28234 missing data: No manvr for obsid=28234

############################################################
/fido.real/conda/envs/ska3-mica/share/mica/update_acq_stats.py
############################################################
<<2025-Apr-17 08:14>> Sleeping
<<2025-Apr-17 09:16>> Processing obsid 29928
<<2025-Apr-17 09:16>> Found obsid manvr at 2025:104:12:46:59.987
<<2025-Apr-17 09:16>> Found dwell at 2025:104:13:12:51.325
<<2025-Apr-17 09:16>> Skipping obsid 29928: Problem looking up starcheck for 29928
<<2025-Apr-17 09:16>> Processing obsid 27993
<<2025-Apr-17 09:16>> Found obsid manvr at 2025:104:19:45:04.814
<<2025-Apr-17 09:16>> Found dwell at 2025:104:20:01:00.627
<<2025-Apr-17 09:17>> Skipping obsid 27993: Problem looking up starcheck for 27993
<<2025-Apr-17 09:17>> Processing obsid 30179
<<2025-Apr-17 09:17>> Found obsid manvr at 2025:104:23:55:49.765
<<2025-Apr-17 09:17>> Found dwell at 2025:105:00:26:45.528
<<2025-Apr-17 09:18>> Skipping obsid 30179: Problem looking up starcheck for 30179
<<2025-Apr-17 09:18>> Processing obsid 30084
<<2025-Apr-17 09:18>> Found obsid manvr at 2025:105:04:39:16.566
<<2025-Apr-17 09:18>> Found dwell at 2025:105:05:16:54.129
<<2025-Apr-17 09:18>> Skipping obsid 30084: Problem looking up starcheck for 30084
<<2025-Apr-17 09:18>> Processing obsid 30905
<<2025-Apr-17 09:18>> Found obsid manvr at 2025:105:09:45:32.767
<<2025-Apr-17 09:18>> Found dwell at 2025:105:10:22:04.730
<<2025-Apr-17 09:19>> Skipping obsid 30905: Problem looking up starcheck for 30905
<<2025-Apr-17 09:19>> Processing obsid 28115
<<2025-Apr-17 09:19>> Found obsid manvr at 2025:105:14:18:28.168
<<2025-Apr-17 09:19>> Found dwell at 2025:105:14:50:58.231
<<2025-Apr-17 09:19>> Skipping obsid 28115: Problem looking up starcheck for 28115
<<2025-Apr-17 09:19>> Processing obsid 42488
<<2025-Apr-17 09:19>> Found obsid manvr at 2025:105:16:52:26.494
<<2025-Apr-17 09:19>> Found dwell at 2025:105:17:19:11.132
<<2025-Apr-17 09:20>> Skipping obsid 42488: Problem looking up starcheck for 42488
<<2025-Apr-17 09:20>> Processing obsid 42487
<<2025-Apr-17 09:20>> Found obsid manvr at 2025:105:17:37:42.744
<<2025-Apr-17 09:20>> Found dwell at 2025:105:18:07:54.432
<<2025-Apr-17 09:21>> Skipping obsid 42487: Problem looking up starcheck for 42487
<<2025-Apr-17 09:21>> Processing obsid 42486
<<2025-Apr-17 09:21>> Found obsid manvr at 2025:105:18:26:08.619
<<2025-Apr-17 09:21>> Found dwell at 2025:105:19:05:55.332
<<2025-Apr-17 09:21>> Skipping obsid 42486: Problem looking up starcheck for 42486
<<2025-Apr-17 09:21>> Processing obsid 42485
<<2025-Apr-17 09:21>> Found obsid manvr at 2025:105:21:20:57.445
<<2025-Apr-17 09:21>> Found dwell at 2025:105:21:57:46.833
<<2025-Apr-17 09:22>> Skipping obsid 42485: Problem looking up starcheck for 42485
<<2025-Apr-17 09:22>> Processing obsid 42484
<<2025-Apr-17 09:22>> Found obsid manvr at 2025:105:23:45:56.620
<<2025-Apr-17 09:22>> Found dwell at 2025:106:00:17:39.533
<<2025-Apr-17 09:22>> Skipping obsid 42484: Problem looking up starcheck for 42484
<<2025-Apr-17 09:22>> Processing obsid 42483
<<2025-Apr-17 09:22>> Found obsid manvr at 2025:106:01:36:15.046
<<2025-Apr-17 09:22>> Found dwell at 2025:106:02:12:39.834
<<2025-Apr-17 09:23>> Skipping obsid 42483: Problem looking up starcheck for 42483
<<2025-Apr-17 09:23>> Processing obsid 28322
<<2025-Apr-17 09:23>> Found obsid manvr at 2025:106:02:55:17.721
<<2025-Apr-17 09:23>> Found dwell at 2025:106:03:26:11.434
<<2025-Apr-17 09:24>> Skipping obsid 28322: Problem looking up starcheck for 28322
<<2025-Apr-17 09:24>> Processing obsid 29782
<<2025-Apr-17 09:24>> Skipping obsid 29782: No obspar for 29782
<<2025-Apr-17 09:24>> Processing obsid 42482
<<2025-Apr-17 09:24>> Skipping obsid 42482: No obspar for 42482
<<2025-Apr-17 09:24>> Processing obsid 30902
<<2025-Apr-17 09:24>> Skipping obsid 30902: No obspar for 30902
<<2025-Apr-17 09:24>> Processing obsid 28234
<<2025-Apr-17 09:24>> Skipping obsid 28234: No obspar for 28234

############################################################
/fido.real/conda/envs/ska3-mica/share/mica/update_guide_stats.py
############################################################
<<2025-Apr-17 09:24>> Processing obsid 30905
<<2025-Apr-17 09:24>> Found obsid manvr at 2025:105:09:45:32.767
<<2025-Apr-17 09:24>> Found dwell at 2025:105:10:22:04.730
<<2025-Apr-17 09:24>> Skipping obsid 30905: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Apr-17 09:24>> Processing obsid 28115
<<2025-Apr-17 09:24>> Found obsid manvr at 2025:105:14:18:28.168
<<2025-Apr-17 09:24>> Found dwell at 2025:105:14:50:58.231
<<2025-Apr-17 09:25>> Skipping obsid 28115: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Apr-17 09:25>> Processing obsid 42488
<<2025-Apr-17 09:25>> Found obsid manvr at 2025:105:16:52:26.494
<<2025-Apr-17 09:25>> Found dwell at 2025:105:17:19:11.132
<<2025-Apr-17 09:26>> Skipping obsid 42488: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Apr-17 09:26>> Processing obsid 42487
<<2025-Apr-17 09:26>> Found obsid manvr at 2025:105:17:37:42.744
<<2025-Apr-17 09:26>> Found dwell at 2025:105:18:07:54.432
<<2025-Apr-17 09:26>> Skipping obsid 42487: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Apr-17 09:26>> Processing obsid 42486
<<2025-Apr-17 09:26>> Found obsid manvr at 2025:105:18:26:08.619
<<2025-Apr-17 09:26>> Found dwell at 2025:105:19:05:55.332
<<2025-Apr-17 09:27>> Skipping obsid 42486: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Apr-17 09:27>> Processing obsid 42485
<<2025-Apr-17 09:27>> Found obsid manvr at 2025:105:21:20:57.445
<<2025-Apr-17 09:27>> Found dwell at 2025:105:21:57:46.833
<<2025-Apr-17 09:28>> Skipping obsid 42485: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Apr-17 09:28>> Processing obsid 42484
<<2025-Apr-17 09:28>> Found obsid manvr at 2025:105:23:45:56.620
<<2025-Apr-17 09:28>> Found dwell at 2025:106:00:17:39.533
<<2025-Apr-17 09:28>> Skipping obsid 42484: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Apr-17 09:28>> Processing obsid 42483
<<2025-Apr-17 09:29>> Found obsid manvr at 2025:106:01:36:15.046
<<2025-Apr-17 09:29>> Found dwell at 2025:106:02:12:39.834
<<2025-Apr-17 09:29>> Skipping obsid 42483: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Apr-17 09:29>> Processing obsid 28322
<<2025-Apr-17 09:29>> Found obsid manvr at 2025:106:02:55:17.721
<<2025-Apr-17 09:29>> Found dwell at 2025:106:03:26:11.434
<<2025-Apr-17 09:30>> Skipping obsid 28322: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Apr-17 09:30>> Processing obsid 29782
<<2025-Apr-17 09:30>> Skipping obsid 29782: No obspar for 29782
<<2025-Apr-17 09:30>> Processing obsid 42482
<<2025-Apr-17 09:30>> Skipping obsid 42482: No obspar for 42482
<<2025-Apr-17 09:30>> Processing obsid 30902
<<2025-Apr-17 09:30>> Skipping obsid 30902: No obspar for 30902
<<2025-Apr-17 09:30>> Processing obsid 28234
<<2025-Apr-17 09:30>> Skipping obsid 28234: No obspar for 28234

############################################################
/fido.real/conda/envs/ska3-mica/share/mica/update_vv.py
############################################################
<<2025-Apr-17 09:30>> Skipping known bad obsid 1283
<<2025-Apr-17 09:30>> Skipping known bad obsid 1283
<<2025-Apr-17 09:30>> Skipping known bad obsid 5542
<<2025-Apr-17 09:30>> Skipping known bad obsid 5542
<<2025-Apr-17 09:30>> Skipping known bad obsid 5542
<<2025-Apr-17 09:30>> Skipping known bad obsid 722
<<2025-Apr-17 09:30>> Skipping known bad obsid 722
<<2025-Apr-17 09:30>> Skipping known bad obsid 721
<<2025-Apr-17 09:30>> Skipping known bad obsid 721
<<2025-Apr-17 09:30>> Skipping known bad obsid 5542
<<2025-Apr-17 09:30>> Skipping known bad obsid 5542
<<2025-Apr-17 09:30>> Skipping known bad obsid 5542
<<2025-Apr-17 09:30>> Skipping known bad obsid 21252
<<2025-Apr-17 09:30>> Skipping known bad obsid 21252
<<2025-Apr-17 09:30>> Skipping known bad obsid 5542
<<2025-Apr-17 09:30>> Skipping known bad obsid 5542
<<2025-Apr-17 09:30>> Skipping known bad obsid 5542
<<2025-Apr-17 09:30>> Skipping known bad obsid 21252
<<2025-Apr-17 09:30>> Skipping known bad obsid 21252
<<2025-Apr-17 09:30>> Skipping known bad obsid 721
<<2025-Apr-17 09:30>> Skipping known bad obsid 721
<<2025-Apr-17 09:30>> Skipping known bad obsid 722
<<2025-Apr-17 09:30>> Skipping known bad obsid 722
<<2025-Apr-17 09:30>> running VV for obsid 22668 run on 2023-04-25 23:27:01
<<2025-Apr-17 09:30>> Generating V&V for obsid 22668
<<2025-Apr-17 09:30>> Processing aspect interval pcadf798846709N001
<<2025-Apr-17 09:30>> Processing GOOD star in slot 3
<<2025-Apr-17 09:30>> Processing GOOD star in slot 4
<<2025-Apr-17 09:30>> Processing GOOD star in slot 5
<<2025-Apr-17 09:30>> Processing GOOD star in slot 6
<<2025-Apr-17 09:30>> Processing GOOD star in slot 7
<<2025-Apr-17 09:30>> Processing fid HRC-I-1 in slot 0
<<2025-Apr-17 09:30>> Processing fid HRC-I-2 in slot 1
<<2025-Apr-17 09:30>> Processing fid HRC-I-3 in slot 2
<<2025-Apr-17 09:30>> Saved JSON to /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/vv_report.json
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_0_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_0_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_0_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_0_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_0_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_1_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_1_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_1_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_1_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_1_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_2_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_2_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_2_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_2_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_2_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_3_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_3_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_3_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_3_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_3_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_4_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_4_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_4_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_4_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_4_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_5_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_5_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_5_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_5_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_5_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_6_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_6_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_6_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_6_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_6_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_7_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_7_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_7_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_7_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs/slot_7_m.png
<<2025-Apr-17 09:30>> obsid dir /fido.real/conda/envs/ska3-mica/data/mica/archive/vv/22/22668_v01 already exists
<<2025-Apr-17 09:30>> moved VV files to /fido.real/conda/envs/ska3-mica/data/mica/archive/vv/22/22668_v01
<<2025-Apr-17 09:30>> removed directory /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpnz5ro2xs
<<2025-Apr-17 09:30>> obsid 22668 is in table
<<2025-Apr-17 09:30>> updating obsid 22668 rev 1 in place
<<2025-Apr-17 09:30>> UPDATE aspect_1_proc set vv_complete = 4
<<2025-Apr-17 09:30>> where obsid = 22668 and revision = 1
<<2025-Apr-17 09:30>> Skipping known bad obsid 1283
<<2025-Apr-17 09:30>> Skipping known bad obsid 1283
<<2025-Apr-17 09:30>> Skipping known bad obsid 29670
<<2025-Apr-17 09:30>> Skipping known bad obsid 29670
<<2025-Apr-17 09:30>> Skipping known bad obsid 30716
<<2025-Apr-17 09:30>> Skipping known bad obsid 30716
<<2025-Apr-17 09:30>> Skipping known bad obsid 29670
<<2025-Apr-17 09:30>> Skipping known bad obsid 29670
<<2025-Apr-17 09:30>> Skipping known bad obsid 30716
<<2025-Apr-17 09:30>> Skipping known bad obsid 30716
<<2025-Apr-17 09:30>> running VV for obsid 28115 run on Apr 16 2025 1:30AM
<<2025-Apr-17 09:30>> Generating V&V for obsid 28115
<<2025-Apr-17 09:30>> Processing aspect interval pcadf861115927N001
<<2025-Apr-17 09:30>> Processing GOOD star in slot 3
<<2025-Apr-17 09:30>> Processing GOOD star in slot 4
<<2025-Apr-17 09:30>> Processing GOOD star in slot 5
<<2025-Apr-17 09:30>> Processing GOOD star in slot 6
<<2025-Apr-17 09:30>> Processing GOOD star in slot 7
<<2025-Apr-17 09:30>> Processing fid ACIS-I-1 in slot 0
<<2025-Apr-17 09:30>> Processing fid ACIS-I-5 in slot 1
<<2025-Apr-17 09:30>> Processing fid ACIS-I-6 in slot 2
<<2025-Apr-17 09:30>> Saved JSON to /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/vv_report.json
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_0_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_0_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_0_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_0_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_0_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_1_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_1_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_1_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_1_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_1_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_2_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_2_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_2_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_2_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_2_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_3_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_3_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_3_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_3_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_3_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_4_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_4_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_4_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_4_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_4_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_5_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_5_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_5_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_5_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_5_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_6_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_6_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_6_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_6_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_6_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_7_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_7_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_7_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_7_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw/slot_7_m.png
<<2025-Apr-17 09:30>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/vv/28/28115_v01
<<2025-Apr-17 09:30>> moved VV files to /fido.real/conda/envs/ska3-mica/data/mica/archive/vv/28/28115_v01
<<2025-Apr-17 09:30>> removed directory /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpspbep8bw
<<2025-Apr-17 09:30>> UPDATE aspect_1_proc set vv_complete = 4
<<2025-Apr-17 09:30>> where obsid = 28115 and revision = 1
<<2025-Apr-17 09:30>> running VV for obsid 30905 run on Apr 16 2025 1:33AM
<<2025-Apr-17 09:30>> Generating V&V for obsid 30905
<<2025-Apr-17 09:30>> Processing aspect interval pcadf861099793N001
<<2025-Apr-17 09:30>> Processing GOOD star in slot 3
<<2025-Apr-17 09:30>> Processing GOOD star in slot 4
<<2025-Apr-17 09:30>> Processing GOOD star in slot 5
<<2025-Apr-17 09:30>> Processing GOOD star in slot 6
<<2025-Apr-17 09:30>> Processing OMITTED star in slot 7
<<2025-Apr-17 09:30>> Processing fid ACIS-S-2 in slot 0
<<2025-Apr-17 09:30>> Processing fid ACIS-S-4 in slot 1
<<2025-Apr-17 09:30>> Processing fid ACIS-S-5 in slot 2
<<2025-Apr-17 09:30>> Saved JSON to /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/vv_report.json
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_0_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_0_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_0_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_0_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_0_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_1_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_1_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_1_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_1_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_1_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_2_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_2_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_2_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_2_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_2_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_3_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_3_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_3_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_3_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_3_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_4_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_4_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_4_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_4_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_4_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_5_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_5_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_5_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_5_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_5_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_6_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_6_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_6_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_6_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_6_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_7_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_7_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_7_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_7_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7/slot_7_m.png
<<2025-Apr-17 09:30>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/vv/30/30905_v01
<<2025-Apr-17 09:30>> moved VV files to /fido.real/conda/envs/ska3-mica/data/mica/archive/vv/30/30905_v01
<<2025-Apr-17 09:30>> removed directory /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmp_od9ler7
<<2025-Apr-17 09:30>> UPDATE aspect_1_proc set vv_complete = 4
<<2025-Apr-17 09:30>> where obsid = 30905 and revision = 1
<<2025-Apr-17 09:30>> running VV for obsid 28322 run on Apr 16 2025 12:56PM
<<2025-Apr-17 09:30>> Generating V&V for obsid 28322
<<2025-Apr-17 09:30>> Processing aspect interval pcadf861161240N001
<<2025-Apr-17 09:30>> Processing GOOD star in slot 3
<<2025-Apr-17 09:30>> Processing GOOD star in slot 4
<<2025-Apr-17 09:30>> Processing GOOD star in slot 5
<<2025-Apr-17 09:30>> Processing GOOD star in slot 6
<<2025-Apr-17 09:30>> Processing GOOD star in slot 7
<<2025-Apr-17 09:30>> Processing fid ACIS-I-3 in slot 0
<<2025-Apr-17 09:30>> Processing fid ACIS-I-5 in slot 1
<<2025-Apr-17 09:30>> Processing fid ACIS-I-6 in slot 2
<<2025-Apr-17 09:30>> Saved JSON to /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/vv_report.json
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_0_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_0_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_0_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_0_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_0_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_1_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_1_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_1_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_1_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_1_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_2_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_2_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_2_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_2_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_2_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_3_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_3_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_3_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_3_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_3_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_4_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_4_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_4_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_4_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_4_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_5_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_5_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_5_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_5_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_5_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_6_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_6_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_6_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_6_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_6_m.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_7_yz.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_7_yzf.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_7_y.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_7_z.png
<<2025-Apr-17 09:30>> Saved plot /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r/slot_7_m.png
<<2025-Apr-17 09:30>> making directory /fido.real/conda/envs/ska3-mica/data/mica/archive/vv/28/28322_v01
<<2025-Apr-17 09:30>> moved VV files to /fido.real/conda/envs/ska3-mica/data/mica/archive/vv/28/28322_v01
<<2025-Apr-17 09:30>> removed directory /fido.real/conda/envs/ska3-mica/data/mica/archive/tempvv/tmpr7p5e92r
<<2025-Apr-17 09:30>> UPDATE aspect_1_proc set vv_complete = 4
<<2025-Apr-17 09:30>> where obsid = 28322 and revision = 1

############################################################
/fido.real/conda/envs/ska3-mica/share/mica/update_reports.py
############################################################
<<2025-Apr-17 09:31>> Traceback (most recent call last):
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
<<2025-Apr-17 09:31>> sock = connection.create_connection(
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
<<2025-Apr-17 09:31>> raise err
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
<<2025-Apr-17 09:31>> sock.connect(sa)
<<2025-Apr-17 09:31>> TimeoutError: timed out
<<2025-Apr-17 09:31>>
<<2025-Apr-17 09:31>> The above exception was the direct cause of the following exception:
<<2025-Apr-17 09:31>>
<<2025-Apr-17 09:31>> Traceback (most recent call last):
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
<<2025-Apr-17 09:31>> response = self._make_request(
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/urllib3/connectionpool.py", line 491, in _make_request
<<2025-Apr-17 09:31>> raise new_e
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/urllib3/connectionpool.py", line 467, in _make_request
<<2025-Apr-17 09:31>> self._validate_conn(conn)
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
<<2025-Apr-17 09:31>> conn.connect()
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/urllib3/connection.py", line 616, in connect
<<2025-Apr-17 09:31>> self.sock = sock = self._new_conn()
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/urllib3/connection.py", line 207, in _new_conn
<<2025-Apr-17 09:31>> raise ConnectTimeoutError(
<<2025-Apr-17 09:31>> urllib3.exceptions.ConnectTimeoutError: (, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)')
<<2025-Apr-17 09:31>>
<<2025-Apr-17 09:31>> The above exception was the direct cause of the following exception:
<<2025-Apr-17 09:31>>
<<2025-Apr-17 09:31>> Traceback (most recent call last):
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
<<2025-Apr-17 09:31>> resp = conn.urlopen(
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
<<2025-Apr-17 09:31>> retries = retries.increment(
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
<<2025-Apr-17 09:31>> raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Apr-17 09:31>>
<<2025-Apr-17 09:31>> During handling of the above exception, another exception occurred:
<<2025-Apr-17 09:31>>
<<2025-Apr-17 09:31>> Traceback (most recent call last):
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/share/mica/update_reports.py", line 9, in
<<2025-Apr-17 09:31>> mica.report.update()
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/mica/report/report.py", line 950, in update
<<2025-Apr-17 09:31>> recent_obs = get_states(
<<2025-Apr-17 09:31>> ^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/kadi/commands/states.py", line 2186, in get_states
<<2025-Apr-17 09:31>> cmds = commands.get_cmds(start, stop, scenario=scenario)
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/kadi/commands/commands_v2.py", line 235, in get_cmds
<<2025-Apr-17 09:31>> cmds_recent = update_archive_and_get_cmds_recent(
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/kadi/commands/commands_v2.py", line 363, in update_archive_and_get_cmds_recent
<<2025-Apr-17 09:31>> loads = update_loads(scenario, cmd_events=cmd_events, lookback=lookback, stop=stop)
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/kadi/commands/commands_v2.py", line 1090, in update_loads
<<2025-Apr-17 09:31>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Apr-17 09:31>> html = get_occweb_page(
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Apr-17 09:31>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/requests/api.py", line 73, in get
<<2025-Apr-17 09:31>> return request("get", url, params=params, **kwargs)
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/requests/api.py", line 59, in request
<<2025-Apr-17 09:31>> return session.request(method=method, url=url, **kwargs)
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
<<2025-Apr-17 09:31>> resp = self.send(prep, **send_kwargs)
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
<<2025-Apr-17 09:31>> r = adapter.send(request, **kwargs)
<<2025-Apr-17 09:31>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Apr-17 09:31>> File "/fido.real/conda/envs/ska3-mica/lib/python3.11/site-packages/requests/adapters.py", line 507, in send
<<2025-Apr-17 09:31>> raise ConnectTimeout(e, request=request)
<<2025-Apr-17 09:31>> requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/MAR (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))