Prev
Index
Next
arc: Sat Aug 23 04:50:26 2025
File: /proj/sot/ska/data/arc/Logs/daily.0/arc.log
| Task | Status | Age | MaxAge | Last Date |
|---|
| arc |
NOT OK
|
0.25 | 2.0 |
Sat Aug 23 04:50:26 2025 |
Errors:
- Line 4: <<2025-Aug-22 06:11>> WARNING: p3 fluence not plotted, error : index -1 is out of bounds for axis 0 with size 0
- Line 4: <<2025-Aug-22 06:11>> WARNING: p3 fluence not plotted, error : index -1 is out of bounds for axis 0 with size 0
- Line 9: <<2025-Aug-22 06:16>> WARNING: p3 fluence not plotted, error : index -1 is out of bounds for axis 0 with size 0
- Line 9: <<2025-Aug-22 06:16>> WARNING: p3 fluence not plotted, error : index -1 is out of bounds for axis 0 with size 0
- Line 15: WARNING - /proj/sot/ska3/flight/share/arc3/plot_goes_x.py command timed out Fri Aug 22 12:21:04 2025
- Line 16: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 13:16:00 2025
- Line 45: <<2025-Aug-22 13:17>> TimeoutError: _ssl.c:983: The handshake operation timed out
- Line 57: <<2025-Aug-22 13:17>> raise reraise(type(error), error, _stacktrace)
- Line 69: <<2025-Aug-22 13:17>> raise ReadTimeoutError(
- Line 70: <<2025-Aug-22 13:17>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 113: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 13:26:00 2025
- Line 142: <<2025-Aug-22 13:27>> TimeoutError: The read operation timed out
- Line 154: <<2025-Aug-22 13:27>> raise reraise(type(error), error, _stacktrace)
- Line 164: <<2025-Aug-22 13:27>> raise ReadTimeoutError(
- Line 165: <<2025-Aug-22 13:27>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 214: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 13:36:00 2025
- Line 232: <<2025-Aug-22 13:39>> TimeoutError: timed out
- Line 250: <<2025-Aug-22 13:39>> raise ConnectTimeoutError(
- Line 251: <<2025-Aug-22 13:39>> urllib3.exceptions.ConnectTimeoutError: (, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)')
- Line 263: <<2025-Aug-22 13:39>> raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
- Line 265: <<2025-Aug-22 13:39>> urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/AUG (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
- Line 307: <<2025-Aug-22 13:39>> requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/AUG (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
- Line 308: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 13:46:00 2025
- Line 337: <<2025-Aug-22 13:48>> TimeoutError: The read operation timed out
- Line 349: <<2025-Aug-22 13:48>> raise reraise(type(error), error, _stacktrace)
- Line 359: <<2025-Aug-22 13:48>> raise ReadTimeoutError(
- Line 360: <<2025-Aug-22 13:48>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 403: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 13:56:00 2025
- Line 437: <<2025-Aug-22 13:58>> TimeoutError: The read operation timed out
- Line 449: <<2025-Aug-22 13:58>> raise reraise(type(error), error, _stacktrace)
- Line 459: <<2025-Aug-22 13:58>> raise ReadTimeoutError(
- Line 460: <<2025-Aug-22 13:58>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 509: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 14:06:00 2025
- Line 515: WARNING - /proj/sot/ska3/flight/share/arc3/make_timeline.py command timed out Fri Aug 22 14:11:00 2025
- Line 516: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 14:16:00 2025
- Line 545: <<2025-Aug-22 14:18>> TimeoutError: The read operation timed out
- Line 557: <<2025-Aug-22 14:18>> raise reraise(type(error), error, _stacktrace)
- Line 567: <<2025-Aug-22 14:18>> raise ReadTimeoutError(
- Line 568: <<2025-Aug-22 14:18>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 611: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 14:26:00 2025
- Line 640: <<2025-Aug-22 14:28>> TimeoutError: The read operation timed out
- Line 652: <<2025-Aug-22 14:28>> raise reraise(type(error), error, _stacktrace)
- Line 662: <<2025-Aug-22 14:28>> raise ReadTimeoutError(
- Line 663: <<2025-Aug-22 14:28>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 706: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 14:36:00 2025
- Line 740: <<2025-Aug-22 14:39>> TimeoutError: The read operation timed out
- Line 752: <<2025-Aug-22 14:39>> raise reraise(type(error), error, _stacktrace)
- Line 762: <<2025-Aug-22 14:39>> raise ReadTimeoutError(
- Line 763: <<2025-Aug-22 14:39>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 812: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 14:46:00 2025
- Line 825: <<2025-Aug-22 14:47>> TimeoutError: timed out
- Line 843: <<2025-Aug-22 14:47>> raise ConnectTimeoutError(
- Line 844: <<2025-Aug-22 14:47>> urllib3.exceptions.ConnectTimeoutError: (, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)')
- Line 856: <<2025-Aug-22 14:47>> raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
- Line 858: <<2025-Aug-22 14:47>> urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/JUL (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
- Line 900: <<2025-Aug-22 14:47>> requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/JUL (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
- Line 901: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 14:56:00 2025
- Line 930: <<2025-Aug-22 14:58>> TimeoutError: _ssl.c:983: The handshake operation timed out
- Line 942: <<2025-Aug-22 14:58>> raise reraise(type(error), error, _stacktrace)
- Line 954: <<2025-Aug-22 14:58>> raise ReadTimeoutError(
- Line 955: <<2025-Aug-22 14:58>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 998: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 15:06:00 2025
- Line 1027: <<2025-Aug-22 15:08>> TimeoutError: The read operation timed out
- Line 1039: <<2025-Aug-22 15:08>> raise reraise(type(error), error, _stacktrace)
- Line 1049: <<2025-Aug-22 15:08>> raise ReadTimeoutError(
- Line 1050: <<2025-Aug-22 15:08>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 1099: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 15:16:00 2025
- Line 1128: <<2025-Aug-22 15:18>> TimeoutError: _ssl.c:983: The handshake operation timed out
- Line 1140: <<2025-Aug-22 15:18>> raise reraise(type(error), error, _stacktrace)
- Line 1152: <<2025-Aug-22 15:18>> raise ReadTimeoutError(
- Line 1153: <<2025-Aug-22 15:18>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 1196: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 15:26:00 2025
- Line 1225: <<2025-Aug-22 15:28>> TimeoutError: The read operation timed out
- Line 1237: <<2025-Aug-22 15:28>> raise reraise(type(error), error, _stacktrace)
- Line 1247: <<2025-Aug-22 15:28>> raise ReadTimeoutError(
- Line 1248: <<2025-Aug-22 15:28>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 1291: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 15:36:00 2025
- Line 1320: <<2025-Aug-22 15:39>> TimeoutError: _ssl.c:983: The handshake operation timed out
- Line 1332: <<2025-Aug-22 15:39>> raise reraise(type(error), error, _stacktrace)
- Line 1344: <<2025-Aug-22 15:39>> raise ReadTimeoutError(
- Line 1345: <<2025-Aug-22 15:39>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 1388: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 15:46:00 2025
- Line 1422: <<2025-Aug-22 15:49>> TimeoutError: _ssl.c:983: The handshake operation timed out
- Line 1434: <<2025-Aug-22 15:49>> raise reraise(type(error), error, _stacktrace)
- Line 1446: <<2025-Aug-22 15:49>> raise ReadTimeoutError(
- Line 1447: <<2025-Aug-22 15:49>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 1490: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 15:56:00 2025
- Line 1503: <<2025-Aug-22 15:58>> TimeoutError: timed out
- Line 1521: <<2025-Aug-22 15:58>> raise ConnectTimeoutError(
- Line 1522: <<2025-Aug-22 15:58>> urllib3.exceptions.ConnectTimeoutError: (, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)')
- Line 1534: <<2025-Aug-22 15:58>> raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
- Line 1536: <<2025-Aug-22 15:58>> urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/JUL (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
- Line 1578: <<2025-Aug-22 15:58>> requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/JUL (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
- Line 1579: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 16:06:00 2025
- Line 1580: WARNING - /proj/sot/ska3/flight/share/arc3/get_web_content.pl command timed out Fri Aug 22 16:16:00 2025
- Line 1581: WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 16:26:00 2025
- Line 1610: <<2025-Aug-22 16:29>> TimeoutError: _ssl.c:983: The handshake operation timed out
- Line 1622: <<2025-Aug-22 16:29>> raise reraise(type(error), error, _stacktrace)
- Line 1634: <<2025-Aug-22 16:29>> raise ReadTimeoutError(
- Line 1635: <<2025-Aug-22 16:29>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 1678: WARNING - /proj/sot/ska3/flight/share/arc3/make_timeline.py command timed out Fri Aug 22 16:36:02 2025
- Line 1707: <<2025-Aug-22 16:38>> TimeoutError: The read operation timed out
- Line 1719: <<2025-Aug-22 16:38>> raise reraise(type(error), error, _stacktrace)
- Line 1729: <<2025-Aug-22 16:38>> raise ReadTimeoutError(
- Line 1730: <<2025-Aug-22 16:38>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
- Line 1773: WARNING - /proj/sot/ska3/flight/share/arc3/make_timeline.py command timed out Fri Aug 22 16:46:02 2025
- Line 1783: <<2025-Aug-22 21:40>> Warning: failed to open URL https://services.swpc.noaa.gov/json/goes/primary//differential-protons-6-hour.json: Extra data: line 1 column 132919 (char 132918)
- Line 1783: <<2025-Aug-22 21:40>> Warning: failed to open URL https://services.swpc.noaa.gov/json/goes/primary//differential-protons-6-hour.json: Extra data: line 1 column 132919 (char 132918)
- Line 1798: <<2025-Aug-23 04:50>> tables.exceptions.HDF5ExtError: HDF5 error back trace
- Line 1805: <<2025-Aug-23 04:50>> open failed
- Line 1807: <<2025-Aug-23 04:50>> open failed
- Line 1813: <<2025-Aug-23 04:50>> driver lock request failed
- Line 1815: <<2025-Aug-23 04:50>> unable to lock file, errno = 11, error message = 'Resource temporarily unavailable'
- Line 1817: <<2025-Aug-23 04:50>> End of HDF5 error back trace
File contents:
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 06:11>> WARNING: p3 fluence not plotted, error : index -1 is out of bounds for axis 0 with size 0
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 06:16>> WARNING: p3 fluence not plotted, error : index -1 is out of bounds for axis 0 with size 0
############################################################
/proj/sot/ska3/flight/share/arc3/get_goes_x.py --h5=/proj/sot/ska3/flight/data/arc3/GOES_X.h5
############################################################
<<2025-Aug-22 06:36>> Malformed GOES_X data from SWPC; Table(json.loads(urldat)) did not succeed: Expecting value: line 1 column 163521 (char 163520)
WARNING - /proj/sot/ska3/flight/share/arc3/plot_goes_x.py command timed out Fri Aug 22 12:21:04 2025
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 13:16:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 13:17>> Traceback (most recent call last):
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
<<2025-Aug-22 13:17>> self._validate_conn(conn)
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
<<2025-Aug-22 13:17>> conn.connect()
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 741, in connect
<<2025-Aug-22 13:17>> sock_and_verified = _ssl_wrap_socket_and_match_hostname(
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 920, in _ssl_wrap_socket_and_match_hostname
<<2025-Aug-22 13:17>> ssl_sock = ssl_wrap_socket(
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 460, in ssl_wrap_socket
<<2025-Aug-22 13:17>> ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 504, in _ssl_wrap_socket_impl
<<2025-Aug-22 13:17>> return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 455, in wrap_socket
<<2025-Aug-22 13:17>> return self.sslsocket_class._create(
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1041, in _create
<<2025-Aug-22 13:17>> self.do_handshake()
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1319, in do_handshake
<<2025-Aug-22 13:17>> self._sslobj.do_handshake()
<<2025-Aug-22 13:17>> TimeoutError: _ssl.c:983: The handshake operation timed out
<<2025-Aug-22 13:17>>
<<2025-Aug-22 13:17>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 13:17>>
<<2025-Aug-22 13:17>> Traceback (most recent call last):
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 13:17>> resp = conn.urlopen(
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 13:17>> retries = retries.increment(
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 13:17>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 13:17>> raise value
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 13:17>> response = self._make_request(
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
<<2025-Aug-22 13:17>> raise new_e
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 466, in _make_request
<<2025-Aug-22 13:17>> self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 13:17>> raise ReadTimeoutError(
<<2025-Aug-22 13:17>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 13:17>>
<<2025-Aug-22 13:17>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 13:17>>
<<2025-Aug-22 13:17>> Traceback (most recent call last):
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 13:17>> main()
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 13:17>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 13:17>> cmds = commands.get_cmds(
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 13:17>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 13:17>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 13:17>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 13:17>> html = get_occweb_page(
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 13:17>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 13:17>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 13:17>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 13:17>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 13:17>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 13:17>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:17>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 13:17>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 13:17>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 13:26:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 13:27>> Traceback (most recent call last):
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request
<<2025-Aug-22 13:27>> response = conn.getresponse()
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 516, in getresponse
<<2025-Aug-22 13:27>> httplib_response = super().getresponse()
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 1428, in getresponse
<<2025-Aug-22 13:27>> response.begin()
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 331, in begin
<<2025-Aug-22 13:27>> version, status, reason = self._read_status()
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 292, in _read_status
<<2025-Aug-22 13:27>> line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/socket.py", line 720, in readinto
<<2025-Aug-22 13:27>> return self._sock.recv_into(b)
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1251, in recv_into
<<2025-Aug-22 13:27>> return self.read(nbytes, buffer)
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1103, in read
<<2025-Aug-22 13:27>> return self._sslobj.read(len, buffer)
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> TimeoutError: The read operation timed out
<<2025-Aug-22 13:27>>
<<2025-Aug-22 13:27>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 13:27>>
<<2025-Aug-22 13:27>> Traceback (most recent call last):
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 13:27>> resp = conn.urlopen(
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 13:27>> retries = retries.increment(
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 13:27>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 13:27>> raise value
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 13:27>> response = self._make_request(
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
<<2025-Aug-22 13:27>> self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 13:27>> raise ReadTimeoutError(
<<2025-Aug-22 13:27>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 13:27>>
<<2025-Aug-22 13:27>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 13:27>>
<<2025-Aug-22 13:27>> Traceback (most recent call last):
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 13:27>> main()
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 13:27>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 13:27>> cmds = commands.get_cmds(
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 13:27>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 13:27>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 13:27>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 13:27>> html = get_occweb_page(
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 13:27>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 13:27>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 13:27>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 13:27>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 724, in send
<<2025-Aug-22 13:27>> history = [resp for resp in gen]
<<2025-Aug-22 13:27>> ^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 265, in resolve_redirects
<<2025-Aug-22 13:27>> resp = self.send(
<<2025-Aug-22 13:27>> ^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 13:27>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 13:27>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:27>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 13:27>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 13:27>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 13:36:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/get_web_content.pl
############################################################
<<2025-Aug-22 13:37>> Warning: 500 read timeout for web data orbit_image (http://occweb.cfa.harvard.edu/occweb/web/webapps/ifot/ifot.php?r=home&t=qserver&format=orbit&detail=properties&fg=000000&bg=F0F1FF&size=300x230&scale=1400&showtimes=false&border=false&ul=6&tstart=2025:234:17:36:00.000&tstop=2025:237:09:06:00.000)
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 13:39>> Traceback (most recent call last):
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
<<2025-Aug-22 13:39>> sock = connection.create_connection(
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
<<2025-Aug-22 13:39>> raise err
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
<<2025-Aug-22 13:39>> sock.connect(sa)
<<2025-Aug-22 13:39>> TimeoutError: timed out
<<2025-Aug-22 13:39>>
<<2025-Aug-22 13:39>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 13:39>>
<<2025-Aug-22 13:39>> Traceback (most recent call last):
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 13:39>> response = self._make_request(
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
<<2025-Aug-22 13:39>> raise new_e
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
<<2025-Aug-22 13:39>> self._validate_conn(conn)
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
<<2025-Aug-22 13:39>> conn.connect()
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
<<2025-Aug-22 13:39>> self.sock = sock = self._new_conn()
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 207, in _new_conn
<<2025-Aug-22 13:39>> raise ConnectTimeoutError(
<<2025-Aug-22 13:39>> urllib3.exceptions.ConnectTimeoutError: (, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)')
<<2025-Aug-22 13:39>>
<<2025-Aug-22 13:39>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 13:39>>
<<2025-Aug-22 13:39>> Traceback (most recent call last):
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 13:39>> resp = conn.urlopen(
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 13:39>> retries = retries.increment(
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
<<2025-Aug-22 13:39>> raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/AUG (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Aug-22 13:39>>
<<2025-Aug-22 13:39>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 13:39>>
<<2025-Aug-22 13:39>> Traceback (most recent call last):
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 13:39>> main()
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 13:39>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 13:39>> cmds = commands.get_cmds(
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 13:39>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 13:39>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 13:39>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 13:39>> html = get_occweb_page(
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 13:39>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 13:39>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 13:39>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 13:39>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 13:39>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 13:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 688, in send
<<2025-Aug-22 13:39>> raise ConnectTimeout(e, request=request)
<<2025-Aug-22 13:39>> requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/AUG (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 13:46:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 13:48>> Traceback (most recent call last):
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request
<<2025-Aug-22 13:48>> response = conn.getresponse()
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 516, in getresponse
<<2025-Aug-22 13:48>> httplib_response = super().getresponse()
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 1428, in getresponse
<<2025-Aug-22 13:48>> response.begin()
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 331, in begin
<<2025-Aug-22 13:48>> version, status, reason = self._read_status()
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 292, in _read_status
<<2025-Aug-22 13:48>> line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/socket.py", line 720, in readinto
<<2025-Aug-22 13:48>> return self._sock.recv_into(b)
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1251, in recv_into
<<2025-Aug-22 13:48>> return self.read(nbytes, buffer)
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1103, in read
<<2025-Aug-22 13:48>> return self._sslobj.read(len, buffer)
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> TimeoutError: The read operation timed out
<<2025-Aug-22 13:48>>
<<2025-Aug-22 13:48>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 13:48>>
<<2025-Aug-22 13:48>> Traceback (most recent call last):
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 13:48>> resp = conn.urlopen(
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 13:48>> retries = retries.increment(
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 13:48>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 13:48>> raise value
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 13:48>> response = self._make_request(
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
<<2025-Aug-22 13:48>> self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 13:48>> raise ReadTimeoutError(
<<2025-Aug-22 13:48>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 13:48>>
<<2025-Aug-22 13:48>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 13:48>>
<<2025-Aug-22 13:48>> Traceback (most recent call last):
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 13:48>> main()
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 13:48>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 13:48>> cmds = commands.get_cmds(
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 13:48>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 13:48>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 13:48>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 13:48>> html = get_occweb_page(
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 13:48>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 13:48>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 13:48>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 13:48>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 13:48>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 13:48>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:48>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 13:48>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 13:48>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 13:56:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/get_web_content.pl
############################################################
<<2025-Aug-22 13:57>> Warning: 500 read timeout for web data orbit_image (http://occweb.cfa.harvard.edu/occweb/web/webapps/ifot/ifot.php?r=home&t=qserver&format=orbit&detail=properties&fg=000000&bg=F0F1FF&size=300x230&scale=1400&showtimes=false&border=false&ul=6&tstart=2025:234:17:56:00.000&tstop=2025:237:09:26:00.000)
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 13:58>> Traceback (most recent call last):
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request
<<2025-Aug-22 13:58>> response = conn.getresponse()
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 516, in getresponse
<<2025-Aug-22 13:58>> httplib_response = super().getresponse()
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 1428, in getresponse
<<2025-Aug-22 13:58>> response.begin()
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 331, in begin
<<2025-Aug-22 13:58>> version, status, reason = self._read_status()
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 292, in _read_status
<<2025-Aug-22 13:58>> line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/socket.py", line 720, in readinto
<<2025-Aug-22 13:58>> return self._sock.recv_into(b)
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1251, in recv_into
<<2025-Aug-22 13:58>> return self.read(nbytes, buffer)
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1103, in read
<<2025-Aug-22 13:58>> return self._sslobj.read(len, buffer)
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> TimeoutError: The read operation timed out
<<2025-Aug-22 13:58>>
<<2025-Aug-22 13:58>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 13:58>>
<<2025-Aug-22 13:58>> Traceback (most recent call last):
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 13:58>> resp = conn.urlopen(
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 13:58>> retries = retries.increment(
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 13:58>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 13:58>> raise value
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 13:58>> response = self._make_request(
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
<<2025-Aug-22 13:58>> self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 13:58>> raise ReadTimeoutError(
<<2025-Aug-22 13:58>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 13:58>>
<<2025-Aug-22 13:58>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 13:58>>
<<2025-Aug-22 13:58>> Traceback (most recent call last):
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 13:58>> main()
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 13:58>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 13:58>> cmds = commands.get_cmds(
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 13:58>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 13:58>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 13:58>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 13:58>> html = get_occweb_page(
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 13:58>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 13:58>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 13:58>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 13:58>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 724, in send
<<2025-Aug-22 13:58>> history = [resp for resp in gen]
<<2025-Aug-22 13:58>> ^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 265, in resolve_redirects
<<2025-Aug-22 13:58>> resp = self.send(
<<2025-Aug-22 13:58>> ^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 13:58>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 13:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 13:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 13:58>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 13:58>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 14:06:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/get_web_content.pl
############################################################
<<2025-Aug-22 14:07>> Warning: 500 Can't connect to occweb.cfa.harvard.edu:443 (Connection timed out) for web data orbit_image (http://occweb.cfa.harvard.edu/occweb/web/webapps/ifot/ifot.php?r=home&t=qserver&format=orbit&detail=properties&fg=000000&bg=F0F1FF&size=300x230&scale=1400&showtimes=false&border=false&ul=6&tstart=2025:234:18:06:01.000&tstop=2025:237:09:36:01.000)
WARNING - /proj/sot/ska3/flight/share/arc3/make_timeline.py command timed out Fri Aug 22 14:11:00 2025
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 14:16:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 14:18>> Traceback (most recent call last):
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request
<<2025-Aug-22 14:18>> response = conn.getresponse()
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 516, in getresponse
<<2025-Aug-22 14:18>> httplib_response = super().getresponse()
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 1428, in getresponse
<<2025-Aug-22 14:18>> response.begin()
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 331, in begin
<<2025-Aug-22 14:18>> version, status, reason = self._read_status()
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 292, in _read_status
<<2025-Aug-22 14:18>> line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/socket.py", line 720, in readinto
<<2025-Aug-22 14:18>> return self._sock.recv_into(b)
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1251, in recv_into
<<2025-Aug-22 14:18>> return self.read(nbytes, buffer)
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1103, in read
<<2025-Aug-22 14:18>> return self._sslobj.read(len, buffer)
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> TimeoutError: The read operation timed out
<<2025-Aug-22 14:18>>
<<2025-Aug-22 14:18>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 14:18>>
<<2025-Aug-22 14:18>> Traceback (most recent call last):
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 14:18>> resp = conn.urlopen(
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 14:18>> retries = retries.increment(
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 14:18>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 14:18>> raise value
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 14:18>> response = self._make_request(
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
<<2025-Aug-22 14:18>> self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 14:18>> raise ReadTimeoutError(
<<2025-Aug-22 14:18>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 14:18>>
<<2025-Aug-22 14:18>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 14:18>>
<<2025-Aug-22 14:18>> Traceback (most recent call last):
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 14:18>> main()
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 14:18>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 14:18>> cmds = commands.get_cmds(
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 14:18>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 14:18>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 14:18>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 14:18>> html = get_occweb_page(
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 14:18>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 14:18>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 14:18>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 14:18>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 14:18>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 14:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 14:18>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 14:18>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 14:26:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 14:28>> Traceback (most recent call last):
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request
<<2025-Aug-22 14:28>> response = conn.getresponse()
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 516, in getresponse
<<2025-Aug-22 14:28>> httplib_response = super().getresponse()
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 1428, in getresponse
<<2025-Aug-22 14:28>> response.begin()
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 331, in begin
<<2025-Aug-22 14:28>> version, status, reason = self._read_status()
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 292, in _read_status
<<2025-Aug-22 14:28>> line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/socket.py", line 720, in readinto
<<2025-Aug-22 14:28>> return self._sock.recv_into(b)
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1251, in recv_into
<<2025-Aug-22 14:28>> return self.read(nbytes, buffer)
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1103, in read
<<2025-Aug-22 14:28>> return self._sslobj.read(len, buffer)
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> TimeoutError: The read operation timed out
<<2025-Aug-22 14:28>>
<<2025-Aug-22 14:28>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 14:28>>
<<2025-Aug-22 14:28>> Traceback (most recent call last):
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 14:28>> resp = conn.urlopen(
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 14:28>> retries = retries.increment(
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 14:28>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 14:28>> raise value
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 14:28>> response = self._make_request(
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
<<2025-Aug-22 14:28>> self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 14:28>> raise ReadTimeoutError(
<<2025-Aug-22 14:28>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 14:28>>
<<2025-Aug-22 14:28>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 14:28>>
<<2025-Aug-22 14:28>> Traceback (most recent call last):
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 14:28>> main()
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 14:28>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 14:28>> cmds = commands.get_cmds(
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 14:28>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 14:28>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 14:28>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 14:28>> html = get_occweb_page(
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 14:28>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 14:28>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 14:28>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 14:28>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 14:28>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 14:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 14:28>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 14:28>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 14:36:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/get_web_content.pl
############################################################
<<2025-Aug-22 14:37>> Warning: 500 Can't connect to occweb.cfa.harvard.edu:443 for web data orbit_image (http://occweb.cfa.harvard.edu/occweb/web/webapps/ifot/ifot.php?r=home&t=qserver&format=orbit&detail=properties&fg=000000&bg=F0F1FF&size=300x230&scale=1400&showtimes=false&border=false&ul=6&tstart=2025:234:18:36:00.000&tstop=2025:237:10:06:00.000)
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 14:39>> Traceback (most recent call last):
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request
<<2025-Aug-22 14:39>> response = conn.getresponse()
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 516, in getresponse
<<2025-Aug-22 14:39>> httplib_response = super().getresponse()
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 1428, in getresponse
<<2025-Aug-22 14:39>> response.begin()
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 331, in begin
<<2025-Aug-22 14:39>> version, status, reason = self._read_status()
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 292, in _read_status
<<2025-Aug-22 14:39>> line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/socket.py", line 720, in readinto
<<2025-Aug-22 14:39>> return self._sock.recv_into(b)
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1251, in recv_into
<<2025-Aug-22 14:39>> return self.read(nbytes, buffer)
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1103, in read
<<2025-Aug-22 14:39>> return self._sslobj.read(len, buffer)
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> TimeoutError: The read operation timed out
<<2025-Aug-22 14:39>>
<<2025-Aug-22 14:39>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 14:39>>
<<2025-Aug-22 14:39>> Traceback (most recent call last):
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 14:39>> resp = conn.urlopen(
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 14:39>> retries = retries.increment(
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 14:39>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 14:39>> raise value
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 14:39>> response = self._make_request(
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
<<2025-Aug-22 14:39>> self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 14:39>> raise ReadTimeoutError(
<<2025-Aug-22 14:39>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 14:39>>
<<2025-Aug-22 14:39>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 14:39>>
<<2025-Aug-22 14:39>> Traceback (most recent call last):
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 14:39>> main()
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 14:39>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 14:39>> cmds = commands.get_cmds(
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 14:39>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 14:39>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 14:39>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 14:39>> html = get_occweb_page(
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 14:39>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 14:39>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 14:39>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 14:39>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 724, in send
<<2025-Aug-22 14:39>> history = [resp for resp in gen]
<<2025-Aug-22 14:39>> ^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 265, in resolve_redirects
<<2025-Aug-22 14:39>> resp = self.send(
<<2025-Aug-22 14:39>> ^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 14:39>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 14:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 14:39>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 14:39>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 14:46:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 14:47>> Traceback (most recent call last):
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
<<2025-Aug-22 14:47>> sock = connection.create_connection(
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
<<2025-Aug-22 14:47>> raise err
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
<<2025-Aug-22 14:47>> sock.connect(sa)
<<2025-Aug-22 14:47>> TimeoutError: timed out
<<2025-Aug-22 14:47>>
<<2025-Aug-22 14:47>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 14:47>>
<<2025-Aug-22 14:47>> Traceback (most recent call last):
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 14:47>> response = self._make_request(
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
<<2025-Aug-22 14:47>> raise new_e
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
<<2025-Aug-22 14:47>> self._validate_conn(conn)
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
<<2025-Aug-22 14:47>> conn.connect()
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
<<2025-Aug-22 14:47>> self.sock = sock = self._new_conn()
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 207, in _new_conn
<<2025-Aug-22 14:47>> raise ConnectTimeoutError(
<<2025-Aug-22 14:47>> urllib3.exceptions.ConnectTimeoutError: (, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)')
<<2025-Aug-22 14:47>>
<<2025-Aug-22 14:47>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 14:47>>
<<2025-Aug-22 14:47>> Traceback (most recent call last):
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 14:47>> resp = conn.urlopen(
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 14:47>> retries = retries.increment(
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
<<2025-Aug-22 14:47>> raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/JUL (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Aug-22 14:47>>
<<2025-Aug-22 14:47>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 14:47>>
<<2025-Aug-22 14:47>> Traceback (most recent call last):
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 14:47>> main()
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 14:47>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 14:47>> cmds = commands.get_cmds(
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 14:47>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 14:47>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 14:47>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 14:47>> html = get_occweb_page(
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 14:47>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 14:47>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 14:47>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 14:47>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 14:47>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 14:47>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:47>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 688, in send
<<2025-Aug-22 14:47>> raise ConnectTimeout(e, request=request)
<<2025-Aug-22 14:47>> requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/JUL (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 14:56:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 14:58>> Traceback (most recent call last):
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
<<2025-Aug-22 14:58>> self._validate_conn(conn)
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
<<2025-Aug-22 14:58>> conn.connect()
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 741, in connect
<<2025-Aug-22 14:58>> sock_and_verified = _ssl_wrap_socket_and_match_hostname(
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 920, in _ssl_wrap_socket_and_match_hostname
<<2025-Aug-22 14:58>> ssl_sock = ssl_wrap_socket(
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 460, in ssl_wrap_socket
<<2025-Aug-22 14:58>> ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 504, in _ssl_wrap_socket_impl
<<2025-Aug-22 14:58>> return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 455, in wrap_socket
<<2025-Aug-22 14:58>> return self.sslsocket_class._create(
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1041, in _create
<<2025-Aug-22 14:58>> self.do_handshake()
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1319, in do_handshake
<<2025-Aug-22 14:58>> self._sslobj.do_handshake()
<<2025-Aug-22 14:58>> TimeoutError: _ssl.c:983: The handshake operation timed out
<<2025-Aug-22 14:58>>
<<2025-Aug-22 14:58>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 14:58>>
<<2025-Aug-22 14:58>> Traceback (most recent call last):
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 14:58>> resp = conn.urlopen(
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 14:58>> retries = retries.increment(
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 14:58>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 14:58>> raise value
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 14:58>> response = self._make_request(
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
<<2025-Aug-22 14:58>> raise new_e
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 466, in _make_request
<<2025-Aug-22 14:58>> self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 14:58>> raise ReadTimeoutError(
<<2025-Aug-22 14:58>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 14:58>>
<<2025-Aug-22 14:58>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 14:58>>
<<2025-Aug-22 14:58>> Traceback (most recent call last):
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 14:58>> main()
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 14:58>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 14:58>> cmds = commands.get_cmds(
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 14:58>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 14:58>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 14:58>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 14:58>> html = get_occweb_page(
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 14:58>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 14:58>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 14:58>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 14:58>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 14:58>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 14:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 14:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 14:58>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 14:58>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 15:06:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 15:08>> Traceback (most recent call last):
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request
<<2025-Aug-22 15:08>> response = conn.getresponse()
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 516, in getresponse
<<2025-Aug-22 15:08>> httplib_response = super().getresponse()
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 1428, in getresponse
<<2025-Aug-22 15:08>> response.begin()
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 331, in begin
<<2025-Aug-22 15:08>> version, status, reason = self._read_status()
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 292, in _read_status
<<2025-Aug-22 15:08>> line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/socket.py", line 720, in readinto
<<2025-Aug-22 15:08>> return self._sock.recv_into(b)
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1251, in recv_into
<<2025-Aug-22 15:08>> return self.read(nbytes, buffer)
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1103, in read
<<2025-Aug-22 15:08>> return self._sslobj.read(len, buffer)
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> TimeoutError: The read operation timed out
<<2025-Aug-22 15:08>>
<<2025-Aug-22 15:08>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 15:08>>
<<2025-Aug-22 15:08>> Traceback (most recent call last):
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 15:08>> resp = conn.urlopen(
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 15:08>> retries = retries.increment(
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 15:08>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 15:08>> raise value
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 15:08>> response = self._make_request(
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
<<2025-Aug-22 15:08>> self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 15:08>> raise ReadTimeoutError(
<<2025-Aug-22 15:08>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 15:08>>
<<2025-Aug-22 15:08>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 15:08>>
<<2025-Aug-22 15:08>> Traceback (most recent call last):
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 15:08>> main()
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 15:08>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 15:08>> cmds = commands.get_cmds(
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 15:08>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 15:08>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 15:08>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 15:08>> html = get_occweb_page(
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 15:08>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 15:08>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 15:08>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 15:08>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 724, in send
<<2025-Aug-22 15:08>> history = [resp for resp in gen]
<<2025-Aug-22 15:08>> ^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 265, in resolve_redirects
<<2025-Aug-22 15:08>> resp = self.send(
<<2025-Aug-22 15:08>> ^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 15:08>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 15:08>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:08>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 15:08>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 15:08>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 15:16:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 15:18>> Traceback (most recent call last):
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
<<2025-Aug-22 15:18>> self._validate_conn(conn)
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
<<2025-Aug-22 15:18>> conn.connect()
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 741, in connect
<<2025-Aug-22 15:18>> sock_and_verified = _ssl_wrap_socket_and_match_hostname(
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 920, in _ssl_wrap_socket_and_match_hostname
<<2025-Aug-22 15:18>> ssl_sock = ssl_wrap_socket(
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 460, in ssl_wrap_socket
<<2025-Aug-22 15:18>> ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 504, in _ssl_wrap_socket_impl
<<2025-Aug-22 15:18>> return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 455, in wrap_socket
<<2025-Aug-22 15:18>> return self.sslsocket_class._create(
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1041, in _create
<<2025-Aug-22 15:18>> self.do_handshake()
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1319, in do_handshake
<<2025-Aug-22 15:18>> self._sslobj.do_handshake()
<<2025-Aug-22 15:18>> TimeoutError: _ssl.c:983: The handshake operation timed out
<<2025-Aug-22 15:18>>
<<2025-Aug-22 15:18>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 15:18>>
<<2025-Aug-22 15:18>> Traceback (most recent call last):
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 15:18>> resp = conn.urlopen(
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 15:18>> retries = retries.increment(
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 15:18>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 15:18>> raise value
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 15:18>> response = self._make_request(
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
<<2025-Aug-22 15:18>> raise new_e
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 466, in _make_request
<<2025-Aug-22 15:18>> self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 15:18>> raise ReadTimeoutError(
<<2025-Aug-22 15:18>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 15:18>>
<<2025-Aug-22 15:18>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 15:18>>
<<2025-Aug-22 15:18>> Traceback (most recent call last):
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 15:18>> main()
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 15:18>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 15:18>> cmds = commands.get_cmds(
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 15:18>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 15:18>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 15:18>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 15:18>> html = get_occweb_page(
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 15:18>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 15:18>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 15:18>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 15:18>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 15:18>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 15:18>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:18>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 15:18>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 15:18>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 15:26:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 15:28>> Traceback (most recent call last):
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request
<<2025-Aug-22 15:28>> response = conn.getresponse()
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 516, in getresponse
<<2025-Aug-22 15:28>> httplib_response = super().getresponse()
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 1428, in getresponse
<<2025-Aug-22 15:28>> response.begin()
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 331, in begin
<<2025-Aug-22 15:28>> version, status, reason = self._read_status()
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 292, in _read_status
<<2025-Aug-22 15:28>> line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/socket.py", line 720, in readinto
<<2025-Aug-22 15:28>> return self._sock.recv_into(b)
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1251, in recv_into
<<2025-Aug-22 15:28>> return self.read(nbytes, buffer)
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1103, in read
<<2025-Aug-22 15:28>> return self._sslobj.read(len, buffer)
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> TimeoutError: The read operation timed out
<<2025-Aug-22 15:28>>
<<2025-Aug-22 15:28>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 15:28>>
<<2025-Aug-22 15:28>> Traceback (most recent call last):
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 15:28>> resp = conn.urlopen(
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 15:28>> retries = retries.increment(
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 15:28>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 15:28>> raise value
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 15:28>> response = self._make_request(
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
<<2025-Aug-22 15:28>> self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 15:28>> raise ReadTimeoutError(
<<2025-Aug-22 15:28>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 15:28>>
<<2025-Aug-22 15:28>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 15:28>>
<<2025-Aug-22 15:28>> Traceback (most recent call last):
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 15:28>> main()
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 15:28>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 15:28>> cmds = commands.get_cmds(
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 15:28>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 15:28>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 15:28>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 15:28>> html = get_occweb_page(
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 15:28>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 15:28>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 15:28>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 15:28>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 15:28>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 15:28>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:28>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 15:28>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 15:28>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 15:36:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 15:39>> Traceback (most recent call last):
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
<<2025-Aug-22 15:39>> self._validate_conn(conn)
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
<<2025-Aug-22 15:39>> conn.connect()
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 741, in connect
<<2025-Aug-22 15:39>> sock_and_verified = _ssl_wrap_socket_and_match_hostname(
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 920, in _ssl_wrap_socket_and_match_hostname
<<2025-Aug-22 15:39>> ssl_sock = ssl_wrap_socket(
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 460, in ssl_wrap_socket
<<2025-Aug-22 15:39>> ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 504, in _ssl_wrap_socket_impl
<<2025-Aug-22 15:39>> return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 455, in wrap_socket
<<2025-Aug-22 15:39>> return self.sslsocket_class._create(
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1041, in _create
<<2025-Aug-22 15:39>> self.do_handshake()
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1319, in do_handshake
<<2025-Aug-22 15:39>> self._sslobj.do_handshake()
<<2025-Aug-22 15:39>> TimeoutError: _ssl.c:983: The handshake operation timed out
<<2025-Aug-22 15:39>>
<<2025-Aug-22 15:39>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 15:39>>
<<2025-Aug-22 15:39>> Traceback (most recent call last):
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 15:39>> resp = conn.urlopen(
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 15:39>> retries = retries.increment(
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 15:39>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 15:39>> raise value
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 15:39>> response = self._make_request(
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
<<2025-Aug-22 15:39>> raise new_e
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 466, in _make_request
<<2025-Aug-22 15:39>> self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 15:39>> raise ReadTimeoutError(
<<2025-Aug-22 15:39>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 15:39>>
<<2025-Aug-22 15:39>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 15:39>>
<<2025-Aug-22 15:39>> Traceback (most recent call last):
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 15:39>> main()
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 15:39>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 15:39>> cmds = commands.get_cmds(
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 15:39>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 15:39>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 15:39>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 15:39>> html = get_occweb_page(
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 15:39>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 15:39>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 15:39>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 15:39>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 15:39>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 15:39>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:39>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 15:39>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 15:39>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 15:46:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/get_web_content.pl
############################################################
<<2025-Aug-22 15:47>> Warning: 500 Can't connect to occweb.cfa.harvard.edu:443 for web data orbit_image (http://occweb.cfa.harvard.edu/occweb/web/webapps/ifot/ifot.php?r=home&t=qserver&format=orbit&detail=properties&fg=000000&bg=F0F1FF&size=300x230&scale=1400&showtimes=false&border=false&ul=6&tstart=2025:234:19:46:00.000&tstop=2025:237:11:16:00.000)
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 15:49>> Traceback (most recent call last):
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
<<2025-Aug-22 15:49>> self._validate_conn(conn)
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
<<2025-Aug-22 15:49>> conn.connect()
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 741, in connect
<<2025-Aug-22 15:49>> sock_and_verified = _ssl_wrap_socket_and_match_hostname(
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 920, in _ssl_wrap_socket_and_match_hostname
<<2025-Aug-22 15:49>> ssl_sock = ssl_wrap_socket(
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 460, in ssl_wrap_socket
<<2025-Aug-22 15:49>> ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 504, in _ssl_wrap_socket_impl
<<2025-Aug-22 15:49>> return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 455, in wrap_socket
<<2025-Aug-22 15:49>> return self.sslsocket_class._create(
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1041, in _create
<<2025-Aug-22 15:49>> self.do_handshake()
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1319, in do_handshake
<<2025-Aug-22 15:49>> self._sslobj.do_handshake()
<<2025-Aug-22 15:49>> TimeoutError: _ssl.c:983: The handshake operation timed out
<<2025-Aug-22 15:49>>
<<2025-Aug-22 15:49>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 15:49>>
<<2025-Aug-22 15:49>> Traceback (most recent call last):
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 15:49>> resp = conn.urlopen(
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 15:49>> retries = retries.increment(
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 15:49>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 15:49>> raise value
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 15:49>> response = self._make_request(
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
<<2025-Aug-22 15:49>> raise new_e
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 466, in _make_request
<<2025-Aug-22 15:49>> self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 15:49>> raise ReadTimeoutError(
<<2025-Aug-22 15:49>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 15:49>>
<<2025-Aug-22 15:49>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 15:49>>
<<2025-Aug-22 15:49>> Traceback (most recent call last):
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 15:49>> main()
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 15:49>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 15:49>> cmds = commands.get_cmds(
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 15:49>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 15:49>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 15:49>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 15:49>> html = get_occweb_page(
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 15:49>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 15:49>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 15:49>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 15:49>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 15:49>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 15:49>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:49>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 15:49>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 15:49>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 15:56:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 15:58>> Traceback (most recent call last):
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
<<2025-Aug-22 15:58>> sock = connection.create_connection(
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
<<2025-Aug-22 15:58>> raise err
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
<<2025-Aug-22 15:58>> sock.connect(sa)
<<2025-Aug-22 15:58>> TimeoutError: timed out
<<2025-Aug-22 15:58>>
<<2025-Aug-22 15:58>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 15:58>>
<<2025-Aug-22 15:58>> Traceback (most recent call last):
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 15:58>> response = self._make_request(
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
<<2025-Aug-22 15:58>> raise new_e
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
<<2025-Aug-22 15:58>> self._validate_conn(conn)
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
<<2025-Aug-22 15:58>> conn.connect()
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
<<2025-Aug-22 15:58>> self.sock = sock = self._new_conn()
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 207, in _new_conn
<<2025-Aug-22 15:58>> raise ConnectTimeoutError(
<<2025-Aug-22 15:58>> urllib3.exceptions.ConnectTimeoutError: (, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)')
<<2025-Aug-22 15:58>>
<<2025-Aug-22 15:58>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 15:58>>
<<2025-Aug-22 15:58>> Traceback (most recent call last):
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 15:58>> resp = conn.urlopen(
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 15:58>> retries = retries.increment(
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
<<2025-Aug-22 15:58>> raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/JUL (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
<<2025-Aug-22 15:58>>
<<2025-Aug-22 15:58>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 15:58>>
<<2025-Aug-22 15:58>> Traceback (most recent call last):
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 15:58>> main()
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 15:58>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 15:58>> cmds = commands.get_cmds(
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 15:58>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 15:58>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 15:58>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 15:58>> html = get_occweb_page(
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 15:58>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 15:58>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 15:58>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 15:58>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 15:58>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 15:58>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 15:58>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 688, in send
<<2025-Aug-22 15:58>> raise ConnectTimeout(e, request=request)
<<2025-Aug-22 15:58>> requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Max retries exceeded with url: /occweb/FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/JUL (Caused by ConnectTimeoutError(, 'Connection to occweb.cfa.harvard.edu timed out. (connect timeout=30)'))
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 16:06:00 2025
WARNING - /proj/sot/ska3/flight/share/arc3/get_web_content.pl command timed out Fri Aug 22 16:16:00 2025
WARNING - /proj/sot/ska3/flight/share/arc3/get_iFOT_events.pl command timed out Fri Aug 22 16:26:00 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 16:29>> Traceback (most recent call last):
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
<<2025-Aug-22 16:29>> self._validate_conn(conn)
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
<<2025-Aug-22 16:29>> conn.connect()
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 741, in connect
<<2025-Aug-22 16:29>> sock_and_verified = _ssl_wrap_socket_and_match_hostname(
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 920, in _ssl_wrap_socket_and_match_hostname
<<2025-Aug-22 16:29>> ssl_sock = ssl_wrap_socket(
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 460, in ssl_wrap_socket
<<2025-Aug-22 16:29>> ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 504, in _ssl_wrap_socket_impl
<<2025-Aug-22 16:29>> return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 455, in wrap_socket
<<2025-Aug-22 16:29>> return self.sslsocket_class._create(
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1041, in _create
<<2025-Aug-22 16:29>> self.do_handshake()
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1319, in do_handshake
<<2025-Aug-22 16:29>> self._sslobj.do_handshake()
<<2025-Aug-22 16:29>> TimeoutError: _ssl.c:983: The handshake operation timed out
<<2025-Aug-22 16:29>>
<<2025-Aug-22 16:29>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 16:29>>
<<2025-Aug-22 16:29>> Traceback (most recent call last):
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 16:29>> resp = conn.urlopen(
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 16:29>> retries = retries.increment(
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 16:29>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 16:29>> raise value
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 16:29>> response = self._make_request(
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
<<2025-Aug-22 16:29>> raise new_e
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 466, in _make_request
<<2025-Aug-22 16:29>> self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 16:29>> raise ReadTimeoutError(
<<2025-Aug-22 16:29>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 16:29>>
<<2025-Aug-22 16:29>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 16:29>>
<<2025-Aug-22 16:29>> Traceback (most recent call last):
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 16:29>> main()
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 16:29>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 16:29>> cmds = commands.get_cmds(
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 16:29>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 16:29>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 16:29>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 16:29>> html = get_occweb_page(
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 16:29>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 16:29>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 16:29>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 16:29>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 16:29>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 16:29>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:29>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 16:29>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 16:29>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/make_timeline.py command timed out Fri Aug 22 16:36:02 2025
############################################################
/proj/sot/ska3/flight/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 16:38>> Traceback (most recent call last):
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 534, in _make_request
<<2025-Aug-22 16:38>> response = conn.getresponse()
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connection.py", line 516, in getresponse
<<2025-Aug-22 16:38>> httplib_response = super().getresponse()
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 1428, in getresponse
<<2025-Aug-22 16:38>> response.begin()
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 331, in begin
<<2025-Aug-22 16:38>> version, status, reason = self._read_status()
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/http/client.py", line 292, in _read_status
<<2025-Aug-22 16:38>> line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/socket.py", line 720, in readinto
<<2025-Aug-22 16:38>> return self._sock.recv_into(b)
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1251, in recv_into
<<2025-Aug-22 16:38>> return self.read(nbytes, buffer)
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/ssl.py", line 1103, in read
<<2025-Aug-22 16:38>> return self._sslobj.read(len, buffer)
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> TimeoutError: The read operation timed out
<<2025-Aug-22 16:38>>
<<2025-Aug-22 16:38>> The above exception was the direct cause of the following exception:
<<2025-Aug-22 16:38>>
<<2025-Aug-22 16:38>> Traceback (most recent call last):
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
<<2025-Aug-22 16:38>> resp = conn.urlopen(
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
<<2025-Aug-22 16:38>> retries = retries.increment(
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
<<2025-Aug-22 16:38>> raise reraise(type(error), error, _stacktrace)
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
<<2025-Aug-22 16:38>> raise value
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
<<2025-Aug-22 16:38>> response = self._make_request(
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
<<2025-Aug-22 16:38>> self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
<<2025-Aug-22 16:38>> raise ReadTimeoutError(
<<2025-Aug-22 16:38>> urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
<<2025-Aug-22 16:38>>
<<2025-Aug-22 16:38>> During handling of the above exception, another exception occurred:
<<2025-Aug-22 16:38>>
<<2025-Aug-22 16:38>> Traceback (most recent call last):
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 1255, in
<<2025-Aug-22 16:38>> main()
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/share/arc3/make_timeline.py", line 634, in main
<<2025-Aug-22 16:38>> states = kadi_states.get_states(start=start, stop=stop)
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/states.py", line 2334, in get_states
<<2025-Aug-22 16:38>> cmds = commands.get_cmds(
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 283, in get_cmds
<<2025-Aug-22 16:38>> cmds_recent = update_cmd_events_and_loads_and_get_cmds_recent(
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 426, in update_cmd_events_and_loads_and_get_cmds_recent
<<2025-Aug-22 16:38>> loads = update_loads(scenario, lookback=lookback, stop_loads=stop_loads)
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/commands/commands_v2.py", line 1190, in update_loads
<<2025-Aug-22 16:38>> contents = occweb.get_occweb_dir(dir_year_month)
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 367, in get_occweb_dir
<<2025-Aug-22 16:38>> html = get_occweb_page(
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/kadi/occweb.py", line 328, in get_occweb_page
<<2025-Aug-22 16:38>> req = requests.get(url, auth=auth, timeout=timeout)
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 73, in get
<<2025-Aug-22 16:38>> return request("get", url, params=params, **kwargs)
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/api.py", line 59, in request
<<2025-Aug-22 16:38>> return session.request(method=method, url=url, **kwargs)
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
<<2025-Aug-22 16:38>> resp = self.send(prep, **send_kwargs)
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
<<2025-Aug-22 16:38>> r = adapter.send(request, **kwargs)
<<2025-Aug-22 16:38>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-22 16:38>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/requests/adapters.py", line 713, in send
<<2025-Aug-22 16:38>> raise ReadTimeout(e, request=request)
<<2025-Aug-22 16:38>> requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='occweb.cfa.harvard.edu', port=443): Read timed out. (read timeout=30)
WARNING - /proj/sot/ska3/flight/share/arc3/make_timeline.py command timed out Fri Aug 22 16:46:02 2025
############################################################
/proj/sot/ska3/flight/share/arc3/get_goes_x.py --h5=/proj/sot/ska3/flight/data/arc3/GOES_X.h5
############################################################
<<2025-Aug-22 21:20>> Malformed GOES_X data from SWPC; Table(json.loads(urldat)) did not succeed: Expecting value: line 1 column 163172 (char 163171)
############################################################
/proj/sot/ska3/flight/share/arc3/get_hrc.py --h5=/proj/sot/ska3/flight/data/arc3/hrc_shield.h5 --data-dir=/proj/sot/ska3/flight/data/arc3
############################################################
<<2025-Aug-22 21:40>> Warning: failed to open URL https://services.swpc.noaa.gov/json/goes/primary//differential-protons-6-hour.json: Extra data: line 1 column 132919 (char 132918)
############################################################
/proj/sot/ska3/flight/share/arc3/get_ace.py --h5=/proj/sot/ska3/flight/data/arc3/ACE.h5
############################################################
<<2025-Aug-23 04:50>> Traceback (most recent call last):
<<2025-Aug-23 04:50>> File "/proj/sot/ska3/flight/share/arc3/get_ace.py", line 67, in
<<2025-Aug-23 04:50>> h5 = tables.open_file(
<<2025-Aug-23 04:50>> ^^^^^^^^^^^^^^^^^
<<2025-Aug-23 04:50>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/tables/file.py", line 296, in open_file
<<2025-Aug-23 04:50>> return File(filename, mode, title, root_uep, filters, **kwargs)
<<2025-Aug-23 04:50>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<<2025-Aug-23 04:50>> File "/proj/sot/ska3/flight/lib/python3.12/site-packages/tables/file.py", line 746, in __init__
<<2025-Aug-23 04:50>> self._g_new(filename, mode, **params)
<<2025-Aug-23 04:50>> File "tables/hdf5extension.pyx", line 514, in tables.hdf5extension.File._g_new
<<2025-Aug-23 04:50>> tables.exceptions.HDF5ExtError: HDF5 error back trace
<<2025-Aug-23 04:50>>
<<2025-Aug-23 04:50>> File "H5F.c", line 836, in H5Fopen
<<2025-Aug-23 04:50>> unable to synchronously open file
<<2025-Aug-23 04:50>> File "H5F.c", line 796, in H5F__open_api_common
<<2025-Aug-23 04:50>> unable to open file
<<2025-Aug-23 04:50>> File "H5VLcallback.c", line 3863, in H5VL_file_open
<<2025-Aug-23 04:50>> open failed
<<2025-Aug-23 04:50>> File "H5VLcallback.c", line 3675, in H5VL__file_open
<<2025-Aug-23 04:50>> open failed
<<2025-Aug-23 04:50>> File "H5VLnative_file.c", line 128, in H5VL__native_file_open
<<2025-Aug-23 04:50>> unable to open file
<<2025-Aug-23 04:50>> File "H5Fint.c", line 1960, in H5F_open
<<2025-Aug-23 04:50>> unable to lock the file
<<2025-Aug-23 04:50>> File "H5FD.c", line 2411, in H5FD_lock
<<2025-Aug-23 04:50>> driver lock request failed
<<2025-Aug-23 04:50>> File "H5FDsec2.c", line 941, in H5FD__sec2_lock
<<2025-Aug-23 04:50>> unable to lock file, errno = 11, error message = 'Resource temporarily unavailable'
<<2025-Aug-23 04:50>>
<<2025-Aug-23 04:50>> End of HDF5 error back trace
<<2025-Aug-23 04:50>>
<<2025-Aug-23 04:50>> Unable to open/create file '/proj/sot/ska3/flight/data/arc3/ACE.h5'