beekeeper | SIGINT randomly closes with `-2` return code and do not removes the `beekeeper.pid` file
FYI: @Trela
This behavior could be observed in random failing clive tests.
In the logs of the failing test, we can observe:
2023-08-31 14:07:33.741 | ℹ️ INFO | clive.__private.core.beekeeper.handle:__start:175 - Starting Beekeeper...
2023-08-31 14:07:33.742 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:listen:36 - Notifications server is listening on 39353...
2023-08-31 14:07:33.764 | ℹ️ INFO | clive.__private.core.beekeeper.notifications:notify:40 - Got notification: {'value': {'current_status': 'starting without a session'}, 'time': '2023-08-31T14:07:33', 'name': 'hived_status'}
2023-08-31 14:07:33.766 | ℹ️ INFO | aiohttp.web_log:log:206 - 127.0.0.1 [31/Aug/2023:14:07:33 +0000] "PUT / HTTP/1.1" 204 0 "-" "-"
2023-08-31 14:07:33.767 | ℹ️ INFO | clive.__private.core.beekeeper.notifications:notify:40 - Got notification: {'value': {'current_status': 'signals attached'}, 'time': '2023-08-31T14:07:33', 'name': 'hived_status'}
2023-08-31 14:07:33.767 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:notify:50 - Beekeeper reports to be ready
2023-08-31 14:07:33.768 | ℹ️ INFO | aiohttp.web_log:log:206 - 127.0.0.1 [31/Aug/2023:14:07:33 +0000] "PUT / HTTP/1.1" 204 0 "-" "-"
2023-08-31 14:07:33.768 | ℹ️ INFO | clive.__private.core.beekeeper.notifications:notify:40 - Got notification: {'value': {'address': '0.0.0.0', 'port': 39881, 'type': 'HTTP'}, 'time': '2023-08-31T14:07:33', 'name': 'webserver listening'}
2023-08-31 14:07:33.769 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:notify:47 - Got notification with http address on: http://127.0.0.1:39881
2023-08-31 14:07:33.769 | ℹ️ INFO | aiohttp.web_log:log:206 - 127.0.0.1 [31/Aug/2023:14:07:33 +0000] "PUT / HTTP/1.1" 204 0 "-" "-"
2023-08-31 14:07:38.756 | 🐞 DEBUG | clive.__private.core.beekeeper.handle:__run_beekeeper:207 - Got webserver http endpoint: `http://127.0.0.1:39881`
2023-08-31 14:07:38.757 | ℹ️ INFO | clive.__private.core.beekeeper.handle:__start:186 - Beekeeper started on http://127.0.0.1:39881.
2023-08-31 14:07:38.763 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=CreateSession(token='d540aaa8ee678152ad698338c37b111ab5f033387ea7c40b4c67e117207b386a')
2023-08-31 14:07:38.765 | ℹ️ INFO | clive.__private.core.commands.abc.command:_log_execution_info:40 - Executing command: CreateWallet
2023-08-31 14:07:38.769 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=Create(password='password')
2023-08-31 14:07:38.769 | ℹ️ INFO | clive.__private.core.app_state:activate:33 - Mode switched to ACTIVE.
2023-08-31 14:07:38.771 | ℹ️ INFO | clive.__private.core.beekeeper.handle:close:160 - Closing Beekeeper...
2023-08-31 14:07:38.772 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=EmptyResponse()
2023-08-31 14:07:38.774 | 🐞 DEBUG | clive.__private.core.beekeeper.executable:close:105 - Beekeeper closed with return code of `-2`.
2023-08-31 14:07:38.775 | 🐞 DEBUG | clive.__private.core.beekeeper.executable:__wait_for_pid_file_to_be_deleted:140 - Beekeeper PID file was deleted in 0.00 seconds.
2023-08-31 14:07:38.775 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:close:76 - Notifications server closed
2023-08-31 14:07:38.775 | ℹ️ INFO | clive.__private.core.beekeeper.handle:close:166 - Beekeeper closed.
2023-08-31 14:07:38.776 | ℹ️ INFO | clive.__private.core.beekeeper.handle:__start:175 - Starting Beekeeper...
2023-08-31 14:07:38.776 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:listen:36 - Notifications server is listening on 34353...
2023-08-31 14:07:38.793 | ℹ️ INFO | clive.__private.core.beekeeper.notifications:notify:40 - Got notification: {'value': {'current_status': 'starting without a session'}, 'time': '2023-08-31T14:07:38', 'name': 'hived_status'}
2023-08-31 14:07:38.794 | ℹ️ INFO | aiohttp.web_log:log:206 - 127.0.0.1 [31/Aug/2023:14:07:38 +0000] "PUT / HTTP/1.1" 204 0 "-" "-"
2023-08-31 14:07:38.795 | ℹ️ INFO | clive.__private.core.beekeeper.notifications:notify:40 - Got notification: {'value': {'current_status': 'signals attached'}, 'time': '2023-08-31T14:07:38', 'name': 'hived_status'}
2023-08-31 14:07:38.796 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:notify:50 - Beekeeper reports to be ready
2023-08-31 14:07:38.796 | ℹ️ INFO | aiohttp.web_log:log:206 - 127.0.0.1 [31/Aug/2023:14:07:38 +0000] "PUT / HTTP/1.1" 204 0 "-" "-"
2023-08-31 14:07:38.797 | ℹ️ INFO | clive.__private.core.beekeeper.notifications:notify:40 - Got notification: {'value': {'address': '0.0.0.0', 'port': 44521, 'type': 'HTTP'}, 'time': '2023-08-31T14:07:38', 'name': 'webserver listening'}
2023-08-31 14:07:38.798 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:notify:47 - Got notification with http address on: http://127.0.0.1:44521
2023-08-31 14:07:38.798 | ℹ️ INFO | aiohttp.web_log:log:206 - 127.0.0.1 [31/Aug/2023:14:07:38 +0000] "PUT / HTTP/1.1" 204 0 "-" "-"
2023-08-31 14:07:43.790 | 🐞 DEBUG | clive.__private.core.beekeeper.handle:__run_beekeeper:207 - Got webserver http endpoint: `http://127.0.0.1:44521`
2023-08-31 14:07:43.791 | ℹ️ INFO | clive.__private.core.beekeeper.handle:__start:186 - Beekeeper started on http://127.0.0.1:44521.
2023-08-31 14:07:43.797 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=CreateSession(token='4af7c8b37f24b1c99ba962773f5ba9e8843cfa7cba4ba2c896942c0528a4a4b3')
2023-08-31 14:07:43.801 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=ListWallets(wallets=[])
2023-08-31 14:07:43.803 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=EmptyResponse()
2023-08-31 14:07:43.806 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=ListWallets(wallets=[WalletDetails(name='wallet', unlocked=False)])
2023-08-31 14:07:43.809 | ℹ️ INFO | clive.__private.core.beekeeper.handle:close:160 - Closing Beekeeper...
2023-08-31 14:07:43.811 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=EmptyResponse()
2023-08-31 14:07:43.813 | 🐞 DEBUG | clive.__private.core.beekeeper.executable:close:105 - Beekeeper closed with return code of `-2`.
2023-08-31 14:07:53.833 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:close:76 - Notifications server closed
2023-08-31 14:07:53.862 | 🐞 DEBUG | asyncio.selector_events:__init__:54 - Using selector: EpollSelector
2023-08-31 14:07:53.913 | 🐞 DEBUG | asyncio.selector_events:__init__:54 - Using selector: EpollSelector
and beekeeper logs
458785ms json_rpc_plugin.cpp:222 initialize ] initializing JSON RPC plugin
458786ms webserver_plugin.cpp:584 plugin_initialize ] initializing webserver plugin
458786ms webserver_plugin.cpp:587 plugin_initialize ] configured with 1 thread pool size
458786ms webserver_plugin.cpp:590 plugin_initialize ] Compression in webserver is disabled
458786ms webserver_plugin.cpp:602 plugin_initialize ] configured http to listen on 0.0.0.0:0
458786ms beekeeper_app_init.cpp:123 initialize_program_o ] initializing options
458787ms notifications.cpp:64 setup ] setting up notification handler for 1 address
458791ms beekeeper_app_init.cpp:166 initialize_program_o ] Backtrace on segfault is enabled.
458792ms webserver_plugin.cpp:290 operator() ] start processing http thread
458792ms webserver_plugin.cpp:305 operator() ] start listening for http requests on 0.0.0.0:44521
463795ms json_rpc_plugin.cpp:439 rpc ] message: {"id":0,"jsonrpc":"2.0","method":"beekeeper_api.create_session","params":{"notifications_endpoint":"127.0.0.1:34353","salt":"139898871443712"}}
463800ms json_rpc_plugin.cpp:439 rpc ] message: {"id":0,"jsonrpc":"2.0","method":"beekeeper_api.list_wallets","params":{"token":"4af7c8b37f24b1c99ba962773f5ba9e8843cfa7cba4ba2c896942c0528a4a4b3"}}
463803ms json_rpc_plugin.cpp:439 rpc ] message: {"id":0,"jsonrpc":"2.0","method":"beekeeper_api.open","params":{"wallet_name":"wallet","token":"4af7c8b37f24b1c99ba962773f5ba9e8843cfa7cba4ba2c896942c0528a4a4b3"}}
463805ms json_rpc_plugin.cpp:439 rpc ] message: {"id":0,"jsonrpc":"2.0","method":"beekeeper_api.list_wallets","params":{"token":"4af7c8b37f24b1c99ba962773f5ba9e8843cfa7cba4ba2c896942c0528a4a4b3"}}
463810ms json_rpc_plugin.cpp:439 rpc ] message: {"id":0,"jsonrpc":"2.0","method":"beekeeper_api.close_session","params":{"token":"4af7c8b37f24b1c99ba962773f5ba9e8843cfa7cba4ba2c896942c0528a4a4b3"}}
463811ms application.cpp:99 handle_signal ] _last_signal_code: 2
463811ms application.cpp:90 generate_interrupt_r ] interrupt requested!
463811ms webserver_plugin.cpp:651 plugin_pre_shutdown ] Shutting down webserver_plugin...
463811ms webserver_plugin.cpp:310 operator() ] http io service exit
463812ms application.cpp:475 finish ] Waiting for logging_thread quit
463812ms application.cpp:477 finish ] logging_thread quit done
Here are artifacts from this failing test (job 670337): failing.tar.gz
And while it's green:
2023-08-31 14:05:30.712 | ❌ ERROR | asyncio.base_events:default_exception_handler:1744 - Task was destroyed but it is pending!
task: <Task pending name='Task-84' coro=<RequestHandler.start() done, defined at /builds/hive/clive/venv/lib/python3.10/site-packages/aiohttp/web_protocol.py:462> wait_for=<Future finished result=None>>
2023-08-31 14:05:30.862 | ℹ️ INFO | clive.__private.core.beekeeper.handle:__start:175 - Starting Beekeeper...
2023-08-31 14:05:30.864 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:listen:36 - Notifications server is listening on 41905...
2023-08-31 14:05:30.884 | ℹ️ INFO | clive.__private.core.beekeeper.notifications:notify:40 - Got notification: {'value': {'current_status': 'starting without a session'}, 'time': '2023-08-31T14:05:30', 'name': 'hived_status'}
2023-08-31 14:05:30.886 | ℹ️ INFO | aiohttp.web_log:log:206 - 127.0.0.1 [31/Aug/2023:14:05:30 +0000] "PUT / HTTP/1.1" 204 0 "-" "-"
2023-08-31 14:05:30.887 | ℹ️ INFO | clive.__private.core.beekeeper.notifications:notify:40 - Got notification: {'value': {'current_status': 'signals attached'}, 'time': '2023-08-31T14:05:30', 'name': 'hived_status'}
2023-08-31 14:05:30.887 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:notify:50 - Beekeeper reports to be ready
2023-08-31 14:05:30.888 | ℹ️ INFO | aiohttp.web_log:log:206 - 127.0.0.1 [31/Aug/2023:14:05:30 +0000] "PUT / HTTP/1.1" 204 0 "-" "-"
2023-08-31 14:05:30.888 | ℹ️ INFO | clive.__private.core.beekeeper.notifications:notify:40 - Got notification: {'value': {'address': '0.0.0.0', 'port': 34139, 'type': 'HTTP'}, 'time': '2023-08-31T14:05:30', 'name': 'webserver listening'}
2023-08-31 14:05:30.889 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:notify:47 - Got notification with http address on: http://127.0.0.1:34139
2023-08-31 14:05:30.889 | ℹ️ INFO | aiohttp.web_log:log:206 - 127.0.0.1 [31/Aug/2023:14:05:30 +0000] "PUT / HTTP/1.1" 204 0 "-" "-"
2023-08-31 14:05:35.878 | 🐞 DEBUG | clive.__private.core.beekeeper.handle:__run_beekeeper:207 - Got webserver http endpoint: `http://127.0.0.1:34139`
2023-08-31 14:05:35.879 | ℹ️ INFO | clive.__private.core.beekeeper.handle:__start:186 - Beekeeper started on http://127.0.0.1:34139.
2023-08-31 14:05:35.885 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=CreateSession(token='b90094a2d0aa2621f1bbe68535a0c7388b62d163641a6e6a3920b06cfe9bc372')
2023-08-31 14:05:35.888 | ℹ️ INFO | clive.__private.core.commands.abc.command:_log_execution_info:40 - Executing command: CreateWallet
2023-08-31 14:05:35.890 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=Create(password='password')
2023-08-31 14:05:35.891 | ℹ️ INFO | clive.__private.core.app_state:activate:33 - Mode switched to ACTIVE.
2023-08-31 14:05:35.892 | ℹ️ INFO | clive.__private.core.beekeeper.handle:close:160 - Closing Beekeeper...
2023-08-31 14:05:35.894 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=EmptyResponse()
2023-08-31 14:05:36.108 | 🐞 DEBUG | clive.__private.core.beekeeper.executable:close:105 - Beekeeper closed with return code of `0`.
2023-08-31 14:05:36.110 | 🐞 DEBUG | clive.__private.core.beekeeper.executable:__wait_for_pid_file_to_be_deleted:140 - Beekeeper PID file was deleted in 0.00 seconds.
2023-08-31 14:05:36.111 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:close:76 - Notifications server closed
2023-08-31 14:05:36.112 | ℹ️ INFO | clive.__private.core.beekeeper.handle:close:166 - Beekeeper closed.
2023-08-31 14:05:36.112 | ℹ️ INFO | clive.__private.core.beekeeper.handle:__start:175 - Starting Beekeeper...
2023-08-31 14:05:36.114 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:listen:36 - Notifications server is listening on 42591...
2023-08-31 14:05:36.137 | ℹ️ INFO | clive.__private.core.beekeeper.notifications:notify:40 - Got notification: {'value': {'current_status': 'starting without a session'}, 'time': '2023-08-31T14:05:36', 'name': 'hived_status'}
2023-08-31 14:05:36.138 | ℹ️ INFO | aiohttp.web_log:log:206 - 127.0.0.1 [31/Aug/2023:14:05:36 +0000] "PUT / HTTP/1.1" 204 0 "-" "-"
2023-08-31 14:05:36.140 | ℹ️ INFO | clive.__private.core.beekeeper.notifications:notify:40 - Got notification: {'value': {'current_status': 'signals attached'}, 'time': '2023-08-31T14:05:36', 'name': 'hived_status'}
2023-08-31 14:05:36.140 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:notify:50 - Beekeeper reports to be ready
2023-08-31 14:05:36.141 | ℹ️ INFO | aiohttp.web_log:log:206 - 127.0.0.1 [31/Aug/2023:14:05:36 +0000] "PUT / HTTP/1.1" 204 0 "-" "-"
2023-08-31 14:05:36.142 | ℹ️ INFO | clive.__private.core.beekeeper.notifications:notify:40 - Got notification: {'value': {'address': '0.0.0.0', 'port': 33197, 'type': 'HTTP'}, 'time': '2023-08-31T14:05:36', 'name': 'webserver listening'}
2023-08-31 14:05:36.143 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:notify:47 - Got notification with http address on: http://127.0.0.1:33197
2023-08-31 14:05:36.143 | ℹ️ INFO | aiohttp.web_log:log:206 - 127.0.0.1 [31/Aug/2023:14:05:36 +0000] "PUT / HTTP/1.1" 204 0 "-" "-"
2023-08-31 14:05:41.129 | 🐞 DEBUG | clive.__private.core.beekeeper.handle:__run_beekeeper:207 - Got webserver http endpoint: `http://127.0.0.1:33197`
2023-08-31 14:05:41.130 | ℹ️ INFO | clive.__private.core.beekeeper.handle:__start:186 - Beekeeper started on http://127.0.0.1:33197.
2023-08-31 14:05:41.136 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=CreateSession(token='712241ececec2f17484a73ab7d03c4e1eaad25f4242e1c9f8b5ef40ac1d33e4c')
2023-08-31 14:05:41.140 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=ListWallets(wallets=[])
2023-08-31 14:05:41.143 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=EmptyResponse()
2023-08-31 14:05:41.145 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=ListWallets(wallets=[WalletDetails(name='wallet', unlocked=False)])
2023-08-31 14:05:41.148 | ℹ️ INFO | clive.__private.core.beekeeper.handle:close:160 - Closing Beekeeper...
2023-08-31 14:05:41.150 | ℹ️ INFO | clive.__private.core.beekeeper.handle:_send:147 - Returning model: id_=0 jsonrpc='2.0' result=EmptyResponse()
2023-08-31 14:05:41.152 | 🐞 DEBUG | clive.__private.core.beekeeper.executable:close:105 - Beekeeper closed with return code of `-2`.
2023-08-31 14:05:41.153 | 🐞 DEBUG | clive.__private.core.beekeeper.executable:__wait_for_pid_file_to_be_deleted:140 - Beekeeper PID file was deleted in 0.00 seconds.
2023-08-31 14:05:41.153 | 🐞 DEBUG | clive.__private.core.beekeeper.notifications:close:76 - Notifications server closed
2023-08-31 14:05:41.153 | ℹ️ INFO | clive.__private.core.beekeeper.handle:close:166 - Beekeeper closed.
2023-08-31 14:05:41.175 | 🐞 DEBUG | asyncio.selector_events:__init__:54 - Using selector: EpollSelector
2023-08-31 14:05:41.177 | 🐞 DEBUG | asyncio.selector_events:__init__:54 - Using selector: EpollSelector
and beekeeper:
336125ms json_rpc_plugin.cpp:222 initialize ] initializing JSON RPC plugin
336125ms webserver_plugin.cpp:584 plugin_initialize ] initializing webserver plugin
336125ms webserver_plugin.cpp:587 plugin_initialize ] configured with 1 thread pool size
336125ms webserver_plugin.cpp:590 plugin_initialize ] Compression in webserver is disabled
336126ms webserver_plugin.cpp:602 plugin_initialize ] configured http to listen on 0.0.0.0:0
336126ms beekeeper_app_init.cpp:123 initialize_program_o ] initializing options
336127ms notifications.cpp:64 setup ] setting up notification handler for 1 address
336134ms beekeeper_app_init.cpp:166 initialize_program_o ] Backtrace on segfault is enabled.
336135ms webserver_plugin.cpp:290 operator() ] start processing http thread
336135ms webserver_plugin.cpp:305 operator() ] start listening for http requests on 0.0.0.0:33197
341134ms json_rpc_plugin.cpp:439 rpc ] message: {"id":0,"jsonrpc":"2.0","method":"beekeeper_api.create_session","params":{"notifications_endpoint":"127.0.0.1:42591","salt":"140566037592720"}}
341139ms json_rpc_plugin.cpp:439 rpc ] message: {"id":0,"jsonrpc":"2.0","method":"beekeeper_api.list_wallets","params":{"token":"712241ececec2f17484a73ab7d03c4e1eaad25f4242e1c9f8b5ef40ac1d33e4c"}}
341142ms json_rpc_plugin.cpp:439 rpc ] message: {"id":0,"jsonrpc":"2.0","method":"beekeeper_api.open","params":{"wallet_name":"wallet","token":"712241ececec2f17484a73ab7d03c4e1eaad25f4242e1c9f8b5ef40ac1d33e4c"}}
341144ms json_rpc_plugin.cpp:439 rpc ] message: {"id":0,"jsonrpc":"2.0","method":"beekeeper_api.list_wallets","params":{"token":"712241ececec2f17484a73ab7d03c4e1eaad25f4242e1c9f8b5ef40ac1d33e4c"}}
341150ms json_rpc_plugin.cpp:439 rpc ] message: {"id":0,"jsonrpc":"2.0","method":"beekeeper_api.close_session","params":{"token":"712241ececec2f17484a73ab7d03c4e1eaad25f4242e1c9f8b5ef40ac1d33e4c"}}
341150ms application.cpp:99 handle_signal ] _last_signal_code: 2
341150ms application.cpp:90 generate_interrupt_r ] interrupt requested!
341150ms webserver_plugin.cpp:651 plugin_pre_shutdown ] Shutting down webserver_plugin...
341150ms webserver_plugin.cpp:310 operator() ] http io service exit
341151ms application.cpp:475 finish ] Waiting for logging_thread quit
341151ms application.cpp:477 finish ] logging_thread quit done
Here are artifacts from this test (job 670336): success.tar.gz
Here is a job when such a situation occurs: https://gitlab.syncad.com/hive/clive/-/jobs/670337
In the CI logs, we can observe that a test failed because of
Beekeeper PID file /builds/hive/clive/tests/unit/beekeeper/generated_during_test_wallet/test_wallet_open/beekeeper/beekeeper.pid was not deleted in 10.0 seconds.
and this is happening when beekeeper closes with -2
(but not always), sometimes this file is deleted though:
2023-08-31 14:05:36.108 | :beetle: DEBUG | clive.\__private.core.beekeeper.executable:close:105 - Beekeeper closed with return code of `0`.
2023-08-31 14:05:36.110 | :beetle: DEBUG | clive.\__private.core.beekeeper.executable:\__wait_for_pid_file_to_be_deleted:140 - Beekeeper PID file was deleted in 0.00 seconds.
2023-08-31 14:05:36.111 | :beetle: DEBUG | clive.\__private.core.beekeeper.notifications:close:76 - Notifications server closed
vs
2023-08-31 14:05:41.152 | :beetle: DEBUG | clive.\__private.core.beekeeper.executable:close:105 - Beekeeper closed with return code of `-2`.
2023-08-31 14:05:41.153 | :beetle: DEBUG | clive.\__private.core.beekeeper.executable:\__wait_for_pid_file_to_be_deleted:140 - Beekeeper PID file was deleted in 0.00 seconds.
2023-08-31 14:05:41.153 | :beetle: DEBUG | clive.\__private.core.beekeeper.notifications:close:76 - Notifications server closed
vs
2023-08-31 14:07:43.813 | :beetle: DEBUG | clive.\__private.core.beekeeper.executable:close:105 - Beekeeper closed with return code of `-2`.
2023-08-31 14:07:53.833 | :beetle: DEBUG | clive.\__private.core.beekeeper.notifications:close:76 - Notifications server closed
and some successful jobs from the same pipeline: https://gitlab.syncad.com/hive/clive/-/jobs/670336 https://gitlab.syncad.com/hive/clive/-/jobs/670335 https://gitlab.syncad.com/hive/clive/-/jobs/670330
This is randomly happening on other tests also, not only the one mentioned above: https://gitlab.syncad.com/hive/clive/-/jobs/670306