Speedup launching CLI commands by ensuring pycache is available or could be created
This MR provides an alternative approach to !732 (closed).
It speedups execution of Clive commands like clive--help and others by around 2,5x-3x, depending on the machine.
Problem
CLI commands (e.g. clive --help) were significantly slower in production (mainnet) env.
- Reduced execution time: ~1.5s → ~0.6s (Ryzen 5600x)
Benchmark Comparison
Before:
- Local dev image:
0.577s -
docker run -it registry.gitlab.syncad.com/hive/clive:testnet-4ee2517a --cli:0.577s(same as 1) -
docker run -it registry.gitlab.syncad.com/hive/clive:4ee2517a --cli:0.902s -
start_clive_cli.sh:1.567s
After:
- Local dev image:
0.577s -
docker run -it registry.gitlab.syncad.com/hive/clive:testnet-c3cecc71 --cli:0.577s(same as 1) -
docker run -it registry.gitlab.syncad.com/hive/clive:c3cecc71 --cli:0.605s- Still ~5–8% slower; likely caused by the minimal base image.
-
start_clive_cli.sh:0.605s(same as 3)
Root Cause
-
Difference between (2) and (3):
Mainnet base image is a minimal image and was missing__pycache__for standard Python modules when compared to testnet base image (e.g./usr/lib/python3.12/json/__pycache__). -
Difference between (3) and (4):
Missing__pycache__for the venv
(e.g./python_venv/lib/python3.12/site-packages/wax/__pycache__/).- Running with
docker run ...→ UID =2001(cache creation works). - Running via
start_clive_cli.sh→ UID = host UID (e.g.1000), no write permissions → cache not created.
- Running with
Fixes in this MR
-
Add prebuilt cache for Python standard modules in the generated
python_runtimeimage- Actual fix in: common-ci-configuration!92 (merged)
- Result: Clive image size increased 67 MiB → 71 MiB
-
Grant group write permissions for the venv cache
- Needed because UID changes from 2001 → host UID.
-
Additional: grant group write permissions for
/clive.
Edited by Mateusz Żebrak