Speedup launching CLI commands by ensuring pycache is available or could be created

This MR provides an alternative approach to !732 (closed).
Only one of them should be merged, not both.

It speedups execution of Clive commands like clive--help and others by around 2,5x-3x, depending on the machine.

Problem

CLI commands (e.g. clive --help) were significantly slower in production (mainnet) env.

  • Reduced execution time: ~1.5s → ~0.6s (Ryzen 5600x)

Benchmark Comparison

Before:

  1. Local dev image: 0.577s
  2. docker run -it registry.gitlab.syncad.com/hive/clive:testnet-4ee2517a --cli: 0.577s (same as 1)
  3. docker run -it registry.gitlab.syncad.com/hive/clive:4ee2517a --cli: 0.902s
  4. start_clive_cli.sh: 1.567s

After:

  1. Local dev image: 0.577s
  2. docker run -it registry.gitlab.syncad.com/hive/clive:testnet-c3cecc71 --cli: 0.577s (same as 1)
  3. docker run -it registry.gitlab.syncad.com/hive/clive:c3cecc71 --cli: 0.605s
    • Still ~5–8% slower; likely caused by the minimal base image.
  4. start_clive_cli.sh: 0.605s (same as 3)

Root Cause

  • Difference between (2) and (3):
    Mainnet base image is a minimal image and was missing __pycache__ for standard Python modules when compared to testnet base image (e.g. /usr/lib/python3.12/json/__pycache__).

  • Difference between (3) and (4):
    Missing __pycache__ for the venv
    (e.g. /python_venv/lib/python3.12/site-packages/wax/__pycache__/).

    • Running with docker run ... → UID = 2001 (cache creation works).
    • Running via start_clive_cli.sh → UID = host UID (e.g. 1000), no write permissions → cache not created.

Fixes in this MR

  1. Add prebuilt cache for Python standard modules in the generated python_runtime image
  2. Grant group write permissions for the venv cache
    • Needed because UID changes from 2001 → host UID.
  3. Additional: grant group write permissions for /clive.
Edited by Mateusz Żebrak

Merge request reports

Loading