Improve cache-manager with immutable local cache and require util-linux

Dan Notesteinrequested to merge
feature/cache-manager-improvements into develop

Summary

  • Refactor cache-manager to use immutable tar-based local cache
  • Add documentation for cache-manager and Docker images
  • Remove BusyBox flock fallback, require util-linux

Changes

Immutable Local Cache

  • Store local cache as tar files instead of directories/symlinks
  • Local-first PUT: create local tar, then push to NFS
  • Always extract fresh on GET (prevents cache corruption from job modifications)

Unified Tar Format

  • Both NFS host and clients use tar archives
  • Remove legacy directory format support

BusyBox Removal

  • BusyBox flock returns "Bad file descriptor" on NFS mounts
  • Fail fast with clear error message instead of unreliable fallback
  • docker-builder and docker-dind already have util-linux installed

Documentation

  • Add cache-manager.md explaining architecture, locking, failure handling
  • Add common-ci-images.md documenting all Docker images and their purposes
  • Document potentially redundant images (python, python_runtime, python_development)

Merge request reports