Add haf-app-tools: shared utilities for HAF applications

Dan Notesteinrequested to merge
feature/haf-app-tools into develop

Summary

Adds a new haf-app-tools/ directory containing shared utilities that HAF applications currently obtain via the haf submodule.

By centralizing these tools here, apps can fetch them at runtime instead of including the entire haf repository as a submodule, significantly simplifying the dependency chain:

Before: app → haf → hive → test-tools/tests_api After: app → tests_api (direct) + fetch utilities at runtime

Contents

Scripts

  • common.sh - Shared bash utilities (logging, git clone helpers)
  • create_haf_app_role.sh - PostgreSQL role creation for HAF apps
  • copy_datadir.sh - Data directory copying with NFS cache fallback

Python

  • process_openapi.py - Generates SQL types/functions and nginx rewrite rules from OpenAPI YAML in SQL comments
  • requirements.txt - Python dependencies

Config

  • config_5M.ini - Standard hived config for 5M block testing

Apps That Will Use This

App Uses
balance_tracker process_openapi.py, config_5M.ini
haf_block_explorer process_openapi.py, config_5M.ini
hafah process_openapi.py, common.sh, create_haf_app_role.sh, config_5M.ini
hivemind common.sh, create_haf_app_role.sh, copy_datadir.sh, config_5M.ini

Usage

Apps fetch tools at runtime:

variables:
  COMMON_CI_REF: "develop"
  COMMON_CI_URL: "https://gitlab.syncad.com/hive/common-ci-configuration/-/raw/${COMMON_CI_REF}"

before_script:
  - curl -fsSL "${COMMON_CI_URL}/haf-app-tools/scripts/common.sh" -o /tmp/common.sh
  - curl -fsSL "${COMMON_CI_URL}/haf-app-tools/python/process_openapi.py" -o /tmp/process_openapi.py

Test plan

  • Pilot with balance_tracker using this branch
  • Verify scripts fetch correctly
  • Verify OpenAPI processing works
  • Verify PostgreSQL role creation works
  • Full CI pipeline passes

Merge request reports