Integration site: a down bot-gate container takes every blog and wallet page down (502); fail open instead

Problem and intended result

Found by the tech lead's pre-promote review of #1622 (closed) (299bb593). stack/integration/caddy/Caddyfile.bot-gate sends every gated page document (GET/HEAD under /blog and /wallet, minus API, _next, _rsc and router requests) to reverse_proxy @page bot-gate:80 with lb_try_duration 5s. When the bot-gate varnish container is stopped, crashing or not yet up, that reverse_proxy fails. Caddy answers 502 after up to 5 s, so every page of both apps is down even though the apps are healthy. The gate is a crawl-load filter, not an access control, so an outage of the gate should not become an outage of the site.

Intended: when the gate cannot be reached (connection refused, DNS failure, timeout), the request goes on to its route as if the gate had answered 204, and caddy logs a warning. A reachable gate's answers (204, challenge, 403, 429) behave exactly as now.

Required behavior

  • In Caddyfile.bot-gate, treat a failed upstream (no answer from bot-gate:80) as a pass. For example, a handle_errors/fallback that continues the route for errors from that reverse_proxy only, or a short lb_try_duration plus a fail-open path. Keep the 5xx of the apps themselves untouched.
  • Log each fail-open (or rate-limit the log) so a dead gate is visible in caddy's log.
  • Keep DENSER_BOT_GATE=off working as today.

Acceptance criteria

  • Under the bot-gate service stopped while DENSER_BOT_GATE=on, a blog page and a wallet page answer 200 with the real page within about 1 s, and caddy logs the fail-open.
  • Under the gate running, the challenge, 403, 429 and LAN exemption behave exactly as #1622 (closed) tests them.

Verification

  • Extend stack/integration/test-cache.sh with a case that stops the gate service and expects real pages.
  • Live: on the integration site, docker stop denser-integration-bot-gate-1, curl a blog page (200), then start it again. Do this only in a quiet window, and restart the gate right after.

Integration points

  • stack/integration/caddy/Caddyfile.bot-gate — fail open when bot-gate:80 is unreachable
  • stack/integration/test-cache.sh — the gate-down case

Depends-On: #1622 (closed)

Edited by AIDEV Production Bot