Integration site: a down bot-gate container takes every blog and wallet page down (502); fail open instead
Problem and intended result
Found by the tech lead's pre-promote review of #1622 (closed) (299bb593). stack/integration/caddy/Caddyfile.bot-gate sends every gated page document (GET/HEAD under /blog and /wallet, minus API, _next, _rsc and router requests) to reverse_proxy @page bot-gate:80 with lb_try_duration 5s. When the bot-gate varnish container is stopped, crashing or not yet up, that reverse_proxy fails. Caddy answers 502 after up to 5 s, so every page of both apps is down even though the apps are healthy. The gate is a crawl-load filter, not an access control, so an outage of the gate should not become an outage of the site.
Intended: when the gate cannot be reached (connection refused, DNS failure, timeout), the request goes on to its route as if the gate had answered 204, and caddy logs a warning. A reachable gate's answers (204, challenge, 403, 429) behave exactly as now.
Required behavior
- In
Caddyfile.bot-gate, treat a failed upstream (no answer frombot-gate:80) as a pass. For example, ahandle_errors/fallback that continues the route for errors from thatreverse_proxyonly, or a shortlb_try_durationplus a fail-open path. Keep the 5xx of the apps themselves untouched. - Log each fail-open (or rate-limit the log) so a dead gate is visible in caddy's log.
- Keep
DENSER_BOT_GATE=offworking as today.
Acceptance criteria
- Under the
bot-gateservice stopped whileDENSER_BOT_GATE=on, a blog page and a wallet page answer 200 with the real page within about 1 s, and caddy logs the fail-open. - Under the gate running, the challenge, 403, 429 and LAN exemption behave exactly as #1622 (closed) tests them.
Verification
- Extend
stack/integration/test-cache.shwith a case that stops the gate service and expects real pages. - Live: on the integration site,
docker stop denser-integration-bot-gate-1, curl a blog page (200), then start it again. Do this only in a quiet window, and restart the gate right after.
Integration points
stack/integration/caddy/Caddyfile.bot-gate— fail open whenbot-gate:80is unreachablestack/integration/test-cache.sh— the gate-down case
Depends-On: #1622 (closed)