p2p: stop punishing honest-but-stale blocks; fix undecodable disconnect messages

Fixes the amplification mechanism behind the 2026-07-09 network-wide P2P disruption (18:54-19:23 UTC: peer counts repeatedly collapsing to 0, participation dropping to ~61%, widespread missed blocks).

What happened

A routine micro-fork at ~block 107,991,17x cascaded because:

  1. The fork database's cache window (max_size = head - LIB + 1) is only ~2 blocks deep under one-block irreversibility. Honest blocks from slightly-behind peers (or witnesses briefly isolated on a fork stub) failed the "attempting to push a block that is too old" check.
  2. That check raised a generic assert, so the p2p layer took the punitive path for invalid blocks: disconnect the sender and every other peer advertising the same block id (node.cpp process_block_during_normal_operation). At a fork moment that's most of the peer list at once - the 20 -> 0 cliffs operators saw.
  3. Disconnected nodes stalled, fell further behind, reconnected, relayed now-stale heads, and were kicked again; witnesses isolated on the stub kept producing fresh triggers. The storm self-sustained for ~30 minutes until sync-mode reconciliation converged everyone onto the majority chain.

As a side effect, the goodbye (closing_connection_message) sent with each punitive disconnect embedded the entire rejected block inside its fc::exception; blocks with transactions serialize deeper than the unpack-side MAX_RECURSION_DEPTH (20), so no peer could decode these goodbyes (6,176 failed unpacks on one node), hiding the real disconnect reasons behind assert noise.

Changes

Commit 1 - fork_database.cpp: a block below the cache window now throws unlinkable_block_exception instead of a generic assert. The p2p layer already handles unlinkable blocks with the forgiving path (restart sync with the peer -> fork reconciliation), which is the intended behavior between honest nodes on different forks. This removes the amplifier: with this change the 2026-07-09 hiccup would have resolved in seconds like any normal micro-fork.

Commit 2 - chain_plugin.cpp + node.cpp: the generic block-rejection capture now records only block_num + block_id (still useful to the disconnected peer) instead of the full block, so the goodbye stays decodable. On the receive side, a closing_connection_message that fails to deserialize (e.g. from an unpatched peer) is logged and treated as a close request without details instead of tearing down the read loop with an assert - the connection is closing either way.

Notes for reviewers

  • In sync mode a below-window block now surfaces as unlinkable_block_exception, which sync's generic catch still treats punitively - same behavior as before this MR (no regression). If we want sync-mode leniency too, the natural shape is mirroring the existing block_older_than_undo_history -> inhibit_fetching_sync_blocks handling; left out to keep this change minimal.
  • We may also want a separate, much more generous "egregiously old" threshold that retains the punitive disconnect for actually-malicious peers; with this MR, genuinely invalid blocks (bad signature, failed validation) still take the punitive path.
  • Existing fork-db tests (block_tests.cpp) only ever expect unlinkable_block_exception; this change widens what throws it, so expectations hold. No tests reference the old generic assert.
  • No hardfork needed: p2p/node-local behavior only, no change to transaction or block validity.

Target is master since most node operators run 1.28.6; will cherry-pick to develop after review.

Post-mortem with full log evidence available separately.

Merge request reports

Loading