Skip to content

random crash: not yet copied to irreversible block is being trying to be marked as irreversible

it happened once on CI: https://gitlab.syncad.com/hive/haf/-/jobs/1100484/raw

psql:/builds/hive/haf/scripts/maintenance-scripts/../../tests/integration/replay/application.sql:88: NOTICE:  App is processing blocks (1015925,1015925)
psql:/builds/hive/haf/scripts/maintenance-scripts/../../tests/integration/replay/application.sql:88: NOTICE:  Max fork id 52
psql:/builds/hive/haf/scripts/maintenance-scripts/../../tests/integration/replay/application.sql:88: NOTICE:  App fork id 52
psql:/builds/hive/haf/scripts/maintenance-scripts/../../tests/integration/replay/application.sql:88: NOTICE:  App current_block_num 1015925
psql:/builds/hive/haf/scripts/maintenance-scripts/../../tests/integration/replay/application.sql:88: NOTICE:  App irreversible_block_num 1015919
psql:/builds/hive/haf/scripts/maintenance-scripts/../../tests/integration/replay/application.sql:88: NOTICE:  Live processing block 1015925
868568ms block_num_rendezvous_trigger.cpp:40 report_complete_thre ] Dump whole block 1016002
psql:/builds/hive/haf/scripts/maintenance-scripts/../../tests/integration/replay/application.sql:88: NOTICE:  HAF instance is ready. Exiting wait loop.
868575ms block_num_rendezvous_trigger.cpp:40 report_complete_thre ] Dump whole block 1016003
868578ms block_num_rendezvous_trigger.cpp:40 report_complete_thre ] Dump whole block 1016004
868584ms livesync_data_dumper.cpp:231  run                  ] Write-ahead log processor detected SQL error: ERROR:  insert or update on table "irreversible_data" violates foreign key constraint "fk_1_hive_irreversible_data"
DETAIL:  Key (consistent_block)=(1015921) is not present in table "blocks".
CONTEXT:  SQL statement "UPDATE hive.irreversible_data SET consistent_block = _block_num"
PL/pgSQL function hive.set_irreversible(integer) line 29 at SQL statement

job.log