Skip to content
GitLab
  • Menu
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • H hivemind
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 61
    • Issues 61
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 18
    • Merge requests 18
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • hive
  • hivemind
  • Issues
  • #2
Closed
Open
Created Apr 06, 2020 by Jolly Pirate@drakosDeveloper

Fork too deep

On a few occasions hivemind froze after hitting a fork too deep error. When frozen, hivemind doen't process any requests.

hive-hivemind | Traceback (most recent call last):
hive-hivemind |   File "/usr/local/bin/hive", line 8, in <module>
hive-hivemind |     sys.exit(run())
hive-hivemind |   File "/usr/local/lib/python3.6/dist-packages/hive/cli.py", line 23, in run
hive-hivemind |     launch_mode(mode, conf)
hive-hivemind |   File "/usr/local/lib/python3.6/dist-packages/hive/cli.py", line 34, in launch_mode
hive-hivemind |     Sync(conf=conf).run()
hive-hivemind |   File "/usr/local/lib/python3.6/dist-packages/hive/indexer/sync.py", line 64, in run
hive-hivemind |     Blocks.verify_head(self._steem)
hive-hivemind |   File "/usr/local/lib/python3.6/dist-packages/hive/indexer/blocks.py", line 129, in verify_head
hive-hivemind |     assert hive_head - cursor < 25, "fork too deep"
hive-hivemind | AssertionError: fork too deep

A workaround is to log into the postgres database and delete ~10000 lines before the last processed block, for example:

ALTER TABLE hive_blocks DISABLE TRIGGER ALL;
DELETE FROM hive_blocks WHERE num > 42200000;
ALTER TABLE hive_blocks ENABLE TRIGGER ALL;

Then restart hivemind to let it synch.

The DISABLE/ENABLE TRIGGER is needed otherwise deleting multiple lines is super slow! (https://dba.stackexchange.com/a/37095/180936)

Suggested solution: Implement this temporary workaround in the code. Or find another way to handle that error.

Edited Apr 06, 2020 by Jolly Pirate
Assignee
Assign to
Time tracking