Fork too deep
On a few occasions hivemind froze after hitting a fork too deep error. When frozen, hivemind doen't process any requests.
hive-hivemind | Traceback (most recent call last):
hive-hivemind | File "/usr/local/bin/hive", line 8, in <module>
hive-hivemind | sys.exit(run())
hive-hivemind | File "/usr/local/lib/python3.6/dist-packages/hive/cli.py", line 23, in run
hive-hivemind | launch_mode(mode, conf)
hive-hivemind | File "/usr/local/lib/python3.6/dist-packages/hive/cli.py", line 34, in launch_mode
hive-hivemind | Sync(conf=conf).run()
hive-hivemind | File "/usr/local/lib/python3.6/dist-packages/hive/indexer/sync.py", line 64, in run
hive-hivemind | Blocks.verify_head(self._steem)
hive-hivemind | File "/usr/local/lib/python3.6/dist-packages/hive/indexer/blocks.py", line 129, in verify_head
hive-hivemind | assert hive_head - cursor < 25, "fork too deep"
hive-hivemind | AssertionError: fork too deep
A workaround is to log into the postgres database and delete ~10000 lines before the last processed block, for example:
ALTER TABLE hive_blocks DISABLE TRIGGER ALL;
DELETE FROM hive_blocks WHERE num > 42200000;
ALTER TABLE hive_blocks ENABLE TRIGGER ALL;
Then restart hivemind to let it synch.
The DISABLE/ENABLE TRIGGER is needed otherwise deleting multiple lines is super slow! (https://dba.stackexchange.com/a/37095/180936)
Suggested solution: Implement this temporary workaround in the code. Or find another way to handle that error.