Skip to content

full sync 221dd70b - potential performance improvements

Related to evaluating full sync of 221dd70b Some of that already discussed with devs, writing here for easier reference.

  • (1) There are still long running tasks remaining
    such as:
    [INIT] update_all_posts_active executed in 10350.5263s
    [INIT] Query SELECT update_account_reputations(0, 49956365, True); done in 5147.3995s
    [INIT] Query SELECT update_posts_rshares(0, 49956365); done in 40442.9312s

  • (2) Reorganizing index creation time (taking into consideration order of other tasks) can speed up things

  • (3) Is it really worth it to VACUUM ANALYZE hive_posts multiple times between queries?
    See:
    207116 INFO - hive.db.db_state - [INIT] Query VACUUM ANALYZE hive_posts done in 423.3412s
    208192 INFO - hive.db.db_state - [INIT] Query select update_hive_posts_root_id(0, 49956365) done in 1075.7588s
    208712 INFO - hive.db.db_state - [INIT] Query VACUUM ANALYZE hive_posts done in 519.6164s
    210296 INFO - hive.db.db_state - [INIT] Query select update_hive_posts_api_helper(0, 49956365) done in 1583.8307s
    220646 INFO - hive.db.db_state - [INIT] update_all_posts_active executed in 10350.5263s
    220947 INFO - hive.db.db_state - [INIT] Query VACUUM ANALYZE hive_posts done in 301.2349s
    221407 INFO - hive.db.db_state - [INIT] Query SELECT update_feed_cache(0, 49956365); done in 459.1969s
    221428 INFO - hive.db.db_state - [INIT] Query SELECT update_hive_posts_mentions(0, 49956365); done in 21.3289s
    226576 INFO - hive.db.db_state - [INIT] Query SELECT update_account_reputations(0, 49956365, True); done in 5147.3995s
    267020 INFO - hive.db.db_state - [INIT] Query SELECT update_posts_rshares(0, 49956365); done in 40442.9312s
    267770 INFO - hive.db.db_state - [INIT] Query VACUUM ANALYZE hive_posts done in 749.6241s

  • (4) Tasks done during full sync's INIT vs. db_upgrade:
    psql:upgrade/upgrade_runtime_migration.sql:72: NOTICE: Performing posts rshares, hot and trend recalculation on range ( 0, head_block_number)...
    i.e. it's surprising that db_upgrade using same revision that it was full synced with takes that long

Edited by Gandalf