full sync 221dd70b - potential performance improvements
Related to evaluating full sync of 221dd70b Some of that already discussed with devs, writing here for easier reference.
-
(1) There are still long running tasks remaining
such as:
[INIT] update_all_posts_active executed in 10350.5263s
[INIT] Query SELECT update_account_reputations(0, 49956365, True); done in 5147.3995s
[INIT] Query SELECT update_posts_rshares(0, 49956365); done in 40442.9312s
-
(2) Reorganizing index creation time (taking into consideration order of other tasks) can speed up things -
(3) Is it really worth it to VACUUM ANALYZE hive_posts
multiple times between queries?
See:
207116 INFO - hive.db.db_state - [INIT] Query VACUUM ANALYZE hive_posts done in 423.3412s
208192 INFO - hive.db.db_state - [INIT] Query select update_hive_posts_root_id(0, 49956365) done in 1075.7588s
208712 INFO - hive.db.db_state - [INIT] Query VACUUM ANALYZE hive_posts done in 519.6164s
210296 INFO - hive.db.db_state - [INIT] Query select update_hive_posts_api_helper(0, 49956365) done in 1583.8307s
220646 INFO - hive.db.db_state - [INIT] update_all_posts_active executed in 10350.5263s
220947 INFO - hive.db.db_state - [INIT] Query VACUUM ANALYZE hive_posts done in 301.2349s
221407 INFO - hive.db.db_state - [INIT] Query SELECT update_feed_cache(0, 49956365); done in 459.1969s
221428 INFO - hive.db.db_state - [INIT] Query SELECT update_hive_posts_mentions(0, 49956365); done in 21.3289s
226576 INFO - hive.db.db_state - [INIT] Query SELECT update_account_reputations(0, 49956365, True); done in 5147.3995s
267020 INFO - hive.db.db_state - [INIT] Query SELECT update_posts_rshares(0, 49956365); done in 40442.9312s
267770 INFO - hive.db.db_state - [INIT] Query VACUUM ANALYZE hive_posts done in 749.6241s
-
(4) Tasks done during full sync's INIT vs. db_upgrade:
psql:upgrade/upgrade_runtime_migration.sql:72: NOTICE: Performing posts rshares, hot and trend recalculation on range ( 0, head_block_number)...
i.e. it's surprising that db_upgrade using same revision that it was full synced with takes that long