Commit 893299b1 authored by Dan Notestein's avatar Dan Notestein
Browse files

Parallelize LOGGED/UNLOGGED table conversion for faster sync transitions

Convert tables to LOGGED/UNLOGGED in parallel using ThreadPoolExecutor,
with each table getting its own database connection. This significantly
reduces the time needed to transition between massive sync and live mode.

Table sizes (on full sync):
- hive_votes:         ~319 GB (bottleneck)
- hive_post_data:     ~127 GB
- hive_posts:         ~83 GB
- hive_permlink_data: ~27 GB
- hive_post_tags:     ~12 GB
- hive_accounts:      ~748 MB

Sequential conversion: ~45-90 minutes
Parallel conversion:   ~25-50 minutes (limited by largest table)

Tables are now ordered by size descending so the largest tables start
first, maximizing parallelism benefit.
parent a2e43f3a
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment