Draft: refactor follows and update_nofications indexing
- Apr 01, 2025
-
-
In previous commit vote notifications were changed to be inserted after processing each block. But in massive sync rshares for posts were calculated only at the end of massive sync. This resulted in vote notifications to not be inserted, because score were always calculated as 0. To fix this, calculate rshares for modified posts after processing each block. `hive_posts` are now modified in posts and votes indexer. An advisory lock was added to synchronise, because deadlock was detected by postgres failing transactions.
-
-
Dan Notestein authored
-
-
-
-
-
-
-
-
-
-
In massive sync we operate an 1000 blocks at a time. Post notifications are filled by posts indexer, mutes are filled by follow indexer. If we process a 1000 blocks batch and there's a mute, it will be processed by the follow indexer, but this won't be seen by the posts indexer in this batch. This means that the posts notifications will be generated even for accounts that are muted. This essentially means that if there is a mute operation in a 1000 blocks batch, notifications for posts from that batch will not get that mute into account. So it would be possible for users to get notifications from an accounts they have recently muted. The fix is to clear any notifications from muted account at the end of the massive sync. The issue should not affect live mode, because we process only one block at a time.
-
-
`account_reputations` might not have all the accounts, so left join is needed.
-
-
This uses reptracker_endpoints.get_account_reputation function, but inlines the call. The first reason is that the function doesn't specify reptracker_app schema directly. Since hivemind doesn't have it in its search_path, the call fail. The second reason is that the calculation is modified to bring the resulting range closer to the scores from hive_accounts_rank_view.
-
-
-
-
-
Also rename is_muted to is_post_muted to avoid confusion.
-
-
-
-
-
-
The issue was that resets were not effective and were not deleting rows. This is because following is set to NULL in the incoming data. So, when the query was joining against accounts table, the resets rows were skipped, because there was no matching account. To fix this the JOIN on the following column needs to be LEFT JOIN.
-
-
The functions was deleting too many records, because of missing check on following column.
-
-
-
-
-
-
-
Also, remove name to id map.
-
This speeds up processing of follows.
-
-
Seems to be unused.
-
-
-
Dan Notestein authored
-
-
-
-
-
-
-
Instead sort by account name.
-
-
-
-
-
-
-
-
-
-
-
Splitting data into separate lists means they are applied in different order, which cause tests to fail. To fix this just put all items in one big list so that they are applied in order.
-
-
-
-
-
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
refactor NewFollow class to optimize database inserts by using account IDs and handling missing records
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
-
Dan Notestein authored
-
