issue #89: net_votes and total_votes are calculated during sync

changed title from issue #87 (closed): net_votes and total_votes is calculated during sync to issue #87 (closed): net_votes and total_votes are calculated during sync

added 1 commit

• cc9c0c7e - issue #87 (closed): net_votes and total_votes are calculated during sync

Compare with previous version

added 1 commit

• 07cd28c7 - issue #87 (closed): net_votes and total_votes are calculated during sync

Compare with previous version

added 2 commits

• 107c90a6 - issue #87 (closed): net_votes and total_votes are calculated during sync
• cb32570f - upgrade for hive_posts total_votes and net_votes

Compare with previous version

changed title from issue #87: net_votes and total_votes are calculated during sync to issue #89: net_votes and total_votes are calculated during sync

added 3 commits

• a0d112f0 - issue #91 (closed): incorrect results when initial sync started not from scratch
• 970c15d0 - issue #89 (closed): net_votes and total_votes are calculated during sync
• eb1c6c0f - upgrade for hive_posts total_votes and net_votes

Compare with previous version

marked as a Work In Progress

added 4 commits

• eb1c6c0f...8d06d8bc - 2 commits from branch develop
• bd5efd5c - issue #89 (closed): net_votes and total_votes are calculated during sync
• 329e4327 - upgrade for hive_posts total_votes and net_votes

Compare with previous version

unmarked as a Work In Progress

added 5 commits

• 329e4327...13868a13 - 3 commits from branch develop
• 431fdaea - issue #89 (closed): net_votes and total_votes are calculated during sync
• 16094486 - upgrade for hive_posts total_votes and net_votes

Compare with previous version

Upgraded production database fails syncing: sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "total_votes" violates not-null constraint:

WARNING - hive.db.adapter - [SQL-ERR] IntegrityError in query 
            SELECT is_new_post, id, author_id, permlink_id, post_category, parent_id, community_id, is_valid, is_muted, depth
            FROM process_hive_post_operation((:author)::varchar, (:permlink)::varchar, (:parent_author)::varchar, (:parent_permlink)::varchar, (:date)::timestamp, (:community_support_start_date)::timestamp, (:block_num)::integer);
             ({'author': 'machan', 'permlink': 'fight-for-your-dreams-for-what-you-want-to-be', 'parent_author': '', 'parent_permlink': 'hive-148441', 'date': '2020-11-13T11:30:57', 'community_support_start_date': '2019-10-22T07:12:36', 'block_num': 48649246})
ERROR - hive.indexer.blocks - exception encountered block 48649246
ERROR - hive.indexer.sync - Exception caught during processing blocks...
Traceback (most recent call last):
  File "/home/syncad/.local/lib/python3.8/site-packages/SQLAlchemy-1.4.0b1-py3.8-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1628, in _execute_context
    self.dialect.do_execute(
  File "/home/syncad/.local/lib/python3.8/site-packages/SQLAlchemy-1.4.0b1-py3.8-linux-x86_64.egg/sqlalchemy/engine/default.py", line 663, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.NotNullViolation: null value in column "total_votes" violates not-null constraint
DETAIL:  Failing row contains (100510589, 0, 0, 1327225, 100510589, 26192444, 1369030, 2020-11-13 11:30:57, 0, 0, f, f, t, 0.000, 0, 0.000, 0.000, 2020-11-20 11:30:57, 1970-01-01 00:00:00, 2020-11-13 11:30:57, f, f, f, f, f, 6688.61, 160527, 0.000 HBD, 0, 0, 0, 0, 0, 0, 0, 2020-11-13 11:30:57, 2020-11-20 11:30:57, 10000, 0.000 HBD, 1000000.000 HBD, t, t, [], 48649246, 48649246, null, null).
CONTEXT:  PL/pgSQL function process_hive_post_operation(character varying,character varying,character varying,character varying,timestamp without time zone,timestamp without time zone,integer) line 59 at RETURN QUERY


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/syncad/.local/lib/python3.8/site-packages/hivemind-0.0.1+16094486-py3.8.egg/hive/indexer/sync.py", line 130, in _block_consumer
    Blocks.process_multi(blocks, preparedVops, is_initial_sync)
  File "/home/syncad/.local/lib/python3.8/site-packages/hivemind-0.0.1+16094486-py3.8.egg/hive/indexer/blocks.py", line 110, in process_multi
    raise e
  File "/home/syncad/.local/lib/python3.8/site-packages/hivemind-0.0.1+16094486-py3.8.egg/hive/indexer/blocks.py", line 107, in process_multi
    last_num = cls._process(block, vops)
  File "/home/syncad/.local/lib/python3.8/site-packages/hivemind-0.0.1+16094486-py3.8.egg/hive/indexer/blocks.py", line 268, in _process
    Posts.comment_op(op, cls._head_block_date)
  File "/home/syncad/.local/lib/python3.8/site-packages/hivemind-0.0.1+16094486-py3.8.egg/hive/indexer/posts.py", line 98, in comment_op
    row = DB.query_row(sql, author=op['author'], permlink=op['permlink'], parent_author=op['parent_author'],
  File "/home/syncad/.local/lib/python3.8/site-packages/hivemind-0.0.1+16094486-py3.8.egg/hive/db/adapter.py", line 101, in query_row
    res = self._query(sql, **kwargs)
  File "/home/syncad/.local/lib/python3.8/site-packages/hivemind-0.0.1+16094486-py3.8.egg/hive/db/adapter.py", line 200, in _query
    raise e
  File "/home/syncad/.local/lib/python3.8/site-packages/hivemind-0.0.1+16094486-py3.8.egg/hive/db/adapter.py", line 192, in _query
    result = self._exec(query, **kwargs)
  File "/home/syncad/.local/lib/python3.8/site-packages/SQLAlchemy-1.4.0b1-py3.8-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1123, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/home/syncad/.local/lib/python3.8/site-packages/SQLAlchemy-1.4.0b1-py3.8-linux-x86_64.egg/sqlalchemy/sql/elements.py", line 310, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/home/syncad/.local/lib/python3.8/site-packages/SQLAlchemy-1.4.0b1-py3.8-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1312, in _execute_clauseelement
    ret = self._execute_context(
  File "/home/syncad/.local/lib/python3.8/site-packages/SQLAlchemy-1.4.0b1-py3.8-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1671, in _execute_context
    self._handle_dbapi_exception(
  File "/home/syncad/.local/lib/python3.8/site-packages/SQLAlchemy-1.4.0b1-py3.8-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1852, in _handle_dbapi_exception
    util.raise_(
  File "/home/syncad/.local/lib/python3.8/site-packages/SQLAlchemy-1.4.0b1-py3.8-linux-x86_64.egg/sqlalchemy/util/compat.py", line 180, in raise_
    raise exception
  File "/home/syncad/.local/lib/python3.8/site-packages/SQLAlchemy-1.4.0b1-py3.8-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1628, in _execute_context
    self.dialect.do_execute(
  File "/home/syncad/.local/lib/python3.8/site-packages/SQLAlchemy-1.4.0b1-py3.8-linux-x86_64.egg/sqlalchemy/engine/default.py", line 663, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "total_votes" violates not-null constraint
DETAIL:  Failing row contains (100510589, 0, 0, 1327225, 100510589, 26192444, 1369030, 2020-11-13 11:30:57, 0, 0, f, f, t, 0.000, 0, 0.000, 0.000, 2020-11-20 11:30:57, 1970-01-01 00:00:00, 2020-11-13 11:30:57, f, f, f, f, f, 6688.61, 160527, 0.000 HBD, 0, 0, 0, 0, 0, 0, 0, 2020-11-13 11:30:57, 2020-11-20 11:30:57, 10000, 0.000 HBD, 1000000.000 HBD, t, t, [], 48649246, 48649246, null, null).
CONTEXT:  PL/pgSQL function process_hive_post_operation(character varying,character varying,character varying,character varying,timestamp without time zone,timestamp without time zone,integer) line 59 at RETURN QUERY

[SQL: 
            SELECT is_new_post, id, author_id, permlink_id, post_category, parent_id, community_id, is_valid, is_muted, depth
            FROM process_hive_post_operation((%(author)s)::varchar, (%(permlink)s)::varchar, (%(parent_author)s)::varchar, (%(parent_permlink)s)::varchar, (%(date)s)::timestamp, (%(community_support_start_date)s)::timestamp, (%(block_num)s)::integer);
            ]
[parameters: {'author': 'machan', 'permlink': 'fight-for-your-dreams-for-what-you-want-to-be', 'parent_author': '', 'parent_permlink': 'hive-148441', 'date': '2020-11-13T11:30:57', 'community_support_start_date': '2019-10-22T07:12:36', 'block_num': 48649246}]
(Background on this error at: http://sqlalche.me/e/14/gkpj) 

added 1 commit

• b44b6b5e - Columns added via alter table should have also defined DEFAULT constrains to...

Compare with previous version

enabled an automatic merge when the pipeline for b44b6b5e succeeds

canceled the automatic merge

Merging since test passed, just post-deploy phase failed due to strange error related to missing script.

merged

mentioned in commit 4837cf9f