HAfAH issueshttps://gitlab.syncad.com/hive/HAfAH/-/issues2023-08-04T22:04:33Zhttps://gitlab.syncad.com/hive/HAfAH/-/issues/41Benchmarking starts before indexes build2023-08-04T22:04:33ZKrzysztof Mochockikmochocki@syncad.comBenchmarking starts before indexes buildhttps://gitlab.syncad.com/hive/HAfAH/-/jobs/640950
As we can see:
```
### PATHS ###
jmeter = /usr/bin/jmeter
root project directory = /builds/hive/HAfAH/haf/hive/tests/hive-local-tools/tests_api
datadir = /builds/hive/HAfAH/wdir
### NE...https://gitlab.syncad.com/hive/HAfAH/-/jobs/640950
As we can see:
```
### PATHS ###
jmeter = /usr/bin/jmeter
root project directory = /builds/hive/HAfAH/haf/hive/tests/hive-local-tools/tests_api
datadir = /builds/hive/HAfAH/wdir
### NETWORK ###
postgres url = postgresql:///haf_block_log
address = app
port = 6543
### TESTING ###
csv = perf_5M_heavy.csv
call style = CALL_STYLE_OPT.OLD_STYLE
threads = 10
api = account_history_api
2023-08-04 22:00:48,403 - performance - INFO - postgres connection string in JMETER: jdbc:postgresql:haf_block_log
2023-08-04 22:00:48,453 - performance - INFO - testing app: PythonHafah version: 70405b27f3884310210de33f8e86cf432260024f
2023-08-04 22:00:48,453 - performance - INFO - starting performance testing
[service:registry.gitlab.syncad.com__hive__haf__base_instance-registry.gitlab.syncad.com-hive-haf-base_instance-haf-instance] 2023-08-04T22:01:07.039403759Z 67039ms indexes_controler.cpp:110 operator() ] Query processor: `SELECT hive.restore_indexes( 'hive.operations' );' Creating of enable indexes done in 42970.01299999999901047 ms
[service:registry.gitlab.syncad.com__hive__haf__base_instance-registry.gitlab.syncad.com-hive-haf-base_instance-haf-instance] 2023-08-04T22:01:07.039431451Z 67039ms indexes_controler.cpp:114 operator() ] The enable indexes have been created...
[service:registry.gitlab.syncad.com__hive__haf__base_instance-registry.gitlab.syncad.com-hive-haf-base_instance-haf-instance] 2023-08-04T22:01:07.271504328Z 67271ms data_processor.cpp:111 operator() ] Leaving data processor thread: Query processor: `SELECT hive.restore_indexes( 'hive.operations' );'
```
Benchmarking started, but service reports, that restoring of indexes are still in progresshttps://gitlab.syncad.com/hive/HAfAH/-/issues/38Timeout test2023-02-22T11:01:07ZKrzysztof Mochockikmochocki@syncad.comTimeout test@akociubinski found problem while creating haf-based functional tests for account history.
It seems that:
- if `postgrest` is queried and after prepare networks:
- minimum half-secnd sleep is applied and any account is queried test...@akociubinski found problem while creating haf-based functional tests for account history.
It seems that:
- if `postgrest` is queried and after prepare networks:
- minimum half-secnd sleep is applied and any account is queried test always pass
- some operation is applied (tested on `create_operation`) test always pass
- existing account is queried test randomly pass
- non-existing account is queried test fail
- if `postgrest` is not queried, checks are done via api node (rocksdb), test always pass
- if `postgrest` is not queried, checks are done via direct postgres sql query, test always pass
test fail has following symptoms:
- core test pass (assertion pass and no exception is thrown during query)
- during test teardown, **Api Node** does not close (exceed 10s timeout and is closed via `SIGTERM`, which invoke warning which is treated as error)
infrstructure:
NetworkA:
- InitNode
- WitnessNode
NetworkB:
- WitnessNode1
- ApiNode with sql_serializer
(InitNode, WitnessNode0, WitnessNode1) -> ApiNode -> haf (postgres) -> HAfAH (postgrest)
Notes:
- it seems that adding some delay after `prepare_networks` with sleep or pushing some transaction eliminates problem with non-closing Api node
- in logs of Api Node it is possible to find line with interrupt signal log
@Trela @bwrona