Restructure and cleanup of python tests
There are several topics we would like to discuss, please share your thoughts and suggestions.
-
Move everything from
functional/python_tests/*
directory just insidefunctional/
as there are no functional tests other than python ones, and with the current way longer imports are created.- Probably there should be a top-level python directory like
tests/python/
instead, where all things related to python tests could be stored. Such things (including/excluding some of the tests/ directories) won't be needed anymore and it will be clear distinction between python and c++ tests-related code. Now we havetests/api_tests
(python)tests/functional
(python) andtests/unit
(c++)
- Probably there should be a top-level python directory like
-
Remove the nested structure of api tests directory created inside functional tests. Probably all tests from
functional/python_tests/datagen_tests/api_tests
should be considered as functional and placed just insidefunctional/python_tests/datagen_tests
. Such structuring will lead to weird hierarchies likefunctional
andapi
distinctions already inside thefunctional
directory. -
Add README.md for all test packages/dirs (e.g.
datagen_tests
) which will explain:- What type of tests should be here? (to simplify decision, where to put newly craeted tests)
- how to run it (e.x run command with flags)
-
Add a dedicated directory for regression tests (tests written especially for reproducing existing bugs and proving that they were already fixed e.g.
test_crash.py
which obviously now couldn't crash a node) -
Solve the problem with two ways of defining pattern tests -> for cli_wallet python-based (api_tests/python_patterns_tests) and hived tavern-based (api_tests/pattern_tests) in pattern tests
- delete tavern from hived but still can be left in hivemind (+ deletion of
tests_api
from hived, + more flexibility while writing tests, - already untested self-made implementation which replaces tavern) - OR transform current python pattern test to the tavern ones
- delete tavern from hived but still can be left in hivemind (+ deletion of
-
Verify if tests placed in
functional/python_tests/comment_payments_tests/
likecomment_payment_test_00*.py
are working and if so, they should be moved to beem_tests, because now they are using beem to test some things. Looks like they are not running on CI now. -
Check what
simulate_curve_payouts.py
is, if it's a test it should be appropriately moved (potentially to remove, because rewards api is not supported) -
Make
comparison_tests
working:-
Add master node as a service to CI -
Add CI job for comparing testing node with master
-
-
Rewrite already not run on CI
tests/python_tests/functional/delayed_votes_tests/basic_delayed_votes_test_001.py
to be based on test-tools -
Create CI in beem repository and potentially move beem tests there
- We can autorun a new pipeline in the beem whenever something is pushed to hived develop, so we have information if something is broken
FYI: @bwrona, @mzebrak, @kmochocki, @kudmich