Implement a Witness Guard
A WitnessNode class could automatically verify witness node health by receiving and analysing a Block Stats. By configuring node config block_stats_report_output
parameter to "NOTIFY" block-stat entries generated by hived node will be sent to the notification server embedded inside test-tools. Then it could be possible to track down:
- block production stability and kill started node when it didn't produce block in some predefined time (let say 150s). To distinguish block-stats record,
gen
type shall be used to detect a block produced by given node andp2p
type to detect a block received from P2P network (produced by another node) - verify missed blocks when witness node because of some overload didn't produce block on time, but further production continues. In this case some warning could be produced
- verify overall node liveness by tracing all block-stat records (regardless to their type). Here we can also introduce some tracking and kill the node when it didn't receive any notification in 90s. Please take care for case when hived node is started in --stop-replay-at-block mode, when it doesn't send notifications
Edited by Bartek Wrona