Skip to content
Snippets Groups Projects
Unverified Commit 964cb6e2 authored by relativityboy's avatar relativityboy Committed by GitHub
Browse files

Merge pull request #148 from steemit/develop 

Durables Support, Basic Semver, Tests
parents 337ec0cf b6521c06
No related branches found
No related tags found
1 merge request!1steem/tinman:develop
Expand all Hide whitespace changes
Inline Side-by-side
Dockerfile
+ 1
− 1
View file @ 964cb6e2
FROM python:3.6
RUN apt-get update && apt-get -y install libyajl-dev
ENV PIPENV_VENV_IN_PROJECT=1
RUN pip install pipenv
WORKDIR /tinman
......@@ -6,4 +7,3 @@ ADD Pipfile Pipfile.lock /tinman/
RUN pipenv install
ADD . /tinman/
ENTRYPOINT ["pipenv", "run", "python", "-m", "tinman"]
README.md
+ 129
− 43
View file @ 964cb6e2
......@@ -16,6 +16,19 @@ This repository contains utilities to create a testnet.
# Installation
## Linux
```bash
$ sudo apt-get install virtualenv python3 libyajl-dev git
```
## macOS
```bash
$ brew install python3 yajl
$ pip3 install virtualenv
```
## Creating a virtualenv
In this step we create a virtualenv to isolate our project from the
......@@ -23,11 +36,16 @@ system-wide Python installation. The virtualenv is *activated*,
modifying the `PATH` and the prompt of the current shell,
by sourcing the `activate` script:
```bash
$ virtualenv -p $(which python3) ~/ve/tinman
$ source ~/ve/tinman/bin/activate
```
sudo apt-get install virtualenv python3 libyajl-dev
virtualenv -p $(which python3) ~/ve/tinman
source ~/ve/tinman/bin/activate
```
## Dependency Notes
Tinman should work right out of the box, but on some more delicately configured machines, some users report `ijson` errors. Running `pip install ijson` or `pip3 install ijson` should take care of that.
The `ijson` requirement also uses `yajl` for performance improvements. But `yajl` is optional and if it cannot be installed, there will be a warning that can be ignored.
## Using tinman
......@@ -36,32 +54,87 @@ assumes the source code lives in `~/src/tinman`:
**Note:**`tinman`'s default branch is develop. `master` is condsidered stablish.
```
mkdir -p ~/src
cd ~/src
git clone --branch master git@github.com:steemit/tinman
cd tinman
git submodule update --init --recursive
pip install pipenv
pipenv install
pip install .
```bash
$ mkdir -p ~/src
$ cd ~/src
$ git clone --branch master https://github.com/steemit/tinman.git
$ cd tinman
$ pip install pipenv
$ pipenv install
$ pip install .
```
If everything is set up correctly, you should be able to run commands
such as `tinman --help` as follows:
```
```bash
# Execute inside tinman virtualenv
tinman --help
$ tinman --help
```
Note, the `tinman` script in `~/ve/tinman/bin/tinman` may be symlinked
elsewhere (for example, `ln -s ~/ve/tinman/bin/tinman ~/bin/tinman`)
to allow `tinman` to run without the `virtualenv` being active.
# Example usage
# Example Usage
```bash
# First, take a snapshot of all accounts on mainnet using your own local mainnet
# node on port 8090.
$ tinman snapshot -s http://127.0.0.1:8090 -o snapshot.json
```
Once the `snapshot.json` file has been created, copy `txgen.conf.example` to
`txgen.conf`:
* `snapshot_file` - make sure this is the same name as your new `snaptshot.json`
```bash
# Next, create actions.
$ tinman txgen -c txgen.conf -o txgen.actions
```
Create a bash script, call it `bootstrap.sh`, make sure the `--get-dev-key` and `--signer` arguments point to the correct binaries:
```bash
# Port the actions over to your local bootstrap node on port 9990, with a secret
# set to "xyz-"
( \
echo '["set_secret", {"secret":"xyz-"}]' ; \
cat txgen.actions \
) | \
tinman keysub --get-dev-key /path/to/steem/programs/util/get_dev_key | \
tinman submit --realtime -t http://127.0.0.1:9990 \
--signer /path/to/steem/programs/util/sign_transaction \
-f fail.json \
-t 600
```
After allowing this script to run, you have now bootstrapped your testnet and you can point your witnesses at this node to start seeding and signing blocks.
To check the number of accounts created on the bootstrap node:
```bash
$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_count", "params":[], "id":1}' http://localhost:9990
```
# Detailed Usage
More detail about general usage:
This section contains a single large example.
* [Mainnet steemd](#mainnet-steemde)
* [Taking a snapshot](#taking-a-snapshot)
* [Generating actions](#generating-actions)
* [Keys substitution](#keys-substitution)
* [Deriving secret keys](#deriving-secret-keys)
* [Command-line key generator](#command-line-key-generator)
* [Running testnet fastgen node](#running-testnet-fastgen-node)
* [Pipelining transactions to testnet](#pipelining-transactions-to-testnet)
* [Durables](#durables)
* [Warden](#warden)
* [Gatling transactions from mainnet](#gatling-transactions-from-mainnet)
* [Running testnet witness node(s)](#running-testnet-witness-node-s-)
* [Tests](#tests)
## Mainnet steemd
......@@ -74,28 +147,28 @@ First, we set up a `steemd` for the main network. This `steemd` must be the fol
## Taking a snapshot
```
tinman snapshot -s http://127.0.0.1:8090 -o snapshot.json
```bash
$ tinman snapshot -s http://127.0.0.1:8090 -o snapshot.json
```
As of this writing, the above command takes approximately 5 minutes, writing an approximately 2 GB JSON file with 1,000,000 lines.
If you're running `tinman snapshot` interactively and you would like a visual progress indicator, you can install the `pv` program
(`apt-get install pv`) and use it to display the output line count in real time:
```
tinman snapshot -s http://127.0.0.1:8090 | pv -l > snapshot.json
```bash
$ tinman snapshot -s http://127.0.0.1:8090 | pv -l > snapshot.json
```
# Generating actions
## Generating actions
Now you can use `tinman txgen` to create a list of *actions*. Actions include
transactions which create and fund the accounts, and wait-for-block instructions
which control the rate at which transactions occur:
```
```bash
# As of this writing, this command takes ~10 minutes to start writing actions,
# consumes ~200MB of RAM, with all actions created in about two hours
tinman txgen -c txgen.conf -o tn.txlist
$ tinman txgen -c txgen.conf -o tn.txlist
```
Some notes about `tinman txgen`:
......@@ -106,7 +179,7 @@ Some notes about `tinman txgen`:
- Therefore, testnet balance is not equal to mainnet balance. Rather, it is proportional to mainnet balance.
- Accounts listed in `txgen.conf` are considered system accounts, any identically named account in the snapshot will not be ported
# Keys substitution
## Keys substitution
To maintain separation of concerns in `tinman` tools, the `tinman txgen` tool
does not directly generate transactions containing private keys (except
......@@ -125,19 +198,19 @@ can be submitted to the network. This is the role of the key substitution tool
`tinman keysub`. The `tinman keysub` tool takes as input a list of actions,
generates the specified keys, and substitutes them into each action.
## Deriving secret keys
### Deriving secret keys
By default, the private keys generated by `tinman keysub` have
known (i.e. insecure) seed strings. However, a secret may be
added to each seed string by prepending a `set_secret` action
to `tinman keysub`.
## Command-line key generator
### Command-line key generator
The `get_dev_key` program provided with `steemd` derives
keys using the same algorithm as `tinman keysub`.
# Running testnet fastgen node
## Running testnet fastgen node
Now that the transactions have been created, let's use them to initialize a testnet.
Since many blocks worth of transactions are created, `tinman submit` will
......@@ -163,9 +236,9 @@ On the testnet, some serializations are different from the main network, and
Therefore, `tinman submit` outsources signing of those transactions to the
`sign_transaction` binary included with `steemd`.
# Pipelining transactions to testnet
### Pipelining transactions to testnet
```
```bash
( \
echo '["set_secret", {"secret":"xyz-"}]' ; \
tinman txgen -c txgen.conf \
......@@ -174,13 +247,17 @@ tinman keysub | \
tinman submit -t http://127.0.0.1:9990 --signer steem/programs/util/sign_transaction -f fail.json
```
# Durables
# Other Modules
Once the testnet has been bootstrapped, other modules can be used to facilitate deeper orchestration.
## Durables
For consistency across testnet deployments, fixture-like object that must exist for external testing are recreated by the `durables` module.
Copy `durables.conf.example` to `durables.conf`, add any desired objects, and run (typically after initial bootstrap and before `gatling`):
```
```bash
( \
echo '["set_secret", {"secret":"xyz-"}]' ; \
tinman durables -c durables.conf \
......@@ -189,7 +266,7 @@ tinman keysub | \
tinman submit -t http://127.0.0.1:9990 --signer steem/programs/util/sign_transaction -f die
```
# Warden
## Warden
Use `warden` to check the current condition of a given chain. It does some
basic checks to make sure the chain is up and running, then returns error codes.
......@@ -197,8 +274,8 @@ basic checks to make sure the chain is up and running, then returns error codes.
Returning error code zero (0) means everything looks good. Non-zero means
something is amiss.
```
tinman warden -s http://127.0.0.1:8090 && echo LGTM || echo Bummer.
```bash
$ tinman warden -s http://127.0.0.1:8090 && echo LGTM || echo Bummer.
```
As an example, you can add `warden` to your deployment script to delay the next step until your seed node has synchronized with the initial bootstrap node.
......@@ -215,37 +292,37 @@ done
echo Ready to proceed.
```
# Gatling transactions from mainnet
## Gatling transactions from mainnet
Populating the test network with transactions from the main network.
To stream from genesis:
```bash
tinman gatling -f 1 -o -
$ tinman gatling -f 1 -o -
```
To stream from block 25066272 to 25066292:
```bash
tinman gatling -f 25066272 -t 25066292 -o -
$ tinman gatling -f 25066272 -t 25066292 -o -
```
To stream starting from block 25066272:
```bash
tinman gatling -f 25066272 -o -
$ tinman gatling -f 25066272 -o -
```
# Running testnet witness node(s)
## Running testnet witness node(s)
At the end of the transactions to be submitted, `tinman txgen` creates witnesses `init-0` through `init-20`
and votes for them with large amount of `TESTS`. The keys of these witnesses are generated by a deterministic
algorithm compatible with the `get_dev_key` utility program, so the keys of the witnesses may be obtained
as follows:
```
programs/util/get_dev_key xxx- block-init-0:21
```bash
$ programs/util/get_dev_key xxx- block-init-0:21
```
where `xxx` is the `"secret"` string in `txgen.conf`.
......@@ -256,7 +333,7 @@ specify the fastgen node using the `p2p-seed-node` option in the config file.
Therefore we may add the witness definitions and private keys to the witness config file:
```
```bash
i=0 ; while [ $i -lt 21 ] ; do echo witness = '"'init-$i'"' >> testnet_datadir/config.ini ; let i=i+1 ; done
steem/programs/util/get_dev_key xxx- block-init-0:21 | cut -d '"' -f 4 | sed 's/^/private-key = /' >> testnet_datadir/config.ini
```
......@@ -272,4 +349,13 @@ long as a sufficient number of other witness nodes are timely producing blocks,
is not necessary to use these flags once 128 blocks have been produced after the
transition.
## Tests
To test `tinman`:
```bash
$ cd test
$ pip install .. && python -m unittest *_test.py
```
<img src="https://i.imgur.com/h57pDVE.png" width="25%" height="25%" />
test/keysub_test.py 0 → 100644
+ 19
− 0
View file @ 964cb6e2
import unittest
import json
from tinman import keysub
class KeysubTest(unittest.TestCase):
def test_process_esc(self):
# Note, resolver needs to be mocked to properly test.
self.assertRaises(AttributeError, keysub.process_esc, 'Bpublickey:owner-initminerB', 'B')
def test_process_esc_ignored(self):
result = keysub.process_esc('foo:bar', 'baz')
expected_result = 'foo:bar'
self.assertEqual(result, expected_result)
def test_compute_keypair_from_seed(self):
# Note, resolver needs to be mocked to properly test.
self.assertRaises(FileNotFoundError, keysub.compute_keypair_from_seed, '1234', 'secret')
self.assertRaises(json.decoder.JSONDecodeError, keysub.compute_keypair_from_seed, '1234', 'secret', '/usr/bin/true')
test/test-future-snapshot.json 0 → 100644
+ 157
− 0
View file @ 964cb6e2
{
"metadata":{
"snapshot:semver":"9.0",
"snapshot:origin_api":"http://calculon.local"
},
"dynamic_global_properties":{
"total_vesting_fund_steem":{
"amount":"196793227573",
"nai":"@@000000021",
"precision":3
}
},
"accounts": [
{
"active":{
"account_auths":[
],
"key_auths":[
[
"STM7Wz3qohJpAbmmqBv9UUKBG14h9ueYkJspWot5yiX1JSiohwZZX",
1
]
],
"weight_threshold":1
},
"balance":{
"amount":"490235373",
"nai":"@@000000021",
"precision":3
},
"can_vote":true,
"comment_count":0,
"created":"2016-03-31T08:43:09",
"curation_rewards":0,
"delegated_vesting_shares":{
"amount":"0",
"nai":"@@000000037",
"precision":6
},
"id":483,
"is_smt":false,
"json_metadata":"",
"last_account_recovery":"1970-01-01T00:00:00",
"last_account_update":"1970-01-01T00:00:00",
"last_owner_update":"1970-01-01T00:00:00",
"last_post":"1970-01-01T00:00:00",
"last_root_post":"1970-01-01T00:00:00",
"last_vote_time":"2017-03-15T07:05:15",
"lifetime_vote_count":0,
"memo_key":"STM7Wz3qohJpAbmmqBv9UUKBG14h9ueYkJspWot5yiX1JSiohwZZX",
"mined":true,
"name":"alpha",
"next_vesting_withdrawal":"1969-12-31T23:59:59",
"owner":{
"account_auths":[
],
"key_auths":[
[
"STM7Wz3qohJpAbmmqBv9UUKBG14h9ueYkJspWot5yiX1JSiohwZZX",
1
]
],
"weight_threshold":1
},
"post_count":0,
"posting":{
"account_auths":[
],
"key_auths":[
[
"STM7Wz3qohJpAbmmqBv9UUKBG14h9ueYkJspWot5yiX1JSiohwZZX",
1
]
],
"weight_threshold":1
},
"posting_rewards":0,
"proxied_vsf_votes":[
0,
0,
0,
0
],
"proxy":"blocktrades",
"received_vesting_shares":{
"amount":"29975761028",
"nai":"@@000000037",
"precision":6
},
"recovery_account":"steem",
"reset_account":"null",
"reward_sbd_balance":{
"amount":"0",
"nai":"@@000000013",
"precision":3
},
"reward_steem_balance":{
"amount":"0",
"nai":"@@000000021",
"precision":3
},
"reward_vesting_balance":{
"amount":"0",
"nai":"@@000000037",
"precision":6
},
"reward_vesting_steem":{
"amount":"0",
"nai":"@@000000021",
"precision":3
},
"savings_balance":{
"amount":"0",
"nai":"@@000000021",
"precision":3
},
"savings_sbd_balance":{
"amount":"0",
"nai":"@@000000013",
"precision":3
},
"savings_sbd_last_interest_payment":"1970-01-01T00:00:00",
"savings_sbd_seconds":"0",
"savings_sbd_seconds_last_update":"1970-01-01T00:00:00",
"savings_withdraw_requests":0,
"sbd_balance":{
"amount":"4931480",
"nai":"@@000000013",
"precision":3
},
"sbd_last_interest_payment":"2018-09-06T00:00:51",
"sbd_seconds":"38421621615378",
"sbd_seconds_last_update":"2018-09-26T00:12:54",
"to_withdraw":"76724740938075",
"vesting_shares":{
"amount":"82084887877",
"nai":"@@000000037",
"precision":6
},
"vesting_withdraw_rate":{
"amount":"0",
"nai":"@@000000037",
"precision":6
},
"voting_manabar":{
"current_mana":9950,
"last_update_time":1489561515
},
"withdraw_routes":0,
"withdrawn":"76724740938075",
"witnesses_voted_for":0
}
]
}
test/test-snapshot.json 0 → 100644
+ 2898
− 0
View file @ 964cb6e2
This diff is collapsed.
test/txgen_test.py 0 → 100644
+ 237
− 0
View file @ 964cb6e2
import unittest
import shutil
from tinman import prockey
from tinman import txgen
FULL_CONF = {
"transactions_per_block" : 40,
"snapshot_file" : "/tmp/test-snapshot.json",
"min_vesting_per_account" : {"amount" : "1", "precision" : 3, "nai" : "@@000000021"},
"total_port_balance" : {"amount" : "200000000000", "precision" : 3, "nai" : "@@000000021"},
"accounts" : {
"initminer" : {
"name" : "initminer",
"vesting" : {"amount" : "1000000", "precision" : 3, "nai" : "@@000000021"}
}, "init" : {
"name" : "init-{index}",
"vesting" : {"amount" : "1000000", "precision" : 3, "nai" : "@@000000021"},
"count" : 21,
"creator" : "initminer"
}, "elector" : {
"name" : "elect-{index}",
"vesting" : {"amount" : "1000000000", "precision" : 3, "nai" : "@@000000021"},
"count" : 10,
"round_robin_votes_per_elector" : 2,
"random_votes_per_elector" : 3,
"randseed" : 1234,
"creator" : "initminer"
}, "porter" : {
"name" : "porter",
"creator" : "initminer",
"vesting" : {"amount" : "1000000", "precision" : 3, "nai" : "@@000000021"}
}, "manager" : {
"name" : "tnman",
"creator" : "initminer",
"vesting" : {"amount" : "1000000", "precision" : 3, "nai" : "@@000000021"}
},
"STEEM_MINER_ACCOUNT" : {"name" : "mners"},
"STEEM_NULL_ACCOUNT" : {"name" : "null"},
"STEEM_TEMP_ACCOUNT" : {"name" : "temp"}
}
}
class TxgenTest(unittest.TestCase):
def test_create_system_accounts_bad_args(self):
self.assertRaises(TypeError, txgen.create_system_accounts)
def test_create_witnesses(self):
keydb = prockey.ProceduralKeyDatabase()
conf = {"accounts": {
"init": {
"name" : "init-{index}",
"vesting" : {"amount" : "1000000", "precision" : 3, "nai" : "@@000000021"},
"count" : 21,
"creator" : "initminer"
}
}}
for witness in txgen.create_system_accounts(conf, keydb, "init"):
self.assertEqual(len(witness["operations"]), 2)
self.assertEqual(len(witness["wif_sigs"]), 1)
account_create_operation, transfer_to_vesting_operation = witness["operations"]
self.assertEqual(account_create_operation["type"], "account_create_operation")
value = account_create_operation["value"]
self.assertEqual(value["fee"], {"amount" : "0", "precision" : 3, "nai" : "@@000000021"})
self.assertEqual(value["creator"], "initminer")
self.assertEqual(transfer_to_vesting_operation["type"], "transfer_to_vesting_operation")
value = transfer_to_vesting_operation["value"]
self.assertEqual(value["from"], "initminer")
self.assertEqual(value["amount"], {"amount" : "1000000", "precision" : 3, "nai" : "@@000000021"})
def test_update_witnesses(self):
keydb = prockey.ProceduralKeyDatabase()
conf = {"accounts": {
"init": {
"name" : "init-{index}",
"vesting" : {"amount" : "1000000", "precision" : 3, "nai" : "@@000000021"},
"count" : 21,
"creator" : "initminer"
}
}}
for witness in txgen.update_witnesses(conf, keydb, "init"):
self.assertEqual(len(witness["operations"]), 1)
self.assertEqual(len(witness["wif_sigs"]), 1)
for op in witness["operations"]:
self.assertEqual(op["type"], "witness_update_operation")
value = op["value"]
self.assertEqual(value["url"], "https://steemit.com/")
self.assertEqual(value["props"], {})
self.assertEqual(value["fee"], {"amount" : "0", "precision" : 3, "nai" : "@@000000021"})
def test_vote_witnesses(self):
keydb = prockey.ProceduralKeyDatabase()
conf = {"accounts": {
"init": {
"name" : "init-{index}",
"vesting" : {"amount" : "1000000", "precision" : 3, "nai" : "@@000000021"},
"count" : 21,
"creator" : "initminer"
}, "elector" : {
"name" : "elect-{index}",
"vesting" : {"amount" : "1000000000", "precision" : 3, "nai" : "@@000000021"},
"count" : 10,
"round_robin_votes_per_elector" : 2,
"random_votes_per_elector" : 3,
"randseed" : 1234,
"creator" : "initminer"
}
}}
for witness in txgen.vote_accounts(conf, keydb, "elector", "init"):
self.assertGreater(len(witness["operations"]), 1)
self.assertEqual(len(witness["wif_sigs"]), 1)
for op in witness["operations"]:
self.assertEqual(op["type"], "account_witness_vote_operation")
value = op["value"]
self.assertTrue(value["approve"])
def test_get_account_stats(self):
shutil.copyfile("test-snapshot.json", "/tmp/test-snapshot.json")
conf = {
"snapshot_file" : "/tmp/test-snapshot.json",
"accounts": {}
}
account_stats = txgen.get_account_stats(conf)
expected_account_names = {"steemit", "binance-hot", "alpha",
"upbitsteemhot", "blocktrades", "steemit2", "ned", "holiday",
"imadev", "muchfun", "poloniex", "gopax-deposit", "dan",
"bithumb.sunshine", "ben", "dantheman", "openledger-dex", "bittrex",
"huobi-withdrawal", "korbit3"
}
self.assertEqual(account_stats["account_names"], expected_account_names)
self.assertEqual(account_stats["total_vests"], 103927115336403598)
self.assertEqual(account_stats["total_steem"], 60859712641)
def test_get_proportions(self):
shutil.copyfile("test-snapshot.json", "/tmp/test-snapshot.json")
conf = {
"snapshot_file" : "/tmp/test-snapshot.json",
"min_vesting_per_account": {"amount" : "1", "precision" : 3, "nai" : "@@000000021"},
"total_port_balance" : {"amount" : "200000000000", "precision" : 3, "nai" : "@@000000021"},
"accounts": {}
}
account_stats = txgen.get_account_stats(conf)
proportions = txgen.get_proportions(account_stats, conf)
self.assertEqual(proportions["min_vesting_per_account"], 1)
self.assertEqual(proportions["vest_conversion_factor"], 1469860)
self.assertEqual(proportions["steem_conversion_factor"], 776237988251)
def test_create_accounts(self):
shutil.copyfile("test-snapshot.json", "/tmp/test-snapshot.json")
conf = {
"snapshot_file" : "/tmp/test-snapshot.json",
"min_vesting_per_account": {"amount" : "1", "precision" : 3, "nai" : "@@000000021"},
"total_port_balance" : {"amount" : "200000000000", "precision" : 3, "nai" : "@@000000021"},
"accounts": {"porter": {"name": "porter"}
}
}
keydb = prockey.ProceduralKeyDatabase()
account_stats = txgen.get_account_stats(conf)
for account in txgen.create_accounts(account_stats, conf, keydb):
self.assertEqual(len(account["operations"]), 3)
self.assertEqual(len(account["wif_sigs"]), 1)
for op in account["operations"]:
value = op["value"]
if op["type"] == "account_create_operation":
self.assertEqual(value["fee"], {"amount" : "0", "precision" : 3, "nai" : "@@000000021"})
elif op["type"] == "transfer_to_vesting_operation":
self.assertEqual(value["from"], "porter")
self.assertGreater(int(value["amount"]["amount"]), 0)
elif op["type"] == "transfer_operation":
self.assertEqual(value["from"], "porter")
self.assertGreater(int(value["amount"]["amount"]), 0)
self.assertEqual(value["memo"], "Ported balance")
def test_update_accounts(self):
shutil.copyfile("test-snapshot.json", "/tmp/test-snapshot.json")
conf = {
"snapshot_file" : "/tmp/test-snapshot.json",
"min_vesting_per_account": {"amount" : "1", "precision" : 3, "nai" : "@@000000021"},
"total_port_balance" : {"amount" : "200000000000", "precision" : 3, "nai" : "@@000000021"},
"accounts": {"manager": {"name": "tnman"}
}
}
keydb = prockey.ProceduralKeyDatabase()
account_stats = txgen.get_account_stats(conf)
for account in txgen.update_accounts(account_stats, conf, keydb):
self.assertEqual(len(account["operations"]), 1)
self.assertEqual(len(account["wif_sigs"]), 1)
for op in account["operations"]:
value = op["value"]
self.assertIn(["tnman", 1], value["owner"]["account_auths"])
def test_build_actions(self):
shutil.copyfile("test-snapshot.json", "/tmp/test-snapshot.json")
for action in txgen.build_actions(FULL_CONF):
cmd, args = action
if cmd == "metadata":
self.assertEqual(args["txgen:semver"], "0.2")
self.assertEqual(args["txgen:transactions_per_block"], 40)
self.assertIsNotNone(args["epoch:created"])
self.assertEqual(args["actions:count"], 60)
self.assertGreater(args["recommend:miss_blocks"], 28968013)
self.assertEqual(args["snapshot:semver"], "0.2")
self.assertEqual(args["snapshot:origin_api"], "http://calculon.local")
elif cmd == "wait_blocks":
self.assertGreater(args["count"], 0)
elif cmd == "submit_transaction":
self.assertGreater(len(args["tx"]["operations"]), 0)
else:
self.fail("Unexpected action: %s" % cmd)
def test_build_actions_future_snapshot(self):
shutil.copyfile("test-future-snapshot.json", "/tmp/test-future-snapshot.json")
conf = FULL_CONF.copy()
conf["snapshot_file"] = "/tmp/test-future-snapshot.json"
with self.assertRaises(RuntimeError) as ctx:
for action in txgen.build_actions(conf):
cmd, args = action
self.assertIn('Unsupported snapshot', str(ctx.exception))
test/util_test.py 0 → 100644
+ 71
− 0
View file @ 964cb6e2
import unittest
from tinman import util
from simple_steem_client.client import SteemRemoteBackend, SteemInterface
class UtilTest(unittest.TestCase):
def test_tag_escape_sequences(self):
result = list(util.tag_escape_sequences('now "is" the time; "the hour" has "come"', '"'))
expected_result = [('now ', False), ('is', True), (' the time; ', False), ('the hour', True), (' has ', False), ('come', True), ('', False)]
self.assertEqual(result, expected_result)
def test_batch(self):
result = list(util.batch("spamspam", 3))
expected_result = [['s', 'p', 'a'], ['m', 's', 'p'], ['a', 'm']]
self.assertEqual(result, expected_result)
def test_find_non_substr(self):
self.assertEqual(util.find_non_substr('steem'), 'a')
self.assertEqual(util.find_non_substr('steemian'), 'b')
self.assertEqual(util.find_non_substr('steemian bob'), 'c')
self.assertEqual(util.find_non_substr('steemian bob can'), 'd')
# skip 'e' because 'steem' contains 'e'
self.assertEqual(util.find_non_substr('steemian bob can do'), 'f')
self.assertEqual(util.find_non_substr('steemian bob can do fun'), 'g')
self.assertEqual(util.find_non_substr('steemian bob can do fun things'), 'j')
def test_iterate_operations_from(self):
backend = SteemRemoteBackend(nodes=["https://api.steemit.com"], appbase=True)
steemd = SteemInterface(backend)
result = util.iterate_operations_from(steemd, True, 1102, 1103, set())
expected_op = {
'type': 'pow_operation',
'value': {
'worker_account': 'steemit11',
'block_id': '0000044df0f062c0504a8e37288a371ada63a1c7',
'nonce': 33097,
'work': {
'worker': 'STM65wH1LZ7BfSHcK69SShnqCAH5xdoSZpGkUjmzHJ5GCuxEK9V5G',
'input': '45a3824498b87e41129f6fef17be276af6ff87d1e859128f28aaa9c08208871d',
'signature': '1f93a52c4f794803b2563845b05b485e3e5f4c075ddac8ea8cffb988a1ffcdd1055590a3d5206a3be83cab1ea548fc52889d43bdbd7b74d62f87fb8e2166145a5d',
'work': '00003e554a58830e7e01669796f40d1ce85c7eb979e376cb49e83319c2688c7e',
}, 'props': {
'account_creation_fee': {"amount" : "100000", "precision" : 3, "nai" : "@@000000021"},
'maximum_block_size': 131072,
'sbd_interest_rate': 1000
}
}
}
# Scan all of the results and match against the expected op. This will
# fail if we get anything other than this exact op.
for op in result:
self.assertEqual(op['type'], expected_op['type'])
self.assertEqual(op['value']['worker_account'], expected_op['value']['worker_account'])
self.assertEqual(op['value']['block_id'], expected_op['value']['block_id'])
self.assertEqual(op['value']['nonce'], expected_op['value']['nonce'])
self.assertEqual(op['value']['work']['worker'], expected_op['value']['work']['worker'])
self.assertEqual(op['value']['work']['input'], expected_op['value']['work']['input'])
self.assertEqual(op['value']['work']['signature'], expected_op['value']['work']['signature'])
self.assertEqual(op['value']['work']['work'], expected_op['value']['work']['work'])
self.assertEqual(op['value']['props']['account_creation_fee']['amount'], expected_op['value']['props']['account_creation_fee']['amount'])
self.assertEqual(op['value']['props']['account_creation_fee']['precision'], expected_op['value']['props']['account_creation_fee']['precision'])
self.assertEqual(op['value']['props']['account_creation_fee']['nai'], expected_op['value']['props']['account_creation_fee']['nai'])
self.assertEqual(op['value']['props']['maximum_block_size'], expected_op['value']['props']['maximum_block_size'])
self.assertEqual(op['value']['props']['sbd_interest_rate'], expected_op['value']['props']['sbd_interest_rate'])
def test_action_to_str(self):
action = ["metadata", {}]
result = util.action_to_str(action)
self.assertEqual(result, '["metadata",{"esc":"b"}]')
tinman/submit.py
+ 15
− 1
View file @ 964cb6e2
......@@ -17,6 +17,9 @@ import traceback
from . import util
ACTIONS_MAJOR_VERSION_SUPPORTED = 0
ACTIONS_MINOR_VERSION_SUPPORTED = 2
class TransactionSigner(object):
def __init__(self, sign_transaction_exe=None, chain_id=None):
if(chain_id is None):
......@@ -172,7 +175,18 @@ def main(argv):
if cmd == "metadata":
metadata = args
transactions_per_block = metadata.get("txgen:transactions_per_block", transactions_per_block)
print("metadata:", metadata)
semver = metadata.get("txgen:semver", '0.0')
major_version, minor_version = semver.split('.')
major_version = int(major_version)
minor_version = int(minor_version)
if major_version == ACTIONS_MAJOR_VERSION_SUPPORTED:
print("metadata:", metadata)
else:
raise RuntimeError("Unsupported actions:", metadata)
if minor_version < ACTIONS_MINOR_VERSION_SUPPORTED:
print("WARNING: Older actions encountered.", file=sys.stderr)
elif cmd == "wait_blocks":
if metadata and args.get("count") == 1 and args.get("miss_blocks"):
if args["miss_blocks"] < metadata["recommend:miss_blocks"]:
......
tinman/txgen.py
+ 20
− 6
View file @ 964cb6e2
......@@ -21,10 +21,13 @@ from . import __version__
from . import prockey
from . import util
SNAPSHOT_MAJOR_VERSION_SUPPORTED = 0
SNAPSHOT_MINOR_VERSION_SUPPORTED = 2
STEEM_GENESIS_TIMESTAMP = 1451606400
STEEM_BLOCK_INTERVAL = 3
NUM_BLOCKS_TO_CLEAR_WITNESS_ROUND = 21
TRANSACTION_WITNESS_SETUP_PAD = 100
STEEM_MAX_AUTHORITY_MEMBERSHIP = 10
DENOM = 10**12 # we need stupidly high precision because VESTS
def create_system_accounts(conf, keydb, name):
......@@ -293,13 +296,13 @@ def update_accounts(account_stats, conf, keydb, silent=True):
new_posting_auth = cur_posting_auth.copy()
# filter to only include existing accounts
for aw in cur_owner_auth["account_auths"]:
for aw in cur_owner_auth["account_auths"][:(STEEM_MAX_AUTHORITY_MEMBERSHIP - 1)]:
if (aw[0] not in account_names) or (aw[0] in system_account_names):
new_owner_auth["account_auths"].remove(aw)
for aw in cur_active_auth["account_auths"]:
for aw in cur_active_auth["account_auths"][:(STEEM_MAX_AUTHORITY_MEMBERSHIP - 1)]:
if (aw[0] not in account_names) or (aw[0] in system_account_names):
new_active_auth["account_auths"].remove(aw)
for aw in cur_posting_auth["account_auths"]:
for aw in cur_posting_auth["account_auths"][:(STEEM_MAX_AUTHORITY_MEMBERSHIP - 1)]:
if (aw[0] not in account_names) or (aw[0] in system_account_names):
new_posting_auth["account_auths"].remove(aw)
......@@ -380,9 +383,6 @@ def build_actions(conf, silent=True):
start_time = now - datetime.timedelta(seconds=predicted_block_count * STEEM_BLOCK_INTERVAL)
miss_blocks = int((start_time - genesis_time).total_seconds()) // STEEM_BLOCK_INTERVAL
miss_blocks = max(miss_blocks-1, 0)
origin_api = None
snapshot_head_block_num = None
snapshot_semver = None
metadata = {
"txgen:semver": __version__,
......@@ -404,6 +404,20 @@ def build_actions(conf, silent=True):
if not prefix == '' and not prefix.startswith("metadata") and not prefix.startswith("dynamic_global_properties"):
break
semver = metadata.get("snapshot:semver", '0.0')
major_version, minor_version = semver.split('.')
major_version = int(major_version)
minor_version = int(minor_version)
if major_version == SNAPSHOT_MAJOR_VERSION_SUPPORTED:
if not silent:
print("metadata:", metadata)
else:
raise RuntimeError("Unsupported snapshot:", metadata)
if minor_version < SNAPSHOT_MINOR_VERSION_SUPPORTED:
print("WARNING: Older snapshot encountered.", file=sys.stderr)
yield ["metadata", metadata]
yield ["wait_blocks", {"count" : 1, "miss_blocks" : miss_blocks}]
yield ["submit_transaction", {"tx" : build_initminer_tx(conf, keydb)}]
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment