Changes

Page history
Update benchmarks authored by Dan Notestein's avatar Dan Notestein
Hide whitespace changes
Inline Side-by-side
benchmarks.md 0 → 100644
View page @ cab7c369
[sql_serializer benchmarks](/hive/haf/-/wikis/sql-serializer-benchmarks)
[io test results](https://gitlab.syncad.com/hive/haf/-/wikis/I/Test-results)
As a longterm alternative to putting block_log on compressed drive, we also have eric's code for maintaining a compressed block_log: 318931202547 bytes compressed / 575437644643 bytes uncompressed.
<table>
<tr>
<th>Machine</th>
<th>test</th>
<th>num of blocks indexed</th>
<th>file system</th>
<th>final size</th>
<th>time taken</th>
<th>file sizes</th>
<th>Load test results</th>
</tr>
<tr>
<td>Shed14: test 1</td>
<td>
single 2TB nvme test (Samsung 980 Pro)
shared memory went to /hafpool/datadir-haf/blockchain on the nvme
</td>
<td>62,000,000</td>
<td>ZFS lz4; comp. ratio = 2.15x</td>
<td>1.5 TB (includes blocklog)</td>
<td>12 Hrs + 7 Hrs adding new indexes</td>
<td>
/hafpool:
/haf_database_store: **1.2**Tb
/datadir-haf: **361**Gb
</td>
<td>summary = 20004 in 00:08:11 = 40.7/s Avg: 243 Min: 17 Max: 5076 Err: 0 (0.00%)</td>
</tr>
<tr>
<td>Steem-11: test 1</td>
<td>
2 x 2TB nvme (Samsung 980 pro) & 16TB HDD to hold the block log; unraided
shared memory went to /storage1/datadir-haf/blockchain on the magnetic drive
</td>
<td>62,000,000</td>
<td>ZFS uncompressed</td>
<td>2.7 TB + 550 GB (blocklog)</td>
<td>20.45 Hrs + 6.8 Hrs to add indexes</td>
<td>
<div>
/hafpool:
/haf_database_store: **2.6** Tb
/storage1:
/datadir-haf: **549** Gb
</div></td>
<td>summary = 20004 in 00:07:48 = 42.7/s Avg: 231 Min: 17 Max: 5212 Err: 0 (0.00%)</td>
</tr>
<tr>
<td>Steem-10: test 1</td>
<td>
2 x 2TB nvme (samsung 980 pro) raided & 1 x 2TB nvme (MP600 Pro) to hold block log
shared memory went to /storage1/datadir-haf/blockchain on the nvme raid
</td>
<td>62,000,000</td>
<td>ZFS uncompressed</td>
<td>2.7 TB + 550 (blocklog)</td>
<td>12.3 Hrs + 6.1 Hrs to add indexes</td>
<td>
/hafpool:
/haf_database_store: **2.7**Tb
/storage2
/datadir-haf: **549**Gb
</td>
<td>summary = 20004 in 00:06:25 = 52.0/s Avg: 189 Min: 16 Max: 5231 Err: 0 (0.00%)</td>
</tr>
<tr>
<td>Shed14: test 2</td>
<td>
2 x 2TB nvme (MP600 Pro XT) raided
shared memory went to /storage1/datadir-haf/blockchain on the nvme raid
</td>
<td>62,000,000</td>
<td>ext4</td>
<td>3.5TB (blocklog included)</td>
<td>11.75 Hrs + 5.25 Hrs to add indexes</td>
<td>
<div>
/hafpool:
/haf_database_store: **2.7**Tb
/datadir-haf: **555**Gb
</div></td>
<td>summary = 20004 in 00:05:58 = 55.9/s Avg: 176 Min: 16 Max: 5061 Err: 0 (0.00%)</td>
</tr>
<tr>
<td>Steem-11: test 2</td>
<td>2 x 2TB nvme (samsung 980 Pro) raided; block log on 16TB magnetic drive shared memory going to /dev/shm</td>
<td>62,000,000</td>
<td>ZFS lz4; comp. ratio = 2.34x</td>
<td>1.3 TB + 530 for blocklog</td>
<td>11.7 Hrs + 7 Hrs to add indexes</td>
<td>
<div>
/hafpool:
/haf_database_store: **1.2**Tb
/storage1:
/datadir-haf: **531**Gb
/dev/shm: **19**Gb
</div></td>
<td>summary = 20004 in 00:08:26 = 39.5/s Avg: 251 Min: 16 Max: 5237 Err: 0 (0.00%)</td>
</tr>
<tr>
<td>Steem-10: test 2</td>
<td>
2 x 2TB nvme (samsung 980 pro) & 1 x 2TB nvme (mp600 Pro) to store block log
shared data going to /dev/shm
</td>
<td>62,000,000</td>
<td>ZFS lz4; comp. ratio = 2.34x</td>
<td>1.2 TB + 530GB for blocklog</td>
<td>10.4 Hrs + 5.5 Hrs to add indexes</td>
<td>
/hafpool:
/haf_database_store: **1.2**TB
/storage2:
/datadir-haf: **531** Gb
/dev/shm: **19**Gb
</td>
<td>summary = 20004 in 00:07:06 = 46.9/s Avg: 211 Min: 15 Max: 4937 Err: 0 (0.00%)</td>
</tr>
<tr>
<td>Shed14: test 3</td>
<td>1 x 2TB nvme (samsung 980 pro) shared data going to /dev/shm</td>
<td>62,000,000</td>
<td>ZFS lz4; comp. ratio = 2.15x</td>
<td>1.5 TB (including blocklog)</td>
<td>10.8 Hrs + 6 Hrs to add indexes</td>
<td>
/hafpool:
/haf_database_store: **1.2**TB
/datadir-haf: **354** Gb
/dev/shm: **19**Gb
</td>
<td>summary = 20004 in 00:06:23 = 52.3/s Avg: 189 Min: 17 Max: 3657 Err: 0 (0.00%)</td>
</tr>
<tr>
<td>Steem-10: test 3</td>
<td>
2 x 2TB nvme (Samsung 980 Pro) raided & 1 x 2TB nvme (MP600 Pro) to store the block log
shared data going to /dev/shm
</td>
<td>62,000,000</td>
<td>ZFS uncompressed</td>
<td>2.7TB + 550 Gb for block log</td>
<td>10.8 Hrs + 6.2 Hrs to add indexes</td>
<td>
<div>
/hafpool:
/haf_database_store: **2.7**TB
/datadir-haf: **550** Gb
/dev/shm: **19**Gb
</div></td>
<td>summary = 20004 in 00:06:15 = 53.3/s Avg: 185 Min: 15 Max: 3974 Err: 0 (0.00%)</td>
</tr>
<tr>
<td>Steem-11: test 3</td>
<td>
<div>2 x 2TB nvme (samsung 980 Pro) raided; block log on 16TB magnetic drive shared memory going to /dev/shm</div></td>
<td>62,000,000</td>
<td>ZFS uncompressed</td>
<td>2.7TB + 550 Gb for block log</td>
<td>11.7 Hrs + 7 Hrs to add indexes</td>
<td>
<div>
/hafpool:
/haf_database_store: **2.7**TB
/datadir-haf: **550** Gb
/dev/shm: **19**Gb
</div></td>
<td>summary = 20004 in 00:05:46 = 57.8/s Avg: 170 Min: 16 Max: 3844 Err: 0 (0.00%)</td>
</tr>
<tr>
<td>Shed14: test 4</td>
<td>1 x 2TB nvme (samsung 980 pro) shared data going to /dev/shm</td>
<td>62,000,000</td>
<td>
ZFS lz4; comp. ratio = 1.93x
changed ZFS to write in 16k blocks instead of the default 128k
</td>
<td>1.3 TB + 381 GB for block log</td>
<td>11.1 Hrs + 10.5 Hrs to ad indexes</td>
<td>
<div>
/hafpool:
/haf_database_store: **1.3**TB
/datadir-haf: **381** Gb
/dev/shm: **18**Gb
</div></td>
<td>summary = 20004 in 00:05:58 = 55.8/s Avg: 177 Min: 16 Max: 4149 Err: 0 (0.00%)</td>
</tr>
<tr>
<td>
Steem-11:
test 4
</td>
<td>
(samsung 980 Pro) raided; block log on 16TB magnetic drive shared memory going to /dev/shm
/var/lib/postgresql now on nvme storage
</td>
<td>62,000,000</td>
<td>ZFS uncompressed</td>
<td>in progress</td>
<td>10.5 Hrs + in progress</td>
<td>in progress</td>
<td>
</td>
</tr>
<tr>
<td>
Shed14:
test 5
</td>
<td>
1 x 2TB nvme (samsung 980 pro) shared data going to /dev/shm
/var/lib/postgresql now on nvme storage
</td>
<td>62,000,000</td>
<td>
ZFS lz4: comp. ratio: \~\~\~
changed recordsize back to 128k
</td>
<td>in progress</td>
<td>in progress</td>
<td>in progress</td>
<td>
</td>
</tr>
<tr>
<td>Shed14</td>
<td>
<div>
8T drive with nvme cache (Samsung 980 Pro)
shared memory went to /hafpool/datadir-haf/blockchain on the magnetic drive
</div></td>
<td>60,000,000</td>
<td>ZFS lz4</td>
<td>N/A (failed)</td>
<td>N/A (failed)</td>
<td>N/A (failed)</td>
<td>N/A (failed)</td>
</tr>
<tr>
<td>Steem-10</td>
<td>16TB drive with nvme cache (MP600 Pro)</td>
<td>62,000,000</td>
<td>ZFS uncompressed</td>
<td>test failed</td>
<td>17.5 Hrs</td>
<td>
<div>
/hafpool:
/haf_database_store: test failed
/storage1:
/datadir-haf: **549**Gb
</div></td>
<td>N/A (failed)</td>
</tr>
<tr>
<td>Steem-10</td>
<td>16TB drive with nvme cache (MP600 Pro)</td>
<td>60,000,000</td>
<td>ZFS uncompressed</td>
<td>test failed</td>
<td>15 Hrs</td>
<td>
<div>
/hafpool:
/haf_database_store: test failed
/storage1:
/datadir-haf: **549**Gb
</div></td>
<td>N/A (test had already been reset)</td>
</tr>
<tr>
<td>Steem-11</td>
<td>2TB nvme & 8TB HDD to hold the block log (Samsung 980 Pro)</td>
<td>60,537,000</td>
<td>ZFS uncompressed</td>
<td>test failed</td>
<td>19 Hrs before cutoff</td>
<td>
<div>
/hafpool:
/haf_database_store: test failed
/storage1:
/datadir-haf: **549**Gb
</div></td>
<td>
N/A
(failed)
</td>
</tr>
</table>