Drop httpx in favour of aiohttp
Closes: #34 (closed)
Requires: !157 (merged) to be merged first
Getting aiohttp
to work with ClientSession
as a singleton causes an:
aiohttp.client_exceptions.ClientOSError: [Errno None] Can not write request body
from underlying:
E ConnectionResetError: Cannot write to closing transport
to be thrown periodically (see this workaround) - looks like hived cant keep connection
but the solution with creating a new session for every request seems not to have significant effect on the performance since aiohttp is much faster than httpx.
Additionally after !157 (merged) , a slight decrease in performance should not be a problem for now, and in the future we can take care of the topic more and make it able to handle requests with aiohttp
on a singleton session
As we can observe in this benchamrk:
current solution (aiohttp session per request):
Running 10s test @ http://localhost:8000/aiohttp/single
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 10.31ms 7.72ms 99.45ms 95.58%
Req/Sec 533.61 84.57 610.00 84.50%
10628 requests in 10.01s, 1.48MB read
Requests/sec: 1062.20
Transfer/sec: 151.45KB
aiohttp
single session:
Running 10s test @ http://localhost:8000/aiohttp/session
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 5.43ms 1.51ms 26.04ms 94.24%
Req/Sec 0.94k 83.98 1.08k 75.00%
18634 requests in 10.01s, 2.59MB read
Requests/sec: 1862.19
Transfer/sec: 265.51KB
and before we got http2 session based solution
:
Running 10s test @ http://localhost:8000/httpx/session/http2
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 8.39ms 1.60ms 39.17ms 96.12%
Req/Sec 601.91 42.89 656.00 81.50%
11981 requests in 10.00s, 1.67MB read
Requests/sec: 1197.51
Transfer/sec: 170.74KB
and just to compare to session per every request with httpx
:
Running 10s test @ http://localhost:8000/httpx/single/http2
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 70.13ms 17.16ms 141.46ms 81.04%
Req/Sec 71.26 20.15 101.00 48.99%
1420 requests in 10.01s, 202.46KB read
Requests/sec: 141.87
Transfer/sec: 20.23KB
so we keep pretty much the same performance as with httpx single session
(around 10-20% performance drop) while introducing many advantages for the moment: