Clever Geek Handbook

Independent HttpBench for D, or are TechEmpower's tests lying?

A simple framework for testing HTTP servers, inspired by the Simple Web Benchmark ( there is a graph for many other languages ), but focused on dlang frameworks and libraries.



It measures the achievable RPS (requests per second) in a plaintext script.



Tests have been collected or modified from various places (including TechEmpower ).



It uses a docker container to build and host services and can run locally or use a load tester from a remote host.



The default load generator is wrk and requests statistics from the collector, but hey is supported too (just use the --tool switch).



Tests can be run without docker, you just need to install compilers for the tested languages ​​and the wrk / hey boot generator (but only tested on linux).



Note for io_uring tests:



  • The tests require at least a Linux 5.7 kernel.
  • There may also be problems with user-defined locked memory limits (ulimit -l) when run with a normal user.
  • , Kernel 5.7.16. . : #189, #8.




:



  • singleCore — / ( ).
  • multiCore —






  • make build —
  • make shell —

    : .




, ( ): 



make all        # runs all tests
make single     # runs tests limited to single CPU core usage
make multi      # runs tests limited to multiple CPU cores usage


_suite/runner.d CLI-.



  • _suite/runner.d list — -
  • _suite/runner.d bench —
  • _suite/runner.d responses —
  • _suite/runner.d versions — Markdown

    _suite/runner.d -h CLI .


:



_suite/runner.d bench --type singleCore dlang rust #     dlang  rust




(. ), CLI .



:



  • , , .
  • - _suite/runner.d bench --type singleCore -r foo@192.168.0.3 --host 192.168.0.2 dlang


-r --remote , ssh. --host , CLI IP , , .



ssh , ssh ( ).



(hey) .



, , .



/



Techempower .



, .



  • (.. ..)
  • HTTP-


, , , , , .



C



. HTTP-, eventloop .



epoll io_uring. raw.



dlang



arsd-official



. http :



  • process — , .
  • threads — threadpool
  • hybrid — Linux , .


singeCore, ( ) eventloop, (.. CGI) .



. — .



during



  • raw — , , ( , ,… — )

    TBD — / io_uring, epoll Linux-.


epoll



, , . , — , during/raw



eventcore



, vibe-d. epoll linux (iocp kqueue windows MacOS).



, http , , .



  • callbacks — .
  • fibers —


hunt





lighttp



code.dlang.org, .



, .



mecca



Weka.



.



: druntime, . . , .



photon



code.dlang.org, , syscalls glibc epoll eventloop , .



nodejs http-parser ( , pico) .



vibe-core



, eventcore .



- , TCPConnection , .



, http .



vibe-d



- dlang, .



dotnet



ASP.Net Core.



, ( , , ...). .. , .



golang



fasthttp .



HTTP , .



rust



Actix :



  • actix-web —
  • actix-raw —




Single core results ()



: AMD Ryzen 7 3700X 8-Core, kernel 5.8.10

: Intel® Core(TM) i5-5300U CPU @ 2.30GHz, kernel 5.8.9

: 1Gbps through cheap gigabit switch

: for i in 8 64 128 256; do _suite/runner.d bench --type singleCore --tool wrk -b 2 -d 120 -c $i -r tomas@10.0.0.2; done
go go1.15.1
ldc2 1.23.0
rust 1.48.0-nightly
dotnet 5.0.100-rc.1.20452.10


From a translator . Full versions of tables with results can be viewed in the original . But everything is not obviously mixed up there, so I made more visual graphs on the number of processed requests in 2 minutes (req, rps proportionally) for two categories - full-fledged frameworks without tweaks and all the rest, including basic libraries. And two results - for 8 workers and for 256 to show scalability.







And yes, TechEmpower's tests, mixing tweaked versions with the usual ones, pretty much embellish in places.



More articles:


All Articles