Today I will tell you about a rather simple but interesting implementation of multithreading in an HTTP server without creating a thread for each client. To my surprise, I hardly found information about such an implementation, so I decided to share it with you. Let's start with a description of the problem.
Problems of "one thread = one client" solution
The problems described below are true for both threads and processes, so "one thread = one client" can also be regarded as "one process, one client" in this context.
The first problem is that the number of threads that can be created in a program is limited. As a result, the number of users connected to our server is also limited. Apache has such a problem, for example.
The second problem is that one thread is occupied by only one client. In this regard, we get an inefficient use of resources. (the thread can be idle while it is waiting for an event from the client)
In addition to all this, you need to understand that creating a thread (or process) is a rather difficult operation, and sometimes it requires more costs than the customer service itself.
The solution I provide below closes these problems.
There is a solution
The first problem is solved by making the number of threads independent of the number of our clients. We control the number of threads ourselves (it can be set both statistically, when the server starts, or dynamically, adjusting to the load, for example).
The second problem is closed by the fact that streams are not attached to clients. In fact, customer service is broken down into tasks that are handled by threads. Thus, we get rid of idle threads, if there is work for them.
Finite state machine
, , . (, ) , . , . , , .
. . : readRequest, generateResponse, sendResponse closeConnection ( , , , ). . readRequest , (, , ), generateResponse, closeConnection. generateResponse sendResponse. sendResponse readRequest, closeConnection. closeConnection, , .
. ( : ) . , . readRequest - parsingRequest, .
. , , :)
ThreadPool ( )
. ( , ) .
: . , , . ( - ) , , « , », ( , ). , , , . . - .
, " = " . ( ) , ( ).
, , , , , .
This time I just outlined the essence of the approach. If you are interested in seeing the continuation of the article already with the practical part (approaches to implementation and their pitfalls) - let me know about it :)
That's all. Share your options, suggestions, additions and criticism in the comments! Thanks for reading :)
Several useful links:
https://habr.com/ru/post/260065/
https://habr.com/ru/company/latera/blog/273283/
http://www.aosabook.org/en/nginx.html