Client-server IPC in Python multiprocessing

The article reflects personal experience of developing CLI applications for Linux.

It discusses how the superuser process can execute privileged system calls when requested by the control program through a well-defined API.

The source code is written in Python for a real commercial application, but abstracted from specific tasks for publication.

Introduction

“Inter-process communication (IPC) is the exchange of data between threads of one or different processes. It is implemented through mechanisms provided by the OS kernel or by a process that uses OS mechanisms and implements new IPC capabilities. " - Wikipedia

Processes can have different reasons for exchanging information. In my opinion, they are all a consequence of the security policy of the Unix kernel.

As you know, the Unix kernel is an autonomous system that functions without human intervention. As a matter of fact, a user is an operating system object that appeared to protect the kernel from unauthorized interference.

Kernel security is about dividing the operating system address space into kernel space and user space . Hence the two modes of operation of the system: user mode and kernel mode. Moreover, a change of modes is a switch between two spaces.

In user mode, areas of memory reserved by the kernel are inaccessible, as are system calls that change the state of the system.

However, the superuser has this access.

Concurrency prerequisites

If your program does not use privileged system calls, you do not need a superuser, which means you can write a monolith without concurrency.

Otherwise, you will have to run your program under root.

, .

, , .

, , , , . , . , , , — .

IPC.

---

---

API .

, , .

, , .

, , daemon. «d». : systemd.

, daemon , . : systemctl.

: ssh sshd.

, . , .

.

.
├── core
│   ├── api.py
│   └── __init__.py
├── main.py
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

core



— , . api



.

API

from multiprocessing.connection import Client
from multiprocessing.connection import Listener

#   (  )  
# 
daemon = ('localhost', 6000)
#   ( )  
#   
cli = ('localhost', 6001)

def send(request: dict) -> bool or dict:
    """
        .
     ,    
          .
    """
    with Client(daemon) as conn:
        conn.send(request)
    with Listener(cli) as listener:
        with listener.accept() as conn:
            try:
                return conn.recv()
            except EOFError:
                return False

def hello(name: str) -> send:
    """
         
    send   .
    """
    return send({
        "method": "hello",
        "name": name
    })

      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

connection



multiprocessing



, API — socket.

Client



— , .

Listener



.

.

, Python, , . « » .

API

main.py



api



.

from core import api

response = api.hello("World!")
print(response)
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

. lick Framework LI , API.

API

, . .

def hello(request: dict) -> str:
    """
      .
    """
    return " ".join(["Hello", request["name"])

      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

API

from core import api
from multiprocessing.connection import Listener
from multiprocessing.connection import Client

#   ( )   
daemon = ('localhost', 6000)
#     
cli = ('localhost', 6001)
while True:
    with Listener(daemon) as listener:
        with listener.accept() as conn:
            request = conn.recv()
            if request["method"] == "hello":
                response = api.hello(request)
            with Client(cli) as conn:
                conn.send(response)

      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

, .

Thus, it always listens on port 6000 and parses the request when a datagram arrives. Then it calls the method specified in the request and returns the execution result to the client.

Additionally

I advise you to equip your server with the systemd package , which allows Python programs to log to journald .

To build you can use pyinstaller - it will pack your code into a binary file with all dependencies. Don't forget about the executable file naming convention mentioned earlier.

Thanks for attention!