👩🏻‍✈️ ✊🏿 📉 How Django can handle 100 million requests per day ⚫️ 👩🏾‍🤝‍👩🏼 🕯️

Today I will be writing about Django, a framework that has served me faithfully for the past five years. He helped me succeed in developing high-load solutions used by millions of users today.

Indeed, Python is not a very “fast” programming language, but it is simple, convenient and people love it. From a performance standpoint, it may not be as fast as Go or Node.js, but it becomes irrelevant when considering modern frameworks and modular development.

Since I've been in this "pot of Django development" for several years now, I've come to some valuable insights that I'm going to share with you.

1. Infrastructure decides

Aside from application performance, the first thing you need is a framework that allows you to scale when the application reaches its limit, and Django can scale easily if you follow these guidelines:

Divide your application into microservices, but take into account the amount of data transferred between them, especially since data redundancy and frequent synchronization cause an increase in server resources and communications, and, consequently, higher costs;
Use Docker containers to push your code to production;
Docker , , Kubernetes, ;
: / ;
: - , (. .: — Kubernetes), Kubernetes, , . — .

2. —

, . , , , , :

. PostgreSQL, , , , ;
. - (IOPS) ;
, ;
, — , : ( SELECT), ( INSERT, UPDATE). Django , , .

3. Django ORM

, ORM, . , , 100 — 20 .

, , settings.py:

LOGGING = {
    'version': 1,
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django.db.backends': {
            'level': 'DEBUG',
        },
    },
    'root': {
        'handlers': ['console'],
    }
}

— PostgreSQL, pghero.

4.

, . Django , - .

CONN_MAX_AGE — , . . 5 . , , , , 100 , .

, 0 300 . AWS Aurora db.r5.8xlarge, db.r5.4xlarge, , , , .

5. (middlewares)

, , Django REST API. (sessions) (messages) — . , .

6. bulk-

bulk-, . Django ORM SQL-.

5000 , batch_size (. .: ). Python, , , .

bulk- Django

7. select_related

, JOIN.

, 11 :

, , :

select_related , ORM SQL- JOIN. , WHERE .

8.

. , .

Django ORM QuerySet .only() , .defer(), Django , :

9. API

, API. JSON , , .

: 1 , 1 , 1 , 30 — (. .: ) .

, Django Python, , : « — ».

Django . , , (. .: «») , «» .

, , . , .

Instagram, Pinterest Disqus — Django « », . , , , , , , .

, bulk-, , . .

How Django can handle 100 million requests per day