Benchmark Prometheus vs VictoriaMetrics on node_exporter metrics

For future students of the course "DevOps Practices and Tools" we prepared a translation of interesting material.



We also invite everyone to participate in the open webinar “Prometheus: Quick Start”. During the lesson, participants will consider the Prometheus architecture and how it works with metrics, as well as how to generate alerts and events in the system.

---

Recently VictoriaMetrics introduced the ability to scrap Prometheus targets . And now we can compare apples to apples: how many resources are used by Prometheus and VictoriaMetrics when scraping large amounts node_exporter



.

Benchmark setup

Testing was carried out in Google Compute Engine on four machines (instances):

• node exporter v1.0.1 . e2-standard-4 : 4 vCPU, 16 RAM, 1 HDD. , node_exporter
  
  
  
  . Prometheus VictoriaMetrics node_exporter
  
  
  
  . node_exporter
  
  
  
  nginx . node_exporter
  
  
  
  , .

  
  
  
  
  

• e2-highmem-4 Prometheus v2.22.2 VictoriaMetrics v1.47.0 : 4 vCPU, 32 RAM, 1 . VictoriaMetrics, Prometheus , (-promscrape.config = prometheus.yml
  
  
  
  VictoriaMetrics -config.file = prometheus.yml
  
  
  
  Prometheus). prometheus.yml
  
  
  
  Jinja2:

  
  
  
  
  

global:
  scrape_interval: 10s
scrape_configs:
  - job_name: node_exporter
    static_configs:
{% for n in range(3400) %}
      - targets: ['host-node-{{n}}:9100']
        labels:
          host_number: cfg_{{n}}
          role: node-exporter
          env: prod
{% endfor %}
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

host-node-{{n}}



node_exporter



. /etc/hosts



. 3400 node_exporter



.

• e2-standard-2 VictoriaMetrics Prometheus. VictoriaMetrics , node_exporter
  
  
  
  VictoriaMetrics Prometheus. .

  
  
  
  
  

node_exporter



:

• node_exporter
  
  
  
  — , Prometheus.

  
  
  
  
  

• node_exporter
  
  
  
  ( , , -, . .), Prometheus .

  
  
  
  
  

VictoriaMetrics, Prometheus nodeexporter . 24 . 

VictoriaMetrics Prometheus :

• (ingestion rate): 280 . / .

  
  
  
  
  

• (active time series): 2,8

  
  
  
  
  

• (samples scraped and stored): 24,5

  
  
  
  
  

:

Prometheus 15 , VictoriaMetrics — . VictoriaMetrics 4 . :

• VictoriaMetrics: 7,2 . 0,3 (7,2 / 24,5 ).

  
  
  
  
  

• Prometheus: 52,3 (32,3 + 18 WAL). 52,3 / 24,5 = 2,1 . , Prometheus 7 (2,1 / 0,3) .

  
  
  
  
  

-:

, Prometheus 95 / , VictoriaMetrics 15 /.

:

• 3400 node_exporter
  
  
  
  1,5–1,75 vCPU. , vCPU   4000 node_exporter
  
  
  
  .

  
  
  
  
  

• . VictoriaMetrics , OOM (out of memory
  
  
  
  ) Prometheus. ( ) VictoriaMetrics .

  
  
  
  
  

:

VictoriaMetrics 4,3 RSS-, Prometheus 6,5 14 23 . OOM , Kubernetes Prometheus. , 32 , . , Prometheus , , - OOM, .

, Prometheus 5,3 (23 / 4,3 ) , VictoriaMetrics.

Prometheus, VictoriaMetrics vCPU. , InfluxDB TimescaleDB.

VictoriaMetrics 5 7 Prometheus node_exporter



. .

P.S. VictoriaMetrics, . VictoriaMetrics ( ). , .

---

«DevOps ».



«Prometheus: ».