Prometheus Server and TLS

Prometheus now supports TLS and Basic Authentication for HTTP endpoints.

HTTPS HTTP . HTTPS, .

Node Exporter , HTTPS. . (. : 6 2021 ) Prometheus 2.24.0. Prometheus — TLS, backfilling ( , 2.24) React.

TLS .

Prometheus , .

Prometheus, - , , , .

API

Prometheus API . ( , ) ( , ).

Prometheus, , .

Prometheus

Prometheus :

, , , , (mangling) . Prometheus , , .

Prometheus, HTTP-.

TLS

, , Prometheus Linux.

:

$ mkdir ~/prometheus_tls_example
$ cd ~/prometheus_tls_example
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

TLS-

TLS-.

$ cd ~/prometheus_tls_example
$ openssl req -new -newkey rsa:2048 -days 365 -nodes -x509 -keyout prometheus.key -out prometheus.crt -subj "/C=BE/ST=Antwerp/L=Brasschaat/O=Inuits/CN=localhost" -addext "subjectAltName = DNS:localhost"
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

localhost — Prometheus.

: prometheus.crt prometheus.key.

- Prometheus

Prometheus v2.24.0, , , :

$ cd ~/prometheus_tls_example
$ wget https://github.com/prometheus/prometheus/releases/download/v2.24.0/prometheus-2.24.0.linux-amd64.tar.gz
$ tar xvf prometheus-2.24.0.linux-amd64.tar.gz
$ cp prometheus.crt prometheus.key prometheus-2.24.0.linux-amd64
$ cd prometheus-2.24.0.linux-amd64
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

. TLS prometheus.yml. , .

web.yml TLS:

tls_server_config:
  cert_file: prometheus.crt
  key_file: prometheus.key
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Prometheus, --web.config.file :

$ ./prometheus --web.config.file=web.yml
[...]
enabled and it cannot be disabled on the fly." http2=true
level=info ts=2021-01-05T13:27:53.677Z caller=tls_config.go:223 component=web
msg="TLS is enabled." http2=true
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

, Prometheus TLS.

: TLS , TLS, Prometheus .

.

TLS

curl TLS. :

$ cd ~/prometheus_tls_example
$ curl localhost:9090/metrics
Client sent an HTTP request to an HTTPS server.
$ curl --cacert prometheus.crt https://localhost:9090/metrics
[...]
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

--cacert prometheus.crt -k,

curl.

TLS — , . , Prometheus TLS, HTTPS.

prometheus prometheus.yml:

global:
  scrape_interval:     15s
  evaluation_interval: 15s
scrape_configs:
  - job_name: 'prometheus'
    scheme: https
    tls_config:
      ca_file: prometheus.crt
    static_configs:
    - targets: ['localhost:9090']
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

tls_config scheme https. tls_config . Prometheus.

Prometheus:

$ killall -HUP prometheus
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

https://localhost:9090/targets https://localhost:9090/metrics .

UP? ! TLS Prometheus .

. TLS , ( ).

-

( bcrypt). htpasswd ( apache2-utils httpd-tools ; , bcrypt ).

$ htpasswd -nBC 10 "" | tr -d ':\n'
New password:
Re-type new password:
$2y$10$EYxs8IOG46m9CtpB/XlPxO1ei7E4BjAen0SUv6di7mD4keR/8JO6m
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

inuitsdemo.

- Prometheus web.yml:

tls_server_config:
  cert_file: prometheus.crt
  key_file: prometheus.key
basic_auth_users:
  prometheus: $2y$10$EYxs8IOG46m9CtpB/XlPxO1ei7E4BjAen0SUv6di7mD4keR/8JO6m
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

: prometheus — .

Prometheus , - https://127.0.0.1:9090, targets 401 Unauthorized.

Prometheus

prometheus.yml, .

global:
  scrape_interval:     15s
  evaluation_interval: 15s
scrape_configs:
  - job_name: 'prometheus'
    scheme: https
    basic_auth:
      username: prometheus
      password: inuitsdemo
    tls_config:
      ca_file: prometheus.crt
    static_configs:
    - targets: ['localhost:9090']
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Prometheus SIGHUP:

$ killall -HUP prometheus
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

, Prometheus targets.

Promtool

Prometheus — promtool, -:

$ ./promtool check web-config web.yml
web.yml SUCCESS
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

web.yml.

Grafana

Grafana Prometheus. CA ( prometheus.crt) .

. CA . , . TLS , .

HTTPS Prometheus , , Alertmanager, Pushgateway.

.

: Prometheus « Kubernetes». .

Prometheus 2.24.0

Prometheus

TLS- ( Prometheus)

TLS- ( Prometheus)