Undoubtedly, the Gravitee interface provides a fairly visual and convenient visualization of the operation of Gravitee gateways. But in any case, there is a need to provide access to these tools to the monitoring service, API owners or consumers, and at the same time they can be outside the closed loop in which the API manager is located. And it is always more convenient to have all the available information on various APIs on one screen.
See what happens on gateways without going into the specifics of the Gravitee user interface, and administrators do not waste time creating users and separating roles and privileges within Gravitee.
Habré already had a couple of articles dedicated to APIM Gravitee, here and here... Therefore, in my note, I will imply that the reader is already familiar with the process of installing / configuring APIM Gravitee and Grafana, I will only consider the process of configuring their integration.
Why can't you go the easy way?
By default, the repository for Gravitee analytics is ElasticSearch. The information is accumulated in four different indices, with a daily breakdown:
gravitee-request-YYYY.MM.DD - information for each request is stored here (similar to access.log in nginx). This is our main goal;
gravitee-log-YYYY.MM.DD - more detailed information about the request is already stored here (provided that debugging is enabled, see the figure below). Namely, the full request and response headers, as well as the payload. Depending on the settings, both the exchange between the consumer and the gateway and / or the gateway and the API provider can be logged;
Advanced logging enable / disable screen
gravitee-monitor-YYYY.MM.DD - we are not interested in this one;
gravitee-health-YYYY.MM.DD - this one doesn't interest us.
, : ElasticSearch Grafana , .
, , .. - , . , Grafana, . Gravitee . , MongoDB PostgreSQL, . ( Grafana) - , - .
?
PostgreSQL , ElasticSearch (). , Grafana - PostgreSQL, ElasticSearch .
( ).
, !
: CentOS 7, APIM Gravitee 3.6, PostgreSQL 11, ElasticSearch 7.+
PostgreSQL ElasticSearch. :
multicorn11 pip, :
yum install multicorn11 python3-pip
pip-, python3 ElasticSearch:
pip3 install pg_es_fdw
, PostgreSQL. multicorn :
GRANT USAGE on FOREIGN DATA WRAPPER multicorn TO gatewaytest; GRANT USAGE ON FOREIGN SERVER multicorn_es TO gatewaytest;
CREATE EXTENSION multicorn; CREATE SERVER multicorn_es FOREIGN DATA WRAPPER multicorn OPTIONS (wrapper 'pg_es_fdw.ElasticsearchFDW');
, . logreader:
GRANT USAGE on FOREIGN DATA WRAPPER multicorn TO logreader; GRANT USAGE ON FOREIGN SERVER multicorn_es TO logreader;
, logging, logreader:
CREATE SCHEMA logging AUTHORIZATION logreader;
, :
CREATE TABLE logging.requests ( id varchar(36), "@timestamp" timestamp with time zone, api varchar(36), "api-response-time" int, application varchar(36), custom json, endpoint text, gateway varchar(36), "local-address" varchar(16), method int, path text, plan varchar(36), "proxy-latency" int, "remote-address" varchar(16), "request-content-length" int, "response-content-length" int, "response-time" int, sort text, status int, subscription varchar(36), uri text, query TEXT, score NUMERIC) PARTITION BY RANGE("@timestamp");
, , , - .
, shell- cron:
#!/bin/sh NEWPART=${1:-$(date +'%Y.%m.%d')} OLDPART=$(date --date='14 days ago' +'%Y.%m.%d') curl http://gateway.corp/test psql gateway -U logreader -c "CREATE FOREIGN TABLE logging.\"requests_${NEWPART}\" PARTITION OF logging.requests FOR VALUES FROM ('${NEWPART} 00:00:00') TO ('${NEWPART} 23:59:59') SERVER multicorn_es OPTIONS ( host 'els-host', port '9200', index 'gravitee-request-${NEWPART}', rowid_column 'id', query_column 'query', query_dsl 'false', score_column 'score', sort_column 'sort', refresh 'false', complete_returning 'false', timeout '20', username 'elastic-ro', password 'Sup3rS3cr3tP@ssw0rd');" psql gateway -U gatewaydev -c "drop foreign table logging.\"requests_${OLDPART}\""
:
NEWPART - , , ElasticSearch;
OLDPART - , , 14 ( ES Curator). , - . , , ;
'curl http://gateway.corp/test', , , API. , , . ;
, ;
- .
TABLE logging.requests LIMIT 1;
,
-[ RECORD 1 ]-----------+------------------------------------- id | 55efea8a-9c91-4a61-afea-8a9c917a6133 @timestamp | 2021-05-16 00:00:02.025+03 api | 9db39338-1019-453c-b393-381019f53c72 api-response-time | 0 application | 1 custom | {} endpoint | gateway | 7804bc6c-2b72-497f-84bc-6c2b72897fa9 local-address | 10.15.79.29 method | 3 path | plan | proxy-latency | 2 remote-address | 10.15.79.27 request-content-length | 0 response-content-length | 49 response-time | 2 sort | status | 401 subscription | uri | /test query | score | 1.0
, - Gravitee. , , PostgreSQL, . , : , ; ; , .
, Metadata, Grafana.
:
Grafana:
SELECT
name "",
value ""
FROM
metadata
WHERE
reference_id='${apis}'
APIs () - .
SELECT COUNT(*) AS "" FROM apis;
SELECT COUNT(*) AS "" FROM apis WHERE lifecycle_state='STARTED';
Applications, , applications
API Hits - .
SELECT
date_trunc('minute',"@timestamp") AS time,
apis.name,ee Grafana

COUNT(*)
FROM
logging.requests al
JOIN
apis ON al.api = apis.id
WHERE
query='@timestamp:[$__from TO $__to]'
GROUP BY 1,2
Average response time by API - , .
SELECT
date_trunc('minute',"@timestamp") AS time,
apis.name,
AVG(al."api-response-time")
FROM
logging.requests al
JOIN
apis ON al.api = apis.id
WHERE
query='@timestamp:[$__from TO $__to]'
GROUP BY 1,2
Hits, by gateways, . :
SELECT
date_trunc('minute',"@timestamp") as time,
al."local-address",
COUNT(*)
FROM
logging.requests al
WHERE
query='@timestamp:[$__from TO $__to]'
GROUP BY 1,2
The above solution, in my subjective opinion, is in no way inferior to the standard APIM Gravitee visualization tools, but is limited only by imagination and needs.
Considering that Grafana is usually the central object of the monitoring infrastructure, the advantages of such a solution are obvious: wider coverage, higher information density and simple customization of visual representations.
PS
In the near future, another article is planned on integrating Gravitee with ActiveDirectory. The process is quite simple, but as always, there are nuances.
Constructive criticism, wishes and suggestions are welcome!