New restrictions on Docker Hub usage and how GitLab reacted to their introduction

It's not news to anyone that starting from November 2, 2020, Docker Hub introduced restrictions on downloading images : for anonymous users it will be equal to one hundred in six hours, and for authorized users it will depend on the subscription level .

At GitLab, we followed these changes very closely, as they affected almost all of our users and customers. Today I would like to talk about what changes we made and how we adapted to the new rules.

What happened?

Docker Hub DevOps : CI/CD , , sandbox, production. , , , , .

100 6 IP docker pull 429 - too many requests, CI/CD , Kubernetes . , , GitLab Runner , .

Dependency Proxy

Tim Rizzi " Dependency Proxy", Docker Hub. GitLab ( 11.11), Enterprise Premium. : " Dependency Proxy open source , - Docker Hub?"

, . , GitLab " ?". , , Core Open Source . Docker Hub . Dependency Proxy CI/CD .

, 13.6 22 2020 , GitLab !

?

, . , , Dependency Proxy / :

• 13.7 (22 , 2020)

  • gitlab-#11582 Dependency Proxy GitLab ( )

  • gitlab-#241639 Docker Hub . , ,

  • gitlab-#21619 pull_policy YAML CI/CD , (always, if-not-present, never) , GitLab Runner

  • gitlab-runner-#26558 GitLab Runner (always, if-not-present)

, Steve Azzopardi, , 100%. , - .

?

Docker  HTTP Docker Hub.

, :

$ IMAGE="ratelimitpreview/test"
$ TOKEN=$(curl "https://auth.docker.io/token?service=registry.docker.io&scope=repository:$IMAGE:pull" | jq -r .token)

$ echo $TOKEN

-  docker pull.  GET  HEAD  ( ).  RateLimit-Limit RateLimit-Remaining.

$ curl --head -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/$IMAGE/manifests/latest

 2500,  2495  . 21600  ( )

RateLimit-Limit: 2500;w=21600
RateLimit-Remaining: 2495;w=21600

Michael Friedrich, GitLab, Python .

$ python check_docker_hub_limit.py --help

usage: check_docker_hub_limit.py [-h] [-w WARNING] [-c CRITICAL] [-v] [-t TIMEOUT]

Version: 2.0.0

optional arguments:
  -h, --help            show this help message and exit
  -w WARNING, --warning WARNING
                        warning threshold for remaining
  -c CRITICAL, --critical CRITICAL
                        critical threshold for remaining
  -v, --verbose         increase output verbosity
  -t TIMEOUT, --timeout TIMEOUT
                        Timeout in seconds (default 10s)

exit

• 0- OK

• 1- WARNING

• 2- CRITICAL

$ python3 check_docker_hub_limit.py
OK - Docker Hub: Limit is 5000 remaining 4997|'limit'=5000 'remaining'=4997

$ echo $?
0

$ python3 check_docker_hub_limit.py -w 10000 -c 3000
WARNING - Docker Hub: Limit is 5000 remaining 4999|'limit'=5000 'remaining'=4999

$ echo $?
1

$ python3 check_docker_hub_limit.py -w 10000 -c 5000
CRITICAL - Docker Hub: Limit is 5000 remaining 4998|'limit'=5000 'remaining'=4998

$ echo $?
2

Prometheus

Prometheus, .

, , Prometheus, Grafana, docker-compose

$ cd example/docker-compose

$ docker-compose up -d

http://localhost:3000 Grafana

, !