Clever Geek Handbook

DNS-as-Code based on dnscontrol

Hello everyone!





This is my first post on Habré, so I ask you to treat it without being too strict. I wanted to write an article on Habr for a very long time, but there was no suitable and unique material. All the topics that could be shared, either have already been sorted out by someone, or are not very interesting ... Today I finally found something interesting that I want to tell about.





So, let's go.





By the nature of my work, I am engaged in system administration. Since I'm a lazy person myself, I like to automate everything as much as possible. Hence the love for all "Everything-as-Code" tools.





In between writing playbooks for Ansible, code for Terraform and other infrastructure automation tools, I began to look for new entertainment for myself , what else can I automate in order to press buttons and run scripts even less in my work.





And then I remembered about our external DNS zone, which we host at nic.ru. At that time, we had several domains, which in total contained about 3000 records, and about 80% of them were records for developers' stands, which differ from each other only by serial numbers or some kind of suffix.





Example 
dev1           A      1.2.3.4
dev1-serviceA  CNAME  dev1
dev1-serviceB  CNMAE  dev1
...
dev2           A      1.2.3.4
dev2-serviceA  CNAME  dev2
dev2-serviceB  CNMAE  dev2
...

We had about 100-150 such dev <N> at different points in time. All this was complicated by the fact that periodically these records had to be changed / added / deleted. For example, some DEV stands need to be wrapped in a different IP, or each DEV stand needs some other separate CNAME, etc.





All this stuff was controlled either manually (fix / add single records), or by tying scripts with a bunch of parameters, bugs and dancing with a tambourine with large manuals.





And then one evening the thought occurred to me "hmm, but these records are very easy to program, they can be generated by some simple algorithm ...", "... for virtual machines in the cloud, we have Terraform, there is Ansible / Puppet / ... for configuration management, maybe there is something for DNS too? ".





"DNS as Code" , . 5-10 Dnscontrol Octodns. , , ...





.





octodns

OctoDNS – , « », DNS-. , , . OctoDNS  GitHub  Python.





: DigitalOcean





, . ? :) ? ? :





OctoDNS DNS, (YAML).





, , YAML- . YAML.





, , :





  • (NIC.RU, , )





  • DNS-





  • CI/CD





DNS- YAML:





~/octodns/config/config.yaml 
---
providers:
  config:
    class: octodns.provider.yaml.YamlProvider
    directory: ./config
    default_ttl: 300
    enforce_order: True
  digitalocean:
    class: octodns.provider.digitalocean.DigitalOceanProvider
    token: your-digitalocean-oauth-token

zones:
  your-domain.:
    sources:
      - config
    targets:
      - digitalocean
~/octodns/config/your-domain.yaml 
---
'':
  - type: A
    value: 1.2.3.4

www:
  type: A
  value: 5.6.7.8

. :





  • ,





  • Bind





. ...





Dnscontrol

DNSControl — ,  « », DNS , , . DNSControl Stack Exchange Go.





: DigitalOcean





- , ? ( Go Stack Exchange). , , , :





DNSControl uses javascript as its primary input language to provide power and flexibility to configure your domains. The ultimate purpose of the javascript is to construct a DNSConfig object that will be passed to the go backend and operated on.





: StackExchange





? "" JavaScript? ? , , .





Dnscontrol:





dnscontrol.js 
// define dummy-registar and Bind-provider
REG_NONE = NewRegistrar('none', 'NONE');
DNS_BIND = NewDnsProvider('bind', 'BIND', {
    // default SOA-records for all domains
    'default_soa': {
        'master': 'ns3-l2.nic.ru.',
        'mbox': 'dns.nic.ru.',
        'refresh': 9999,
        'retry': 9999,
        'expire': 9999000,
        'minttl': 999,
    },
    // default NS-records for all domains
    'default_ns': [
        'ns8-cloud.nic.ru.',
        'ns3-l2.nic.ru.',
        'ns4-l2.nic.ru.',
        'ns8-l2.nic.ru.',
        'ns4-cloud.nic.ru.'
    ]
});

:





my-zone.ru.js 
function myzone_ru(REG, PROVIDER){
    return D('pcbltools.ru', REG, DnsProvider(PROVIDER),
        DefaultTTL('5m')
        ,A('@', '1.2.3.4')
        ,MX('@', 10, 'mx.yandex.net.', TTL('6h'))
        ,MX('@', 20, 'mx.yandex.ru.', TTL('6h'))
        ,A('www', '1.2.3.4')
        ,CNAME('portal', 'www')
        ,AAAA('configurator', '2a00:56:2:2:1:1:0:64f')
    )
}

, YAML . , JavaScript, . , :





Advanced Topics:





Code Tricks: Safely use macros and loops.





, . :





The dnsconfig.js language is JavaScript. On the plus side, this means you can use loops and variables and anything else you want...





Sure, you can do a lot of neat tricks with if/then



s and macros and loops. Yes, YOU understand the code. However, think about your coworkers who will be the next person to edit the file. Are you setting them up for failure?





() JavaScript. , . , , ( ). , .





- :





my-zone.ru.js 
function generate_DEV_records (REG, PROVIDER){
    DEV_CNAME_RECORDS = [
        'serviceA'
        ,'serviceB'
    ]
	  dev_stand_count = 5
    dev_public_ip = '1.2.3.4'
    RECORDS = []
    for (var i = 1; i <= dev_stand_count; i++){
        RECORDS.push(
            A('dev' + i, dev_public_ip)
        )
        for (var j = 0; j < DEV_CNAME_RECORDS.length; j++){
            RECORDS.push(
                CNAME('dev' + i + '-' + DEV_CNAME_RECORDS[j], 'dev' + i)
            )
        }
    }

	  return D('myzone.ru', REG, DnsProvider(PROVIDER),
        RECORDS)
}

Bind, Bind. .





dnscontrol push:





❯ dnscontrol push
******************** Domain: myzone.ru
----- Getting nameservers from: bind
----- DNS Provider: bind...File does not yet exist: "zones/myzone.ru"
1 correction
#1: GENERATE_ZONEFILE: 'myzone.ru' (new file with 21 records)

WRITING ZONEFILE: zones/myzone.ru
SUCCESS!
----- Registrar: none...0 corrections
Done. 1 corrections.

zones Bind:





myzone.ru 
$TTL 300
; generated with dnscontrol 2021-03-24T23:15:09+03:00
@                IN SOA   ns3-l2.nic.ru. dns.nic.ru. 2021032400 1440 3600 2592000 600
                 IN NS    ns3-l2.nic.ru.
                 IN NS    ns4-cloud.nic.ru.
                 IN NS    ns4-l2.nic.ru.
                 IN NS    ns8-cloud.nic.ru.
                 IN NS    ns8-l2.nic.ru.
dev1             IN A     1.2.3.4
dev1-servicea    IN CNAME dev1.myzone.ru.
dev1-serviceb    IN CNAME dev1.myzone.ru.
dev2             IN A     1.2.3.4
dev2-servicea    IN CNAME dev2.myzone.ru.
dev2-serviceb    IN CNAME dev2.myzone.ru.
dev3             IN A     1.2.3.4
dev3-servicea    IN CNAME dev3.myzone.ru.
dev3-serviceb    IN CNAME dev3.myzone.ru.
dev4             IN A     1.2.3.4
dev4-servicea    IN CNAME dev4.myzone.ru.
dev4-serviceb    IN CNAME dev4.myzone.ru.
dev5             IN A     1.2.3.4
dev5-servicea    IN CNAME dev5.myzone.ru.
dev5-serviceb    IN CNAME dev5.myzone.ru.

, dnscontrol preview, . dnscontrol push





, ?





. JS- , :





DNS as Code.





CI

, , CI. Infrastructure as Code , . :













  • CI/CD









, , .





Gitlab. Container Registry CI , , .





, , . :





  • validate -





  • prepare - , NIC.RU





  • plan -





  • build -





  • test - DNS-





  • deploy - NIC.RU





docker-, .





2 :





  • stackexchange/dnscontrol - , test





  • internetsystemsconsortium/bind9 - test





, .. ( ).





Gitlab CI :





.gitlab-ci.yml 
image: $CI_REGISTRY_IMAGE/dnscontrol

variables:
  CA_CERT_FILE: /etc/gitlab-runner/certs/ca.crt
  ZONES_OUT_DIR: $CI_PROJECT_DIR/zones
  NIC_API_URL: https://api.nic.ru
  NIC_SERVICE: MYSERVICE

cache:
  key: dns-nic-ru
  paths:
    - .nic_token

stages:
  - validate
  - prepare
  - plan
  - build
  - test
  - deploy

check:
  stage: validate
  script:
    - dnscontrol -v check

prepare:
  stage: prepare
  script:
    - mkdir -p $ZONES_OUT_DIR
    - dnscontrol push
    - ls -la $ZONES_OUT_DIR
    #   NIC.RU (  )
    - . ci/scripts/nic_auth.sh
    #      NIC.RU
    - ci/scripts/nic_download.sh
    - ls -la $ZONES_OUT_DIR
  artifacts:
    public: false
    paths: [ zones/ ]
    expire_in: 5 mins

plan:
  stage: plan
  script:
    #      
    - dnscontrol preview | tee plan.txt
  artifacts:
    #    Merge Request,     
    expose_as: plan
    paths: [ plan.txt ]
    public: false
    expire_in: 3 mos

build:
  stage: build
  script:
    - dnscontrol -v push
  artifacts:
    name: zones
    expose_as: zones
    paths: [ zones/ ]

test:
  stage: test
  image: $CI_REGISTRY_IMAGE/bind9
  variables:
    BIND_MAIN_CONFIG: /etc/bind/named.conf
    BIND_ZONES_DIR: /var/lib/bind/
    BIND_TESTS_DIR: $CI_PROJECT_DIR/tests
  script:
    - cat ci/bind9/named.conf > $BIND_MAIN_CONFIG
    - cp $ZONES_OUT_DIR/* $BIND_ZONES_DIR/
    #   bind    
    - ci/scripts/zones.conf.sh >> $BIND_MAIN_CONFIG
    - cat $BIND_MAIN_CONFIG
    #   
    - /usr/sbin/named-checkconf /etc/bind/named.conf
    #  bind   
    - /usr/sbin/named -g -c /etc/bind/named.conf -u bind &
    #    bind
    - while ! (ss -4tulnp | grep 53 > /dev/null); do echo "Waiting for a socket to go up"; sleep 1; done
    - ps aux
    #   (,     )
    - ci/scripts/bind_test.sh

deploy:
  stage: deploy
  script:
    #      
    - . ci/scripts/nic_auth.sh
    #    NIC.RU
    - ci/scripts/nic_upload.sh
  dependencies:
    - build
  rules:
    - if: '$CI_COMMIT_BRANCH == "master"'
      when: manual

, plan test.





plan , . expose_as. , Merge Request, . , . :





plan, Job, :





test bind-. .. DNS- Bind, . , , .





At some steps, additional shell scripts are used, which contain the necessary logic. It all depends on your needs.





results

What we managed:





  1. Build DNS as Code process based on dnscontrol tool





  2. Enforce all development practices with Gitlab





  3. Reduce the time it takes to add DNS records





  4. Create a single source of truth for DNS in the form of a code repository





Thank you all for your attention, I will be happy to answer any questions about the presented material.







More articles:


All Articles