Data validation

Hello, Habr! On the eve of the start of the course "Network Architect" we suggest reading the translation of a useful article.

Optimizing the data model and removing duplicates is great, but how can we make sure that we are working with a valid data model?

IPAM/CMDB , REST API, , . , , , .

, (   source-of-truth), , , . , , :

• ;

• ;

• .

per-link , Ansible host_vars network.yml , :

• YAML ( , yamllint);

• hostname bgp_as ;

• links, core edge ;

• Core prefix ;

• Edge , ( , JSON) .

( ), YANG, JSON Schema XML Schema, . YAML JSON, jsonschema.

. , , , , , .

Ansible. « » Ansible. , — ansible-inventory, JSON (external inventory script).

$ ansible-inventory -i ../hosts --list
{
    "_meta": {
        "hostvars": {
            "S1": {
                "bgp_as": 65001,
                "description": "Unexpected",
                "hostname": "S1"
            },
            "S2": {
                "bgp_as": 65002,
                "hostname": "S2"
            }
        }
    },
    "all": {
        "children": [
            "ungrouped"
        ]
    },
...

JSON , jq :

$ ansible-inventory -i ../hosts --list|jq ._meta.hostvars
{
  "S1": {
    "bgp_as": 65001,
    "description": "Unexpected",
    "hostname": "S1"
  },
  "S2": {
    "bgp_as": 65002,
    "hostname": "S2"
  }
}

jsonschema, , jsonschema JSON Schema, .

$ jsonschema -i /tmp/hosts.json hosts.schema.json
{'bgp_as': 65001, 'description': 'Unexpected', 'hostname': 'S1'}: 
Additional properties are not allowed ('description' was unexpected)

network.yml :

• Convert YAML file into JSON format with yq

• YAML JSON yq

• Run jsonschema on the resulting JSON file

• jsonschema JSON

yq <network.yml . >/tmp/$$.network.json
jsonschema -i /tmp/$$.network.json network.schema.json

, JSON Schema , — . :

• , , core edge .

• , , , , , .

JSON Schema

, JSON Schema . ( ISO IEEE). , - JSON Schema .

, JSON Schema: JSON, , Ansible:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://www.ipSpace.net/hosts.schema.json",
  "title": "Ansible inventory data",
  "definitions": {
    ...
  },
  "type": "object",
  "patternProperties": {
    ".*" : { "$ref" : "#/definitions/router" }
  },
  "minProperties": 1
}

:

• Ansible (Ansible inventory data);

• (. ).

• — () ( , ), router

• — ( ).

router definitions:

"router" : {
  "type" : "object",
  "properties": {
    "bgp_as": {
      "type": "number",
      "minimum": 1,
      "maximum": 65535
    },
    "hostname": {
      "type": "string"
    }
  },
  "required": [ "bgp_as","hostname" ],
  "additionalProperties": false
}

( , Ansible, ) — :

• bgpas, 1 65535;

• hostname 

• , .

JSON , GitHub. , host_vars .

JSON Schema , :

• , JSON network;

• description ;

• bgp_as, 4- AS .

:

• jq

• yq

• jsonschema

CI/CD Validation, Error Handling and Unit Tests - Building Network Automation Solutions.

• GitHub.

• , 3 - Building Network Automation Solutions.

• Data Model Hierarchy

" ". "Overlay. " .