👪 🤚🏽 ✍🏾 So how can you not suffer from functional tests? 📡 🧑🏾‍🤝‍🧑🏻 🎵

I was prompted to write this article by discussing reports from Heisenbug 2021 in our corporate chat. This is due to the fact that a lot of attention is paid to the "correct" writing of tests. In quotes - because on paper everything is really logical and well-reasoned, but in practice such tests turn out to be rather slow.

This article is aimed rather at beginners in programming, but maybe someone can be inspired by one of the approaches described below.

I think everyone knows the principles of good tests:

, .. (, HTTP- )
, ..
, .. CI

, : pipeline 3000 !

, , , .. . .

API :

( )
( )
HTTP-

, , . HTTP API, API.

( , ) , . , , : Redis/RabbitMQ HTTP , .

, DI- .

{
  "method": "patch",
  "uri": "/v2/project/17558/admin/items/physical_good/sku/not_existing_sku",
  "headers": {
    "Authorization": "Basic MTc1NTg6MTIzNDVxd2VydA=="
  },
  "data": {
    "name": {
      "en-US": "Updated name",
      "ru-RU": " "
    }
  }
}

{
  "status": 404,
  "data": {
    "errorCode": 4001,
    "errorMessage": "[0401-4001]: Can not find item with urlSku = not_existing_sku and project_id = 17558",
    "statusCode": 404,
    "transactionId": "x-x-x-x-transactionId-mock-x-x-x"
  }
}

<?php declare(strict_types=1);

namespace Tests\Functional\Controller\Version2\PhysicalGood\AdminPhysicalGoodPatchController;

use Tests\Functional\Controller\ControllerTestCase;

class AdminPhysicalGoodPatchControllerTest extends ControllerTestCase
{
    public function dataTestMethod(): array
    {
              return [
                // Negative cases
                'Patch -- item doesn\'t exist' => [
                        '001_patch_not_exist'
                ],
            ];
    }
}

TestFolder
├── Fixtures
│   └── store
│   │   └── item.yml
├── Request
│   └── 001_patch_not_exist.json
├── Response
│   └── 001_patch_not_exist.json
│   Tables
│   └── 001_patch_not_exist
│       └── store
│           └── item.yml
└── AdminPhysicalGoodPatchControllerTest.php

, . json yml ( ), ( ).

...

, , , , .

1.

— , .

( , ), . , , .

— .

— , ? .. ?

, 1 , , , , ! .

2.

, . , ( ).

667 . . , ?

, , CI-.

#!/usr/bin/env bash

if [[ ! -f "dump-cache.sql" ]]; then
    echo 'Generating dump'
    #     
    migrations_dir="./migrations" sh ./scripts/helpers/fetch_migrations.sh
    #    
    migrations_dir="./migrations" host="percona" sh ./scripts/helpers/migrate.sh

    #        (store, delivery)
    mysqldump --host=percona --user=root --password=root \
      --databases store delivery \
      --single-transaction \
      --no-data --routines > dump.sql

    cp dump.sql dump-cache.sql
else
    echo 'Extracting dump from cache'
    cp dump-cache.sql dump.sql
fi

, CI . - , git- .

CI-job (gitlab)

build migrations:
  stage: build
  image: php72:1.4
  services:
    - name: percona:5.7
  cache:
    key:
      files:
        - scripts/helpers/fetch_migrations.sh
    paths:
      - dump-cache.sql
  script:
    - bash ./scripts/ci/prepare_ci_db.sh
  artifacts:
    name: "$CI_PROJECT_NAME-$CI_COMMIT_REF_NAME"
    paths:
      - dump.sql
    when: on_success
    expire_in: 30min

3.

. , , . :

19 ( 27 ) 10 ( ): 10 18 .

, . , DI-.
AUTO INCREAMENT , TRUNCATE. , .

public static function setUpBeforeClass(): void
{
        parent::setUpBeforeClass();
        foreach (self::$onSetUpCommandArray as $command) {
            self::getClient()->$command(self::getFixtures());
        }
}

...

/**
 * @dataProvider dataTestMethod
 */
public function testMethod(string $caseName): void
{
        /** @var Connection $connection */
        $connection = self::$app->getContainer()->get('doctrine.dbal.prodConnection');
        $connection->beginTransaction();
        
        $this->traitTestMethod($caseName);
        $this->assertTables(\glob($this->getCurrentDirectory() . '/Tables/' . $caseName . '/**/*.yml'));
        
        $connection->rollBack();
}

4.

API , , .. . / , , ( , ).

, , . , - , .

dbunit, . , .

public function tearDown(): void
{
        parent::tearDown();
        //       DB-  
        //        
        self::$onSetUpCommandArray = [];
}

public static function tearDownAfterClass(): void
{
        parent::tearDownAfterClass();
        self::$onSetUpCommandArray = [
            Client::COMMAND_TRUNCATE,
            Client::COMMAND_INSERT
        ];
}

5.

— , . , . , .

pipeline’, .

pipeline’ ( testsuite phpunit). .

<testsuite name="functional-v2">
        <directory>./../../tests/Functional/Controller/Version2</directory>
</testsuite>

functional-v2:
  extends: .template_test
  services:
    - name: percona:5.7
  script:
    - sh ./scripts/ci/migrations_dump_load.sh
    - ./vendor/phpunit/phpunit/phpunit --testsuite functional-v2 --configuration config/test/phpunit.ci.v2.xml --verbose

, , , paratest. .

, .. . , ( ), , .. - .

CI —
, -
, - ( , ) . CI, .

...

6.

, . . , , . - bootstrap , .

( ). , , , .. DI- (, - , ..).

, , . , .

interface StateResetInterface
{
    public function resetState();
}

$container = self::$app->getContainer();
foreach ($container->getKnownEntryNames() as $dependency) {
        $service = $container->get($dependency);
        if ($service instanceof StateResetInterface) {
                $service->resetState();
        }
}

Writing tests is always the same compromise as writing the actual application itself. It is necessary to proceed from the fact that for you is more priority, and what can be donated. We are often told one-sidedly about “ideal” tests, which in reality can be difficult to implement, work is slow, or support is labor-intensive.

After all the optimizations, the run time in CI for functional tests has decreased to 12-15 minutes. Of course, I doubt that the techniques described above will be useful in their original form, but I hope they inspired and gave me my own ideas!

What approach to writing tests do you use? Do you need to optimize them, and what methods did you use?

So how can you not suffer from functional tests?

1.

2.

3.

4.

5.

6.

More articles: