Unstable tests are one of the main problems in automated testing (Part 2)

This is the continuation of a series of articles on unstable tests.

The first article ( original / translation in Habré ) talked about 4 components in which unstable tests can occur.

In this article, we will give tips on how to avoid unstable tests in each of the 4 components.

Components

So there are 4 components in which unstable tests can occur:

• The tests themselves;

  
  
  
  
  

• A framework for running tests;

  
  
  
  
  

• Services and libraries on which the system under test and the test framework depend;

  
  
  
  
  

• The operating system and device with which the autotesting framework interacts.

  
  
  
  
  

This is shown in Figure 1.

The reasons, options for localizing the problem and options for resolving instability are discussed below.

The tests themselves

The tests themselves may be unstable.

The reasons can be in test data, autotest scripts, autotest preconditions, and initial states of other dependencies.

Table 1 - Reasons, options for localizing the problem and options for solving instability in the tests themselves.

2 — , ,

,

( ) .

, .

.

3 — , ,

, .

4 — , ,

, . , , .

• ? (/ )

  
  
  
  
  

• Google ()

  
  
  
  
  

• Selenium ! ()

  
  
  
  
  

• ()

  
  
  
  
  

• — (/ )