Clever Geek Handbook

Fast-Unit or declarative approach to unit tests



Hello! My name is Yuri Skvortsov , our team is engaged in automated testing at Rosbank. One of our tasks is to develop tools to automate functional testing.



In this article, I want to talk about a solution that was conceived as a small auxiliary utility for solving other problems, but in the end turned into an independent tool. We are talking about the Fast-Unit framework, which allows you to write unit tests in a declarative style and turns the development of unit tests into a component constructor. The project was developed primarily for testing our main product - Tladianta - a unified BDD framework for testing 4 platforms: Desktop, Web, Mobile and Rest.



To begin with, testing an automation framework is not a common task. However, in this case, it was not a part of a testing project, but an independent product, so we quickly realized the need for units.



At the first stage, we tried to use ready-made tools such as assertJ and Mockito, but quickly ran into some of the technical features of our project:



  • Tladianta already uses JUnit4 as a dependency, which makes it difficult to use a different version of JUnit and makes it harder to work with Before;
  • Tladianta contains components for working with different platforms, it has many entities that are “extremely close” in terms of functionality, but with different hierarchies and different behaviors;
  • «» ( ) ;
  • , , , , ;
  • - (, Appium , , , );
  • , : Mockito .




Initially, when we just learned to replace the driver, create fake Selenium elements and wrote the basic architecture for the test harness, the tests looked like this: 



@Test
public void checkOpenHint() {
    ElementManager.getInstance().register(xpath,ElementManager.Condition.VISIBLE,
ElementManager.Condition.DISABLED);
    new HintStepDefs().open(("");
    assertTrue(TestResults.getInstance().isSuccessful("Open"));
    assertTrue(TestResults.getInstance().isSuccessful("Click"));
}

@Test
public void checkCloseHint() {
    ElementManager.getInstance().register(xpath);
    new HintStepDefs().close("");
    assertTrue(TestResults.getInstance().isSuccessful("Close"));
    assertTrue(TestResults.getInstance().isSuccessful("Click"));
}


Or even like this: 



@Test
public void fillFieldsTestOld() {
    ElementManager.getInstance().register(ElementManager.Type.CHECK_BOX,"//check-box","",
ElementManager.Condition.NOT_SELECTED);
        ElementManager.getInstance().register(ElementManager.Type.INPUT,"//input","");
        ElementManager.getInstance().register(ElementManager.Type.RADIO_GROUP, 
"//radio-group","");
        DataTable dataTable = new Cucumber.DataTableBuilder()
                .withRow("", "true")
                .withRow("", "not selected element")
                .withRow(" ", "text")
                .build();
        new HtmlCommonSteps().fillFields(dataTable);
        assertEquals(TestResults.getInstance().getTestResult("set"), 
ElementProvider.getInstance().provide("//check-box").force().getAttribute("test-id"));
        assertEqualsTestResults.getInstance().getTestResult("sendKeys"), 
ElementProvider.getInstance().provide("//input").force().getAttribute("test-id"));
        assertEquals(TestResults.getInstance().getTestResult("selectByValue"), 
ElementProvider.getInstance().provide("//radio-group").force().getAttribute("test-id"));
    }


It is not difficult to find what is being tested in the code above, as well as to understand the checks, but there is a huge amount of code. If you include software for checking and describing errors, then it becomes very difficult to read. And we are just trying to check that the method was called on the desired object, while the real logic of the checks is extremely primitive. In order to write such a test, you need to know about ElementManager, ElementProvider, TestResults, TickingFuture (a wrapper to implement a change in the state of an element during a given time). These components were different in different projects, we had no time to synchronize changes.



Another challenge was the development of some standard. Our team has the advantage of automators, many of us do not have enough experience in developing unit tests, and although, at first glance, it is simple, reading each other's code is quite laborious. We tried to liquidate technical debt quickly enough, and when hundreds of such tests appeared, it became difficult to maintain. In addition, the code turned out to be overloaded with configurations, real checks were lost, and thick straps led to the fact that instead of testing the functionality of the framework, our own straps were tested.



And when we tried to transfer the developments from one module to another, it became clear that we needed to bring out the general functionality. At that moment, the idea was born not only to create a library with best practices, but also to create a single unit development process within this tool.



Changing philosophy



If you look at the code as a whole, you can see that many blocks of code are repeated “without meaning”. We test methods, but we use constructors all the time (to avoid the possibility of some error being cached). The first transformation - we have moved the checks and generation of tested instances into annotations. 



@IExpectTestResult(errDesc = "    set", value = "set",
expected = "//check-box", convertedBy = Converters.XpathToIdConverter.class, soft = true)
@IExpectTestResult(errDesc = "    sendKeys", value = "sendKeys", 
expected = "//input", convertedBy = Converters.XpathToIdConverter.class, soft = true)
@IExpectTestResult(errDesc = "    selectByValue", value = "selectByValue",
expected = "//radio-group", convertedBy = Converters.XpathToIdConverter.class, soft = true)
@Test
public void fillFieldsTestOld() {
    ElementManager.getInstance().register(ElementManager.Type.CHECK_BOX, "//check-box", "",
ElementManager.Condition.NOT_SELECTED);
    ElementManager.getInstance().register(ElementManager.Type.INPUT, "//input", "");
    ElementManager.getInstance().register(ElementManager.Type.RADIO_GROUP, 
"//radio-group", "");
    DataTable dataTable = new Cucumber.DataTableBuilder()
            .withRow("", "true")
            .withRow("", "not selected element")
            .withRow(" ", "text")
            .build();
    runTest("fillFields", dataTable);
}


What changed?



  • The checks have been delegated to a separate component. Now you don't need to know about how items are stored, test results.
  • : errDesc , .
  • , , , – runTest, , .
  • .
  • - , .


We liked this form of notation, and we decided to simplify another complex component in the same way - the generation of elements. Most of our tests are devoted to ready-made steps, and we must be sure that they work correctly, however, for such checks, it is necessary to completely “launch” the fake application and fill it with elements (recall that we are talking about Web, Desktop and Mobile, the tools for which differ quite strongly). 



@IGenerateElement(type = ElementManager.Type.CHECK_BOX)
@IGenerateElement(type = ElementManager.Type.RADIO_GROUP)
@IGenerateElement(type = ElementManager.Type.INPUT)
@Test
@IExpectTestResult(errDesc = "    set", value = "set", 
expected = "//check-box", convertedBy = Converters.XpathToIdConverter.class, soft = true)
@IExpectTestResult(errDesc = "    sendKeys", value = "sendKeys", 
expected = "//input", convertedBy = Converters.XpathToIdConverter.class, soft = true)
@IExpectTestResult(errDesc = "    selectByValue", value = "selectByValue",
expected = "//radio-group", convertedBy = Converters.XpathToIdConverter.class, soft = true)
public void fillFieldsTest() {
    DataTable dataTable = new Cucumber.DataTableBuilder()
            .withRow("", "true")
            .withRow("", "not selected element")
            .withRow(" ", "text")
            .build();
    runTest("fillFields", dataTable);
}


Now the test code has become completely template, the parameters are clearly visible, and all the logic is moved to the template components. The default properties made it possible to remove empty lines and gave ample opportunities for overloading. This code is almost in line with the BDD approach, precondition, validation, action. In addition, all the bindings have taken off from the logic of the tests, you no longer need to know about managers, storages of test results, the code is simple and easy to read. Since annotations in Java are almost not customizable, we introduced a mechanism for converters that can receive the final result from a string. This code not only checks the fact of calling the method, but also the id of the element that executed it. Almost all the tests that existed at that time (more than 200 units), we quickly transferred to this logic, bringing them to a single template. Tests have become what they should be - documentation,not code, so we came to declarativeness. It is this approach that formed the basis of Fast-Unit - declarativeness, self-documenting of tests and isolation of the tested functionality, the test is completely devoted to checking one tested method.



We continue to develop



Now it was necessary to add the ability to create such components independently within the framework of projects, add the ability to control the sequence of their operation. To do this, we developed the concept of phases: unlike Junit, all these phases exist independently within each test and are executed at the time the test is run. As a default implementation, we have laid down the following life cycle:



  • Package-generate - processing annotations related to package-info. Components associated with these provide configuration downloads and general harness preparation.
  • Class-generate - processing annotations associated with a test class. Configuration actions related to the framework are performed here, adapting it to the prepared binding.
  • Generate - processing annotations associated with the test method itself (entry point).
  • Test - preparing the instance and executing the tested method.
  • Assert - performing checks.


The annotations to be processed are described something like this: 



@Target(ElementType.PACKAGE) //  
@IPhase(value = "package-generate", processingClass = IStabDriver.StabDriverProcessor.class,
priority = 1) //    (      )
public @interface IStabDriver {

    Class<? extends WebDriver> value(); //   ,     

    class StabDriverProcessor implements PhaseProcessor<IStabDriver> { // 
        @Override
        public void process(IStabDriver iStabDriver) {
            //  
        }
    }
}


The Fast-Unit feature is that the life cycle can be overridden for any class - it is described by the ITestClass annotation, which is designed to indicate the class and phases under test. The list of phases is specified simply as a string array, allowing for composition change and phase sequence. The methods that handle phases are also found using annotations, so it is possible to create the necessary handler in your class and mark it (plus, overriding within the class is available). A big plus was that this separation allowed us to divide the test into layers: if an error in the finished test occurred during the package-generate or generate phase, then the test harness is damaged. If class-generate - there are problems in the configuration mechanisms of the framework. If within the framework of test there is an error in the tested functionality.The test phase can technically throw errors both in the binding and in the functionality under test, so we wrapped possible binding errors in a special type - InnerException.



Each phase is isolated, i.e. does not depend on and does not interact directly with other phases, the only thing that is passed between phases are errors (most phases will be skipped if an error occurred in the previous ones, but this is not necessary, for example, the assert phase will work anyway).



Here, probably, the question has already arisen, where do the testing instances come from. If the constructor is empty, it's obvious: using the Reflection API, you simply create an instance of the class under test. But how can you pass parameters in this construct or configure the instance after the constructor has fired? What if the object is being built by the builder or in general it is about testing statics? For this, the mechanism of providers has been developed, which hide the complexity of the constructor.



Default parameterization: 



@IProvideInstance
CheckBox generateCheckBox() {
    return new CheckBox((MobileElement) ElementProvider.getInstance().provide("//check-box")
.get());
}


No parameters - no problem (we are testing the CheckBox class and registering a method that will create instances for us). Since the default provider is overridden here, there is no need to add anything in the tests themselves, they will automatically use this method as a source. This example clearly illustrates the Fast-Unit logic - we hide the complex and unnecessary. From a test point of view, it doesn't matter at all how and where the mobile element wrapped with the CheckBox class comes from. All that matters to us is that there is some CheckBox object that meets the specified requirements.



Automatic argument injection: let's assume we have a constructor like this: 



public Mask(String dataFormat, String fieldFormat) {
    this.dataFormat = dataFormat;
    this.fieldFormat = fieldFormat;
}


Then a test of this class using argument injection will look like this: 



Object[] dataMask={"_:2_:2_:4","_:2/_:2/_:4"};

@ITestInstance(argSource = "dataMask")
@Test
@IExpectTestResult(errDesc = "  ", value = FAST_RESULT,
expected = "12/10/2012")
public void convert() {
    runTest("convert","12102012");
}


Named providers



Finally, if we need multiple providers, we use name binding, not only hiding the complexity of the constructor, but also showing its real meaning. The same problem can be solved like this: 



@IProvideInstance("")
Mask createDataMask(){
    return new Mask("_:2_:2_:4","_:2/_:2/_:4");
} 

@ITestInstance("")
@Test
@IExpectTestResult(errDesc = "  ", value = FAST_RESULT,
expected = "12/10/2012")
public void convert() {
    runTest("convert","12102012");
}


IProvideInstance and ITestInstance are associated annotations that allow you to tell the method where to get the instance under test (for statics, it simply returns null, since this instance is ultimately used through the Reflection API). The provider approach gives much more information about what actually happens in the test, replacing the call to the constructor with some parameters with text describing the preconditions, so if the constructor suddenly changes, we will only have to correct the provider, but the test will remain unchanged until the actual functionality changes. If, during the review, you see several providers, you will notice the difference between them, and therefore the peculiarities of the behavior of the tested method. Even without knowing the framework at all, but only knowing the principles of Fast-Unit operation,the developer will be able to read the test code and understand what the tested method does.



Conclusions and results



Our approach turned out to have many advantages:



  • Easy test portability.
  • Hiding the complexity of the bindings, the ability to refactor them without breaking tests.
  • Backward compatibility guaranteed - changes to method names will be recorded as errors.
  • The tests have turned into fairly detailed documentation for each method.
  • The quality of inspections has improved significantly.
  • Unit test development has become a pipeline process, and the speed of development and review has increased significantly.
  • Stability of the developed tests - although the framework and the Fast-Unit itself are actively developing, there is no degradation of tests


Despite the apparent complexity, we were able to quickly implement this tool. Now most of the units are written in it, and they have already confirmed their reliability with a fairly complex and voluminous migration, they were able to identify rather complex defects (for example, in waiting for elements and text checks). We were able to quickly eliminate technical debt and establish effective work with units, making them an integral part of the development. Now we are considering options for a more active implementation of this tool in other projects outside our team.



Current problems and plans:



  • , . , ( - ).
  • .
  • .
  • , -.
  • Fast-Unit junit4, junit5 testng


More articles:


All Articles