Associated with this approach are cucumber, robot framework, behave, and others, which separate the execution scripts and the implementation of each construct. This separation helps to write readable scripts, but it is time consuming and therefore can be impractical when writing an implementation.
Let's take a look at how you can simplify your work with BDD by using the right tools - for example, the Spock framework, which combines the beauty, convenience of BDD principles and the features of jUnit.
Spock framework
Spock is a testing and specification framework for Java and Groovy applications. Using the JUnit platform as the basis, this framework is compatible with all popular IDEs (in particular, IntelliJ IDEA), various build tools (Ant, Gradle, Maven) and continuous integration (CI) servers.
How to write the developers of the framework, Spock «inspired the JUnit , the RSpec , jMock , Mockito , Groovy's , Scala's , Vulcans and other exciting forms of life."
In this article, we'll take a look at the latest available version, Spock Framework 2.0. Its features: the ability to use JUnit5, Java 8+, groovy 2.5 (there is also an assembly with version 3.0). Spock is licensed under Apache 2.0 and has a responsive user community. The framework developers continue to refine and develop Spock, which already includes many extensions that allow you to fine-tune your test run. For example, one of the most interesting announced areas of improvement is the addition of parallel test execution.
Groovy
Groovy is an object-oriented programming language developed for the Java platform as an add-on with Python, Ruby, and Smalltalk capabilities. Groovy uses a Java-like syntax with dynamic compilation to JVM bytecode and works directly with other Java code and libraries. The language can be used in any Java project or as a scripting language.
The features of groovy include: both static and dynamic typing; built-in syntax for lists, arrays and regular expressions; overloading operations. However, closures in Groovy appeared long before Java.
Groovy is well suited for rapid test development, where you can use python-like syntactic sugar without worrying about typing objects.
Features of Spock Framework
One of the key features of the framework is that the developer has the ability to write specifications with the expected system characteristics using the principles of the BDD approach. This approach makes it possible to compose business-oriented functional tests for software products with high subject and organizational complexity.
The spec is a groovy class that extends spock.lang.Specification
class MyFirstSpecification extends Specification {
// fields
// fixture methods
// feature methods
// helper methods
}The BOM can contain various auxiliary fields that are triggered for each BOM class.
Using the @Shared annotation, you can give access to the field to classes inherited from the specification.
abstract class PagesBaseSpec extends Specification {
@Shared
protected WebDriver driver
def setup() {
this.driver = DriverFactory.createDriver()
driver.get("www.anywebservice.ru")
}
void cleanup() {
driver.quit()
}
}BOM class customization methods:
def setupSpec() {} // feature
def setup() {} // feature
def cleanup() {} // feature
def cleanupSpec() {} // feature The following table shows which keywords and methods in the Spock framework have JUnit counterparts.
Dough blocks
In Spock Framework, each test phase is separated into a separate block of code (see the documentation for an example ).
A block of code starts with a label and ends with the start of the next block of code or the end of a test.
The given block is responsible for setting the initial test conditions. When , then
blocks are always used together. The when block contains the stimulant, stimulus of the system, and the then block contains the system's response. In cases where it is possible to shorten the when-then clause to a single expression, you can use a single expect block
... The following examples will be used from the official Spock framework documentation:
when:
def x = Math.max(1, 2)
then:
x == 2
or one expression
expect:
Math.max(1, 2) == 2The cleanup block is used to free up resources before the next iteration of the test.
given:
def file = new File("/some/path")
file.createNewFile()
// ...
cleanup:
file.delete()
The where clause is used to transfer data for testing (Data Driven Testing).
def "computing the maximum of two numbers"() {
expect:
Math.max(a, b) == c
where:
a << [5, 3]
b << [1, 9]
c << [5, 9]
}The types of input data transfer will be discussed below.
Sample test implementation on Spock Framework
Next, we will consider approaches to the implementation of testing the web page for user authorization in the system using selenium.
import helpers.DriverFactory
import org.openqa.selenium.WebDriver
import spock.lang.Shared
import spock.lang.Specification
abstract class PagesBaseSpec extends Specification {
@Shared
protected WebDriver driver
def setup() {
this.driver = DriverFactory.createDriver()
driver.get("www.anywebservice.ru")
}
void cleanup() {
driver.quit()
}
}
Here we see the base class of the page specification. At the beginning of the class, we see the import of the required classes. The following is the shared annotation , which allows derived classes to access the web driver. In the setup () block, we see the code for initializing the web driver and opening the web page. The cleanup () block contains the code for terminating the web driver.
Next, let's move on to an overview of the user login page specification.
import pages.LoginPage
import spock.lang.Issue
class LoginPageTest extends PagesBaseSpec {
@Issue("QAA-1")
def "QAA-1: Authorization with correct login and password"() {
given: "Login page"
def loginPage = new LoginPage(driver)
and: "Correct login and password"
def adminLogin = "adminLogin"
def adminPassword = "adminPassword"
when: "Log in with correct login and password"
loginPage.login(adminLogin, adminPassword)
then: "Authorized and moved to main page"
driver.currentUrl == "www.anywebservice.ru/main"
}
}
The authorization page specification inherits from the base page specification. The Issue annotation specifies the ID of the test in an external tracking system (such as Jira). In the next line we see the name of the test, which by convention is set by string literals, which allows using any characters in the name of the test (including Russian ones). In the given block , the page object of the authorization page class is initialized, and the correct login and password for authorization in the system are obtained. In the when block , the authorization action is performed. The then block is used to check the expected action, namely, successful authorization and redirection to the main page of the system.
On the example of this specification, we see the most significant plus of using the BDD paradigm in spock - the system specification is at the same time its documentation. Each test describes a certain behavior, each step in the test has its own description, which is understandable not only for developers, but also for customers. Descriptions of blocks can be presented not only in the source code of the test, but also in diagnostic messages or reports on the test operation.
The framework provides the ability to transfer various logins and passwords for testing (parameterize the test).
Data Driven Testing in Spock Framework
Data Driven Testing = table-driven testing = parameterized testing
To test a scenario with multiple parameters, you can use various options for passing them.
Data Tables
Let's look at a few examples from the official framework documentation.
class MathSpec extends Specification {
def "maximum of two numbers"() {
expect:
Math.max(a, b) == c
where:
a | b | c
1 | 3 | 3
7 | 4 | 7
0 | 0 | 0
}
}
Each row in the table is a separate test iteration. Also, the table can be represented by one column.
where:
a | _
1 | _
7 | _
0 | __ Is a stub object of the BOM class.
For a better visual perception of the parameters, you can rewrite the example above in the following form:
def "maximum of two numbers"() {
expect:
Math.max(a, b) == c
where:
a | b || c
1 | 3 || 3
7 | 4 || 7
0 | 0 || 0
}
Now we can see that a , b are inputs and c is the expected value.
Data pipes
In some cases, using the design table will be very cumbersome. In such cases, you can use the following type of parameter passing:
...
where:
a << [1, 7, 0]
b << [3, 4, 0]
c << [3, 7, 0]Here, the left shift << is an overloaded groovy operator that now serves as adding items to the list.
For each test iteration, the following data from the list will be requested for each variable:
1 iteration: a = 1, b = 3, c = 3;
2nd iteration: a = 7, b = 4, c = 7;
3 iteration: a = 0, b = 0, c = 0.
Moreover, the input data can not only be transmitted explicitly, but also be requested, if necessary, from various sources. For example, from the database:
@Shared sql = Sql.newInstance("jdbc:h2:mem:", "org.h2.Driver")
def "maximum of two numbers"() {
expect:
Math.max(a, b) == c
where:
[a, b, c] << sql.rows("select a, b, c from maxdata")
}Data Variable Assignment
...
where:
a = 3
b = Math.random() * 100
c = a > b ? a : bHere we see the dynamically computed variable c in the test data.
Combination of different types of parameter transfer
...
where:
a | _
3 | _
7 | _
0 | _
b << [5, 0, 0]
c = a > b ? a : bNobody forbids you to use several types of transfer at once, if necessary.
An example of implementing a parameterized test on the Spock Framework
@Issue("QAA-1-parametrized")
def "QAA-1-parametrized: Authorization with correct login and password"() {
given: "Login page"
def loginPage = new LoginPage(driver)
when: "Log in with correct login and password"
loginPage.login(login, password)
then: "Authorized and moved to main page"
driver.currentUrl =="www.anywebservice.ru/main"
where: "Check for different logins and passwords"
login | password
"adminLogin" | "adminPassword"
"moderatorLogin" | "moderatorPassword"
"userLogin" | "userPassword"
}Here we see the already familiar where block, in which the keys of the parameters (logins and passwords) are set, which are stored in the configuration file.
Due to the peculiarities of the used implementation of the specification, the web driver configuration cycle and its closing (and hence closing the browser) will be performed for each parameter, which will negatively affect the test execution time. We suggest finalizing the specification and improving the test runtime.
An example of the implementation of a parameterized test with a modified specification
Before revision
abstract class PagesBaseSpec extends Specification {
@Shared
protected WebDriver driver
def setup() {
this.driver = DriverFactory.createDriver()
driver.get("www.anywebservice.ru")
}
void cleanup() {
driver.quit()
}
}After revision
import helpers.DriverFactory
import org.openqa.selenium.WebDriver
import spock.lang.Shared
import spock.lang.Specification
abstract class PagesNoRestartBaseSpec extends Specification {
@Shared
protected WebDriver driver
def setupSpec() {
this.driver = DriverFactory.createDriver()
}
def setup() {
this.driver.get("www.anywebservice.ru")
}
def cleanup() {
this.driver.get("www.anywebservice.ru/logout")
this.driver.manage().deleteAllCookies();
}
void cleanupSpec() {
this.driver.quit()
}
}In the updated specification, we see that the procedure for creating a web driver will be performed only when setting up the specification class, and closing the browser only after the tests from the specification have finished running. In the setup () method, we see the same code for getting the web address of the service and opening it in a browser, and in the cleanup () method, we go to www.anywebservice.ru/logout to finish working with the service for the current user and delete cookies (for testing the current web service, this procedure is sufficient to simulate a "unique" launch). The test code itself has not changed.
As a result, with the help of simple improvements, we got at least a twofold decrease in the autotest operation time, compared to the initial implementation.
Comparison of tests for testNG, pytest, pytest-bdd
First, we will look at the implementation of a test on the testNG test framework in the Java programming language, which, like Spock Framework, is inspired by the jUnit framework and supports data-driven testing.
package javaTests;
import org.testng.Assert;
import org.testng.annotations.*;
import pages.LoginPage;
public class LoginPageTest extends BaseTest {
@BeforeClass
public final void setup() {
createDriver();
driver.get("www.anywebservice.ru");
}
@DataProvider(name = "userParameters")
public final Object[][] getUserData(){
return new Object[][] {
{"adminLogin", "adminPassword"},
{"moderatorLogin", "moderatorPassword"},
{"userLogin", "userPassword"}
};
}
@Test(description = "QAA-1-1: Authorization with correct login and password",
dataProvider = "userParameters")
public final void authorizationWithCorrectLoginAndPassword(String login, String password){
//Login page
LoginPage loginPage = new LoginPage(driver);
//Log in with correct login and password
loginPage.login(login, password);
//Authorized and moved to main page
Assert.assertEquals("www.anywebservice.ru/main", driver.getCurrentUrl());
}
@AfterMethod
public final void cleanup() {
driver.get("www.anywebservice.ru/logout");
driver.manage().deleteAllCookies();
}
@AfterClass
public final void tearDown() {
driver.quit();
}
}Here we can see the test class with all the necessary setup (), cleanup () methods, as well as parameterization of the test in the form of an additional getUserData () method with the @DataProvider annotation, which looks a bit cumbersome after what we examined in the test using Spock Framework. Also, to understand what is happening in the test, comments similar to the description of the steps were left.
It should be noted that testNG, unlike Spock Framework, supports parallel test execution.
Next, let's move on to a test using the pytest test framework in the Python programming language.
import pytest
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support.wait import WebDriverWait
from PageObjects.LoginPage import LoginPage
class TestLogin(object):
@pytest.mark.parametrize("login,password", [
pytest.param(("adminLogin", "adminPassword"), id='admin'),
pytest.param(("moderatorLogin", "moderatorPassword"), id='moderator'),
pytest.param(("userLogin", "userPassword"), id='user')
])
def test_authorization_with_correct_login_and_password(self, login, password, driver, test_cleanup):
# Login page
login_page = LoginPage(driver)
# Log in with correct login and password
login_page.login(login, password)
# Authorized and moved to main page
assert expected_conditions.url_to_be("www.anywebservice.ru/main")
@pytest.fixture()
def test_cleanup(self, driver):
yield "test"
driver.get("www.anywebservice.ru/logout")
driver.delete_all_cookies()
Here we also see support for data-driven testing as a separate construct, similar to @DataProvider in testNG. The method for configuring the web driver is "hidden" in the driver fixture. Thanks to dynamic typing and pytest fixtures, the code for this test looks cleaner than Java.
Next, let's move on to an overview of the test code using the pytest-bdd plugin, which allows you to write tests in the form of Gherkin feature files (pure BDD approach).
login.feature
Feature: Login page
A authorization
Scenario: Authorizations with different users
Given Login page
When Log in with correct login and password
Then Authorized and moved to main page
test_login.py
import pytest
from pytest_bdd import scenario, given, when, then
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support.wait import WebDriverWait
from PageObjects.LoginPage import LoginPage
@pytest.mark.parametrize("login,password", [
pytest.param(("adminLogin", "adminPassword"), id='admin'),
pytest.param(("moderatorLogin", "moderatorPassword"), id='moderator'),
pytest.param(("userLogin", "userPassword"), id='user')
])
@scenario('login.feature', 'Authorizations with different users')
def test_login(login, password):
pass
@given('Login page')
def login_page(driver):
return LoginPage(driver)
@when('Log in with correct login and password')
def login_with_correct_login_and_password(login_page, login, password):
login_page_object = login_page
login_page_object.login(login, password)
@then('Authorized and moved to main page')
def authorized_and_moved_to_main_page(driver, login):
assert expected_conditions.url_to_be("www.anywebservice.ru/main")One of the advantages is that it is still a pytest framework, which has many plugins for various situations, including for running tests in parallel. The downside is the pure BDD approach itself, which will constantly limit the developer with its own characteristics. Spock Framework makes it possible to write more concise and easy-to-design code compared to the PyTest + pytest-bdd bundle.
Conclusion
In this article, we looked at how to simplify working with BDD using the Spock framework. Summing up, let's briefly highlight the main, in our opinion, the pros and cons of Spock in comparison with some other common test frameworks.
Pros:
- Using BDD principles instead of a pure BDD approach gives you more flexibility when writing tests.
- The written test specification is also the documentation of the system.
- .
- groovy ( , , closures ).
:
- groovy. , , IDE , . Intellij IDEA, , , , .
- groovy JVM -. , groovy, , . java, groovy .
- The set of extensions is not as extensive as that of testNG, for example. As a result, there is no parallel run of tests. There are plans to add this functionality, but the timing of their implementation is unknown.
Ultimately, the Spock Framework undoubtedly deserves attention as it is suitable for solving the complex common problem of automating business scenarios for software products with high subject and organizational complexity.
What else can you read: