The agile methodology introduces its own rules in the software testing process, and each team independently develops, tests and deploys their services. However, in complex systems, when the coordinated work of dozens of services is required to support the business process, there is a need for an additional level of testing. That is why we at M.Video - Eldorado have created a special department for cross-system testing. Read about how it was formed, what cross-system testers do and how testing processes are organized here.
In a previous post ( tyk ), we already talked about why cross-system testing is important for our company. Today we will focus on how our specialists work and talk about the methodology of this process.
Four pilot projects
To form a typical approach to cross-system testing, we needed to accumulate information and track interactions between systems. It was not easy to approach this topic, so we decided not to be alone with her and turned to the contractor.
Together we outlined our "pain" and came to the conclusion that our company should have specialized testers who are engaged in cross-system testing, as well as test managers who oversee such projects. Their work is complemented by cross-system analysts who, even at the design stage, monitor the interaction between systems and create a knowledge base about the relationships between different components of business processes.
To establish this work, it was decided to launch a number of pilot projects. The first was a project testing promotions that our colleagues tested on a regular basis. This pilot allowed us to start building the knowledge base of the department (137 pages of guidelines were created in Confluence), and employees began to work with standard instructions - which means they became more interchangeable.
The second pilot project was organized entirely on the basis and role model of the new unit. We tested the process of generating digital product codes. The project called Digital Content 2.0 lasted for 5 months. It affected 15 systems, and 114 test cases were developed for its implementation. During this pilot, we came to the need for centralized management of test environments and project master data, and we created a team of specialists to support the work of all teams in the test environment.
The third pilot project of MarketPlace was carried out completely independently, without the participation of a contractor. Our team tested 15 systems, the joint work of which made it possible to sell goods from the Goods marketplace on the M.Video website. Another 209 test cases were developed, but most importantly, we created a common high-level template for the cross-system testing process that could be used on new projects.
During MarketPlace, we started running test cases in Jira, collecting reports and statistics in a convenient form.
In the course of this project, in the spring of 2020, the work of the entire department was transferred to a remote work schedule and it became obvious that additional materials were needed to train testers working at a distance from each other, and we began to form a base of training videos.
The fourth pilot, after which our work moved to a new quality, is testing the Eldorado mobile application. Testing under the name Eldo Mobile became a product test - that is, its customer was not the Projet Manager, but the Product Owner. And although we tested only 16 related systems during the three-month pilot stage, the new approach allowed us to plan cross-system testing for each new release of the mobile application.
The work of the cross-system testing division today
Today, the cross-system testing division acts as an internal QA service provider for product and process owners. We receive applications from different departments, and test managers draw up a schedule and distribute testing tasks. Scheduling takes place once a week - large and small tasks are distributed between test managers and testers, taking into account their workload and experience.
If a complex process is being tested, the tester can work on the same project all week. In other cases, the specialist manages to solve two or three tasks from different customers. Sometimes a tester works with two test managers at once, who distribute his time according to a dynamic digital schedule.
A pre-built methodology involves working with typical test cases. Thanks to this, a person can be easily switched from one task to another. In our work, this is often necessary, especially if something suddenly starts to "burn", and for other projects, the deadline allows you to slightly shift the deadline.
So that our department can interact with testers from product teams, we also develop clear regulations for the execution of test cases: criteria for starting and ending testing, time and order of tasks, a list of persons responsible for each stage, and so on.
In fact, today we are bringing cross-systems testing methodology to established agile teams. This approach allows us to avoid constant approvals and to conduct testing within a clear time frame.
Qualities of a cross-system tester
Even when all development is in the agile rhythm, cross-system testing remains inherently project-based, with all the ensuing needs for planning, reallocation of resources, both in each team and within the entire group.
The point is that the working environment for cross-system testers and test managers is constantly changing. For example, we recently received tasks for cross-system testing of a chatbot, as well as the process of signing contracts for non-commercial purchases. To complete these tasks, testers have to join a new area, understand the functionality and features of previously unfamiliar systems.
In such projects, we include a training phase for testers, which can range from one week to a month. At the same time, people are required to be able to quickly engage in new topics, communicate with analysts from different teams, and effectively manage their time.
However, even without this, there are quite a lot of uncertainties in the work of the unit. Test managers can start the day with one plan, and change it drastically by lunchtime. All customers have different deadlines and different priorities for tasks, any system can suddenly go offline and require a reallocation of resources.
Finally, the test environment can be overwhelmed. Therefore, test managers themselves must be strong PMs and can handle planning literally on the fly.
Product testing
After the completion of our pilot projects, a clear course for product testing was outlined in the company. I must say that this approach facilitates cross-functional testing, because some of the test tasks remain in the product team.
We determine in advance at what point in time and why the cross-functional testing team connects to the process and clearly allocate the resources of the testers.
In this sense, product testing is easier to plan because it goes on schedule, according to the waves of development. When we receive a new release, we ask the team what needs to be checked, adjust test cases and get down to work. At the same time, certain cross-system testers are assigned to each product.
The advantage of the centralized approach is that one tester can work with several projects or products at once. For example, a mobile app regularly takes 50% of the assigned specialists' time. They can spend the remaining 50% on testing other systems - in accordance with the tasks of test managers.
Results of the new division
We have not yet lived in the paradigm of cross-system testing for so long, but it is already clear that the timing of the tests has become shorter (sometimes several times), and the quality of testing is higher.
A team of cross-system testers is cheaper for the company than expanding with an additional tester for product teams, and thanks to the presence of a competence accumulation center, cross-system tests pass faster and faster with each new task. Moreover, we also developed a methodology for entering “unfamiliar projects” so that the launch of cross-functional testing in new directions was faster.
The new methodology has added transparency to the work of all teams and has improved QA. The continuous accumulation of expertise and cross-system analytics make it possible to more and more accurately predict what changes in some systems can affect the operation of other systems.
The knowledge base is growing, and today we can already take on such test cases that a few months ago we could not have performed without additional preparation. Therefore, the practice of cross-system testing is gradually becoming a natural part of the M.Video-Eldorado development ecosystem and the corporate culture of the company as a whole.
PS Our team is expanding. We need talented testers . If you are, welcome aboard!