My name is Sasha Zubarchuk. I came to Solid three years ago as Junior Manual QA. Since that time, I became an automation engineer, graduated students from the first QA School we worked with, and moved to the position of Product Manager.
In this article I will talk about the features of testing payment systems and share the experience of our team: what we are testing, how, what challenges we face and how we respond to them. This experience will be useful to those who build the testing process on the project - both QA engineers and Product / Project managers.
I will not delve into the technique and specific tools, but I will talk about the main idea: how to make the process of testing technically complex services as transparent and calm as possible.
What it consists of and how it works
Solid is a gateway that allows businesses around the world to accept online customer payments in a variety of ways, from bank cards and e-wallets to PayPal and Alipay.
There are four main players in this whole story:
- user (he is also a buyer);
- merchant - any business that sells its goods / services on the Internet and wants to accept money for them;
- payment gateway - Solid;
- bank (acquirer) - a financial institution that performs transactions with real money.
In simplified
terms, it looks like this: Simply put, we help merchants to monetize and make money.
You might have a question: why do merchants choose Solid, if they can work directly with banks? It's simple: 1) there are many banks, and each of them has its own specifics - by combining different banks from different countries into a single infrastructure, we achieve maximum conversions and revenue; 2) we have vast expertise in payment logics and antifraud; 3) in addition to bank payments, we accept all alternative payment methods popular in a particular region, and 4) we are cheaper.
Let's start with the first point - what are these features of making payments in different countries? For example, in the USA, oddly enough, you will not be able to make a payment as easily as in Ukraine - using only card details. There banks are required to verify additional payer details such as billing address and postal code.
There are many similar nuances. As well as alternatives to card payments. In the Netherlands and Germany, for example, people like to use local internet wallets. The most popular payment methods in China are Alipay, WeChat Pay, and UnionPay. And in Nigeria , to pay for goods on the Internet, users go to the bank with cash!
We accept all popular payments, without exception - from bank cards to PayPal, e-wallets, Google Pay, Apple Pay or SMS. With the help of Solid, the merchant gets access to work with all types of payment in one integration. In addition, the gateway has its own fraud protection, a payment routing system, a tool to prevent unreasonable chargebacks, and much more.
Why test payment systems and the risks of not testing
Many merchants are connected to Solid. Therefore, we are responsible not only for our business, but also for the monetization of all clients connected to us. If something goes wrong on our side, we are failing too many companies. And vice versa - by testing and improving our payment service, we improve the products of other businesses.
Unfortunately, in addition to a successful payment, errors can occur - both system and user. It is important to understand that 100% of payments will never pass successfully at any location. But for our part, we are obliged to do everything to make the conversion of payments as possible as possible.
Testing a payment gateway is not a routine task, since it is a rather complex mechanism of dozens of microservices and interconnections. We test everything in three stages. First, we check each task in an isolated environment, then we combine them and check them in the release candidate at the stage, we see how everything works in integration, then we release and test everything again in production.
Let's take a closer look at what specifically we have to test.
Features of testing payment systems
The payment system consists of both API and web interfaces. There are no fundamental differences in API and web testing for payment systems, unlike any other products. The main feature is in testing specific payments for different regions, as well as all auxiliary services.
It seems that testing the payment is not difficult: you need to make a payment, check that the money was debited from one account, and was credited to another. But there are nuances. Testers need to know the specifics of payments in different countries, and sometimes the nuances of local laws and banking regulations.
The second point is different types of payments: subscriptions, the first payment, authorization (freezing money in the account), payment through an electronic wallet. Each of them has its own testing specifics.
Particular attention should be paid to working with currencies: not all of them have a fractional part. For example, the hryvnia has a penny, but the Chilean peso does not. If you transfer the payment amount 100 in Ukraine, the bank will write off 1 UAH, that is, 100 kopecks. And in Chile that would mean 100 pesos. As you can see, such moments should not be missed.
What needs to be tested in payment systems
All our clients (web, mobile applications, as well as back-end services) communicate with us using the Solid API . The gateway itself is located on a separate cluster and communicates with various systems (antifraud, tokenizer, banks, etc.).
Solid developers are engaged in solving several types of problems (and all these tasks end up at the disposal of the QA team):
- ( , , , , );
- (PayPal, Alipay Visa Mastercard);
- : API, ;
- ( , );
- , (, , -);
- .
In addition to the tasks that come directly from the developers, we receive others: checking all data and UAT when connecting new merchants, checking tasks from the DevOps team, setting up anti-fraud rules.
To summarize, everything we test can be broken down into the following categories:
Solid backend services:
- antifraud;
- subscription service;
- payment routing;
- accounting and financial controlling system;
- integration with external services.
Integration with banks:
- checking the correctness of work with currencies;
- verification of different types of payments (first payment by card data, payment by token, refunds, money freezing, etc.);
- processing of notifications.
Alternative (non-card) payment methods:
- verification of the payment;
- checking the features of the location.
Admins:
- internal admin panel (everything that helps Solid analysts run A / B tests for conversion of a payment form, set up rules for antifraud);
- admin panel for merchants.
User interfaces:
- appearance of payment forms and pages;
- whether the form is displayed in the language of a specific region (the Solid payment form is available in all languages of the world);
- correct display of the amount and currencies;
- tracking user actions on forms;
- page status.
Other:
- UAT when connecting a new merchant to Solid;
- tasks from the risk department to check new configurations;
- functional health studies (for example, does Apple Pay work in WKWebView).
Steps to Test Success
Automation
When working with large-scale IT solutions, such as payment providers, you need to constantly test not only the operation of individual elements of functionality, but also their interaction. It won't work without automation. If some service is not covered by autotests, failures cannot be avoided. Run autotests whenever possible. Poured the task into the environment - run the tests. Merged several tasks into a release candidate - run the tests.
In our case, it goes something like this:
- developers independently run tests when implementing a task;
- testers run tests when testing a task in an isolated environment;
- tests are automatically launched when a new version of the build is built;
- autotests are constantly running on the Prod environment.
Running autotests takes time, and therefore we always strive to optimize this process as much as possible. Tests are run in a multithreaded manner, and they are also divided into tasks by importance.
We have a minimal set of tests that validate Solid's core payment processing functionality - it runs in less than a minute. All other tests of Solid API and other microservices take about 3-4 minutes. UI tests are, of course, slightly slower. But even here we do not stop working on improvement and optimization.
Why is isolated testing not the best option when testing payments? I'll tell you on our antifraud case.
Each Solid merchant has an anti-fraud account connected with the setting of rules, how they should work. For example, if a user has made more than three payments per day with a merchant, then we block the fourth. Or if more than five users make payments with the same card, we also block and add them to the black list. We covered it with autotests, but ran into a problem.
Initially, we duplicated all the rules for a test account, simulated fraudulent actions, and the tests seemed to work. But it turned out that the merchant himself often has situations when anti-fraud rules are combined, for example, three of them worked.
By isolating each payment for each specific rule, we got rid of the possibility of a combination of rules and the influence of other services on the payment process.
How we made each payment: we cleared the test data, created all the conditions for the payment to fall under a certain rule, and then it worked. But it is not a fact that it will be so in a work environment, because there will never be an ideal situation for a payment to fall under one rule.
We decided to connect real anti-fraud rules of our clients to the test merchant. Then they began to work out more clearly. That is, it was necessary to create not isolated rules, but to test them in aggregate as they are for a particular client.
Now Solid customers can customize anti-fraud rules for themselves. Because transactions that look like fraud for one merchant may be the norm for another.
If a person makes five purchases per day in an online store, then this is the norm: first he liked the case for the mobile, then the notebook, etc. But for fitness applications this is already an anomaly. A person is unlikely to buy five workout programs a day.
Automation really helps to build a balance: only new features and those scenarios that require human attention are manually checked, everything else is automated. Automation can help reduce both testing costs and human risk.
Testing at the stage of technical specifications
In addition to directly checking the functionality of the functionality, we devote a lot of time to ensuring that developers and managers perceive the implemented functionality in the same way. It seems obvious, but many neglect it.
All configuration services are quite complex, they have a huge range of capabilities, and even the smallest details can lead to the service not working as expected.
The "early testing" technique has several advantages at once:
- the development team will correctly understand the task, and we will not waste time on fixes;
- a well-written technical specification is 70% of good documentation;
- due to the fact that the testing team familiarized itself with the TOR in advance, the test scenarios are also thought out in advance, and at the moment when the implemented task comes to the test, the process goes faster.
Good test documentation
Really good and structured internal documentation is half the battle when testing. Despite the fact that almost all functionality should be automated, manual work will never come to naught.
Description of testing processes for various functionalities, articles with possible problem solving and various manuals speed up the work of the testing team.
We at Solid have created our own knowledge base. It describes how each bank and an alternative payment method work in different locations, what types of payments the bank supports, and how, in principle, payments are made in a particular region.
Such a base is a capacious and complex task that has become a challenge for us. It was necessary to bring all the documentation together and describe the processes in an accessible way. But now, when a new employee comes to us, there are no problems. It's hard to remember how something should work the first time, and if you have test documentation, the option to make a mistake is minimal. The clear documentation allows the tester to determine exactly why the payment does not go through: it is a mistake or this type of payment simply should not work in this bank.
Let me give you another example. Once we changed the logos of international payment systems on payment forms. We have more than 200 different designs for our clients. For each design there can be several configurations of fields on the form. For example, for Brazil, an additional CPF field is added - an analogue of our identification code.
Due to the new size of the logo, some fields on the form could move out, hide or become unclickable. Nobody on the Solid testing team just physically knows how all 200 shapes should look.
Testing this task was nerve-wracking, but as a result, we created a knowledge base with reference forms for each of our merchants, covered it with autotests and now we are not afraid of any changes related to designs.
***
Finally, I will tell you a little interesting fact from the world of processing: the share of Restricted Card declines in Ukraine is quite low - 1-2%. Either Ukrainian banks are so good at fraud prevention, or no one wants to steal the card data of Ukrainian users ...
Nevertheless, there is no product with ideal development and testing processes, but we can improve them. After all, the task of every business is to release a quality product. Who else, if not QA engineers, will help with this?
I would be glad if you share your principles of a good testing process in the comments.