Experienced QA mobiles are kindly requested to share their experiences in the comments. It does not take much time. And beginners need it.
In the first partwe looked at the ready-made list and went through the first four steps: we tried to get our statistics, analyzed the application and target audience, prepared a requirements / characteristics template by studying the statistics of manufacturers. And we separately thought whether we need a tablet (s).
In the second, we move on to the selection of specific models, along the way having studied the leaders and general market trends.
Let's outline the basic principles of device selection:
- strive to cover more parameters from the first table;
- between popularity and diversity, the choice towards diversity, but taking into account the popularity;
- for Android, the most important criterion is the manufacturer, followed by the screen (aspect ratio);
- we take into account not only the budget, but also the number of testers on the project.
We will match devices for a conditional new application on both platforms. CA is the average Russian-speaking. Geo: Russia (2/3 users) and Ukraine, Belarus. The app itself does not use a camera, GPS etc. The team is supposed to have three testers.
Our team said yes to questions about the tablet and Huawei.
Now we "step" further.
5. Preparing the device table
She, too, is likely to be individual, the option below is the very basis for our "average".
I indicate the prices in Moscow, translated into $. It is cheaper in Europe, more expensive in the regions.
It is also desirable to add a column "Explanations", for example, "top-1 Android in the Russian Federation, traffic 20 ... g" and a column for logical resolution. Later, add the OS column here and distribute what to whom.
6. Be the first to add the reference device to the list of applicants (relevant for Android)
This is usually the Google Pixel . It is with him that it is desirable to start testing.
The most favorably accepted was the budget model 3a (half the price of flagships), discontinued in July 2020 (4a is expected in August-September), while we write Google Pixel 3a , you can still find it on sale.
Sometimes (in a distributed team) this item can be replaced (or included in the list as the second item) with the " favorite device of the developer ", quite possibly it will be a big geek OnePlusor Samsung on pure Android. It is advisable to use it to run bugs found on other devices. It helps a lot from "I can't reproduce" in difficult cases. Or direct indication of device dependency in bug reports makes life easier for developers when you are one team.
However, with a very modest budget, this can be left for later: it will be more difficult for developers, but your audience is most likely not sitting on Pixels and VanPlus.
7. Most Popular
These devices will bear the greatest load, you will spend the most time with them.
If your app is targeting a non-technologically advanced audience, go to AppBrain for stats .
We will go to DeviceAtlas .
According to their assurances, they cover about 20% of users and this is really high coverage, but like any "not your" statistics, it is not 100% correct (for example, you can compare it with the AppBrain statistics on the link above). Top-12 mobile traffic in several dozen countries is openly available. There are Russia and Ukraine. There are still data for Q2 2019.
Let's start with iOS, everything is simple and clear, we write down iPhone7... The leader in traffic, a successful model with good sales, the percentage will fall over time, but it will remain in the top for a long time.
On Android, we write Xiaomi Redmi Note 4 into the table .
To understand the situation with leading devices, I will show three diagrams from three different sites: general, broad-topic and narrow-topic.
The first thing I want to draw your attention to is cutting the cake into narrow segments, in other words, the minimum difference between the first, second, third popular device. So if, let's say, your group already has Xiaomi 4X in stock (Xiaomi-leader of the past years), there is no need to buy Note 4.
The second point is that the target audience may differ in preferences from the average statistical use of devices, as can be seen in the third diagram, pay attention to positions 2 and 6 (Meizu M3s and ASUS ZB602KL).
8. Promising beginners
Now let's move on from the traffic leaders to the sales leaders. We look at the general trends, see if there were clear leaders (sales by a large margin, and not just the first among those sold) and pay attention to the iPhone , which is still only one in our table and the models of those leading manufacturers (Android) that we have not yet closed ( Samsung and Huawei ).
It's just a search for help. We start with general ones, then move on to the region. According to global trends, we learn that at the end of 2019 Apple (73.8 million units were sold in the IV quarter) surpassed Samsung, largely due to the sales of 11 models , and in 2020 Huawei took the lead, but mainly due to the Chinese market. Samsung did not meet the expectations of the sale of Galaxy S20, but went greatGalaxy A51 . Xiaomi has “conquered” India by becoming the number 1 there, and continues to move to Europe.
In August, the sale of the Samsung Galaxy M31s began , which is essentially the same A51 , but with a very large battery (6000mAh), the company is pinning its hopes on this model, hoping to win back the lost leadership from Xiaomi.
The Russian market is actively selling Huawei phones (Huawei and Honor brands) despite the problems with Google Play, and among Samsung's sales, more than a third fell on Samsung Galaxy A51 , this is a clear leader.
We add the iPhone 11 to the table (although with a larger budget it would be possible to take the iPhone Xsand 11 Pro Max , to better cover the options on the screens) and Samsung Galaxy A51
9. Replenish the top three manufacturers (Android)
We have closed Xiaomi and Samsung, it remains to pick up a Huawei device .
Open DeviceAtlas and search the page for and write down which models are leading among neighbors:
- Spain: Huawei P8 Lite (1), Huawei 20 Lite (3)
- Italy: Huawei P10 Lite (2), Huawei P8 Lite (4), Huawei P9 Lite (5), Huawei P20 Lite (6);
- Poland: Huawei P9 Lite (3), Huawei Mate 10 Lite (5).
The position in the rating among Android devices for this country is indicated in brackets.
We check the characteristics of those that occur several times, it is advisable to find 18: 9.
Huawei P8 and P9 with the same screen as Xiaomi Redmi Note 4 which is already on the list, but Huawei P20 Lite 19: 9, which is fine too. In addition, according to the site statistics that scored higher, this model is in 4th position on the middle chart, i.e. presented and actively used on the local market. Excellent. We write it down.
If there is no real site statistics, be sure to check whether the model is being sold in the Russian Federation.
10. The fifth player on the Adroid
Now take a close look at both tables, checking what is missing.
It is quite possible that after a couple of years, when choosing this scheme, you will have an uncovered aspect ratio of 16: 9. We don't have the 18: 9 option. And there is no Android device with an eyebrow / bang, and the resolution is entirely 1080, although it now accounts for 75% of devices, it would be nice to close 720/750.
We go to the catalog, which makes it possible to select devices by parameters, including the aspect ratio. There is such an opportunity on e-katalog .
We look at the result, paying attention to Lenovo, LG, Sony, Nokia (in the first part, when we looked at manufacturers, I paid your attention), Meizu, ASUS (they were in the statistics of sites above).
This device will not be used very actively, it makes no sense to take an expensive Sonya, we are looking at cheaper options.
Set it to 18: 9, sort by popularity. We choose "with bangs" and ... there is not a single match.
We remove 18: 9 and see what is with the bangs.
A good $ 125 option for the Nokia 5.1 Plus, but no longer available.
Choosing the Motorola Moto G7 Play . The aspect ratio is not what you were looking for, but the resolution is only 720 and the bangs are not bad parameters for testing at a price of $ 97.
11. Tablets
The first part was a discussion of whether tablets are needed at all. They also saw that Apple is definitely in the lead .
In iPad four model series: just iPad, iPad Air, iPad mini, iPad Pro.
Itself would gladly read the opinion of the pros, is there any difference on which one to test. In my opinion, it is important that there is at least one iPad in the device pool and if this is not a specialized application, it does not really matter which one.
I made a bet on iPad Pro 12.9` .
Firstly, the largest screen, and secondly, it has good prospects and its percentage will grow: designers love it, they buy it instead of a laptop (with a keyboard), and many large firms supply their employees / representatives with firmware.
But it was possible to take mini 5 (I now have mini 5, Pro 12.9 in the plans).
Because the previous four mini were at one time the bestsellers and, based on this, the number of devices on hand should be noticeable and it has the same resolution as the Air 2 and Pro 9.7 ( resolutions ).
You could have taken a simple iPad because it is the "golden mean" or iPad Air 3, because according to statistics, there are most of them in the world, the younger version of iPad Air 2.
Apple has a comparison of models www.apple.com/ru/ipad/comparemodels are available for sale, as well as those already out of production (as a separate block in the general list).
Now let's move on to Android .
We look at statistics on manufacturers at Statcounter .
Statistics by country are slightly different, but Samsung is in the lead everywhere .
Without access to traffic statistics for tablet models, we google what happened with sales last year.
In 2019 , Samsung's most popular models were the cheap Galaxy Tab A and Galaxy Tab E models .
I look at the site statistics available to me - it is.
A lot of Galaxy Tab A and Galaxy Tab E, with no clear leader.
At the beginning of the list, the Galaxy Tab A 10.1 and Galaxy Tab E 9.6 models are repeated, you can take them.
When there is no stat at all, taking into account the very small percentage of android tablets in the total traffic, you can choose a model conditionally "by eye".
Scroll through the offers on the market sorted by "popular first" in large active directories:
- Yandex Market
- e-katalog
- Online (Belarus, dial-up moderated reviews)
We also pay attention to the number of reviews / ratings.
(We don't look at the expensive S series, only A and E)
In two of the three catalogs, the leader is Galaxy Tab A 10.1 for both parameters , and we write it down.
This is almost the final table.
12. Operating system version
The final step is to distribute the operating system (OS) versions across devices.
Do not forget that not any OS will be on any phone. Android software to help GSMArena .
I will not cover this part.
Clearly described in the already available publications:
Statistics services for mobile applications
What do mobile QA and an octopus have in common ? It's 16 years old, but the approach is relevant.
Well, the courses teach the choice of OS.
Yandex.Practicum devotes several lessons to choosing a desktop environment, the basic principles are the same.
On the LearnQA mobile courses, they tell in detail about the choice of mobile operating systems and the basic choice of devices, enough for the first choice, especially since you will have to look for a diploma yourself for a specific project and budget.
Summarizing
In total, in the middle of 2020, the purchase of an average pool of devices without tablets will cost approximately $ 2500- $ 3000, for tablets another + $ 1500. I emphasize - average. If the application or the mobile version of the site is aimed at the premium segment, the purchase of flagships will require completely different investments.
The fact that today testing is no longer the prerogative of only large projects is wonderful. But it hurts to read in QA-channels the throwing of testers when "they said one device will pay." This is not serious. It doesn't work that way. If it's really tight, you can at the very beginning reduce to five devices, but definitely not to one or two.
Happy testing everyone and fewer live bugs.
Svetlana Skrebneva