Diesel generator. Spontaneous emotional buying. We went out for a loaf and sausage, came back with it
We choose only homogeneous iron, removed the sales department, we have not gone to negotiations since 2015, we write robots to control people, we choose a stable architecture, we insure risks (including jumps in the dollar exchange rate) at the beginning of each year.
This is perhaps the main thing.
VPS hosting management and business management are strategic planning and control over the implementation of what is planned. That is, first you need to sit down and think, set a goal, and then very carefully monitor how we are going to it. If you think for at least five minutes about hosting business in our price niche, then a chain of obvious decisions follows. We came from finance to hosting to earn money, so we immediately made the main mistake of doing business in Russia. They began to plan for five years ahead. It was quite optimistic, but we survived and are now reaping the fruits of this strategy. If we had failed in the first year, then, of course, the losses would have been greater than those of hosting without such a strategy.
A lot of automation is needed to control the execution of decisions. A lot of information is needed to make strategic decisions. One of the most important parts of our business is monitoring everything and everyone. And since we are very lazy, we shift it to robots.
Monitoring
In order to have a very detailed idea of what is happening with the infrastructure, the team and the entire surrounding world, you can collect manual reports, or you can make them automatic, where possible. The next part - when viewing reports, it makes no sense to look at where everything is fine. I'm interested in reports that show a jamb or warn about it. A report that shows everything good is just wasting time.
For example, we have a robot that analyzes traffic. If anomalies begin, he shows them to another robot, which notifies our team about the beginning of DDoS on the hosting client. A third robot looks after them and checks their performance. All three types of work were once done by hand, but were quickly automated.
Infrastructure monitoring is done somewhere by Zabbiks, but for the most part, the data is combined into our custom self-written system. Physical nodes are monitored out of business interests. For example, we constantly need to know the number of free IP addresses. If they run out, you need to have time to order new ones so that the client does not press the button in vain in the interface. There is a service that monitors the free resources of servers: one application is needed so that we do not announce a promotion where there are not very many resources - otherwise, clients will not be able to start a VPS in this data center. Another application is quoting in the resource site configurator to create a new machine or sizing available in a specific data center. If there are 10 GB of disk left, then the interface in this data center will not allow creating a machine with 20 GB.A personal account on the site is directly related to monitoring and receives up-to-date data every time a server is created. In fact, this means that there are no many support tickets for manually creating cars and resolving such cases.
By the way, this is an automation created by another automation. If the machines are created manually by support, then there will be no cases of going over the limit. If machines are made by a robot, then another robot is needed to control such situations.
An example of how the configurator works when it is not possible to create a server with a maximum configuration of 16 GB RAM, 600 GB in the data center; for servers with 9-16 GB only 20 GB of disk is available, for servers with 4-8 GB 430 GB is available, 600 GB disk is available for servers with 1-3 GB of memory
Complete lack of servers available for creation
Support uses OTRS (by the way, we have a ready-made image for quick deployment on client machines directly from the internal marketplace). OTRS monitors a robot that "highlights" all the first responses for more than 15 minutes, and then sends a report on the quality of work in the mail to the team leader and me (arrives on Monday over Friday and weekends).
Report on the number of processed tickets
Since our greatest value is information, we ask marketers to read the tickets. And we assume that the support manager also reads the tickets. Again, we are from finance. If you once agreed that this should be done, you can't just leave it unattended. Therefore, another robot appeared very quickly, which counts and checks the tickets read by the marketer, whether conversations are being tapped for quality control. While he was in the tests, the new employee was asleep: he said that he spends an hour a day on reading, but in fact he opened four tickets in a month.
Report on the number of read tickets: not only support, but also marketing teams.
Bots send SMS, notifications to mail and Slaka. Our first corporate messenger was Skype somewhere around 2012 (this is also a tradition from the world of finance), but it slowed down so much that we switched to Slack ( during an interview with Romero, he told me about Slack's convenience, and we decided give it a try) - not least thanks to its cool APIs. For example, marketing receives notifications about mentioning us in the media, about the release of posts on Habré on topics of interest to us, and so on.
Bot that reports on duty admins and support staff
Bot that sends statistics from the data center.
Bots do not report that they do not require a response. The less the bot writes and the more of this on the case, the better. If there is a message in the chat, you need to respond. If you pour general information on a regular basis, it will worsen the situation in all chats - it is important that only what is really worth the distraction comes. Non-urgent can be collected in the daily report. Unimportant can be given control to another robot.
Business model with minimal costs
I already talked about the automation of all actions: for example, machines are created automatically, they are sized automatically (downward too), most of the typical tickets can be made directly from the interface (except for sizing with a change in the processor frequency, this is still manual). If you look at domestic hosting companies that are five to ten years old, you will see that they simply cannot afford such a system. Because it turns out a huge zoo of tariffs, the characteristics of each individual data center and specific hardware.
We decided that we would have exactly two types of hardware and the same prices around the world.
This immediately means very simple interfaces where the person does not get confused. This means easier automation on the hosting side. This is a much simpler iron support. If there were four types of hardware, then the same spare parts for the sites would have to be doubled, which would affect the tariffs (yes, we have a next business day-guarantee from Huawei, but we keep our spare parts in order to react immediately).
A significant part of the cost of VDS hosting services is hardware and salaries. If we optimize the payroll with automation and monitoring, then the hardware becomes much cheaper if it is bought immediately with large contracts. And even cheaper if you calculate its maintenance in advance - that's why we don't have "kitchen assemblies", but normal corporate-class servers. And therefore, even on tariffs with HDD, we have SSD disks (with a speed limiter). We calculate the final cost of ownership several years before retirement. And we consider not only taking into account a specific system, but throughout the company, that is, taking into account the rise in the cost of the same development for different types of hardware.
The most risky thing in hardware is the change in exchange rates. A surprise with a fall in the ruble in the middle of the year can be worth the annual profit if you don't think about it in advance. We seek fixed contracts in rubles, or we hedge dollar contracts. Yes, this makes them more expensive by several percent, but without such a measure it is simply impossible. In the same way, we insure the most important risks - including the risk of hacking the end customer, which we have insured with AIG. More precisely, we insure our liability in the event of a breach of an information system, including the cost of finding burglars. And for our clients, we offer individual policies that can protect them, in particular, from data loss due to criminal activities.
When we first entered the market, we dumped a little, that is, we were in the “cheapest VDS hosting” segment. It is financially reasonable to be a little higher: this allows you not to cut the quality where it comes around on the client. Next, it was important to build the product so that only what was needed was included, it was provided with sufficient quality and did not create costs. If you want perversions - there are expensive hosting. If you want 5% cheaper - there are less reliable hosting services with support, perhaps a little worse. We raised prices during all this time exactly twice: by 5% and by 10% during crises, when the ruble opened new horizons for us to fall. And not in the middle of the year, but at the end of the year. That is, during the hosting operation, we indexed prices even slower than inflation. So, when we first raised prices, we foundthat the audience of the cheapest hosting is very different from the audience of “normal fast food on the hosting market”. In just a few months, the support noted how the audience had changed qualitatively: then many experienced admins came who knew exactly what they wanted.
In any data center, we use plus or minus identical servers with identical software, which from anywhere in the world works the same with the end site, and this allows support workers and customers to work with each server in the same way. Of course, this imposes restrictions on those data centers where we stand.
Opening of a new data center
We will not get into a data center that does not meet our internal price. The internal price is determined by the tariff, and the tariff is determined by the business model, starting from the choice of iron. It turned out that in Europe it is much easier to filter the sites themselves than to constantly play with prices on different sites. We got it wrong a couple of times, but we ended up with today's model:
- We select the region of interest, it contains the entire list of available data centers, we look at the requirements for physical access, proximity to the nodes for the exchange of backbone traffic, and so on.
- , «» . , , . , . , , . , . , — . : -.
- . , : , 18:15 : « ». .
- . , ( ).
- . , , - . : , . , . . , -. . , . : . . , .
- . , , . : , , .
Switzerland as a second data center was actually a risk maneuver. When we opened it, the location was very important for a number of our customers. For many, "Switzerland" meant "the reliability of the safe." It was very difficult for us to explain to some of them that we are transporting iron to another place. But we could not do that, since it was extremely important to get away from the model of one data center (then we only had a bomb shelter in Korolev) - even in such a safe place, you cannot put all your eggs in one basket. We saw a fire transport a couple of data centers to the cloud. Together with the business owners. Since then, we have insured ALL iron from fires, floods and tsunamis.
In general, this is a strategy of constantly searching for a weak point (the concept of antifragility). Already in the second year of hosting, the collapse of the data center was a nightmare for some reason beyond our control. Crazy excavators find even the best of the best. Now there are ten data centers, and this issue has long been closed.
Now we are considering another non-standard data center. You can set up a site in Ostankino, where there are TV people. They have the highest uptime - 100% non-stop since 1991. By the way, there is a sign that if you write about this on Habré, something will immediately fall. Let's see)
Another place where we got away from negotiations is working with legal entities.
Working with legal entities
The original plan was to generate invoices and acts automatically. The client fills in the TIN and issues an invoice to himself, an agreement is automatically generated for him. No calls. Everything was so, but customers still broke through to support because of the strangeness and wanted us to send them a commercial offer.
We have a hypothesis that we are missing something important. And we created a sales department. It worked for a while, and then we closed it. We missed kickbacks: some people wanted to raise the price of the contract to reward themselves. We thought for a few seconds about automating the rebate process. The "rollback percentage" slider did not appear in the interface, but our affiliate programallows you to choose what percentage of the discount to leave to the referrer, and what percentage to take to the one who brought the leads. So not a rollback, but a rebate to the agent.
By the way, it turned out that another reason to make commercial offers manually is the ability to register the same license for Windows Server under an asterisk. The price will be good until you need to install the OS, then the surprise will be revealed. Our configurator immediately shows the final price.
Until everything
If you are interested, I will continue about the purchase of hardware in Europe, about the pricing policy and its features in the segment “a little better than a low-cost airline” and about how we searched for and found customers.
