Public clouds emerged as a response to the agony of companies that maintained their own server fleet. Too many problems arose with the ordering of equipment, its configuration and support, so cloud computing quickly became in demand - says Alexander Volochnev, Developer Advocate at DataStax and author of the AWS video course at Slurm .
The pains of sysadmins and business, as well as the opportunities that clouds brought them, Alexander Volochnev described in detail at the webinar "Creating an effective infrastructure using cloud solutions." The recording of the webinar is available on YouTube , here is a brief summary.
Problems of maintaining your infrastructure
Maintaining their own server park, everyone suffered: developers, support, and sysadmins. Sysadmins got the most of the rest: there weren't enough servers, and they didn't give new orders - there was no budget; one engineer worked for ten; the support of the fleet was eating up all the time, there was no time left for development; Developers did poorly assess the load on the infrastructure; the servers ordered with grief in half traveled for several months.
Not only ordinary employees were tormented, but also managers: they constantly had to look for money for new equipment and at the same time - normal people for the team; admins did not develop infrastructure, but only patched the current fleet; The database was slow, customers were unhappy; it was difficult to estimate how much hardware is needed for a new project.
With the support of their server fleet, companies have and continue to face several major challenges.
Slow changes. Expansion, scaling, adding capacity - all this takes time. Equipment must be ordered several weeks or even months in advance, payment in advance. You can rent it, but the minimum rental period is a month, even if the server is needed for one day. More often the rent is annual. Any operations require the participation of people, and there is always a shortage of intelligent ones.
High cost and low reliability. The system is either fragile or very expensive. And often fragile and expensive.
Lack of flexibility in geography.The solution to the problem of geographical distribution fits into the DDH scheme - long, expensive, shitty. At the same time, there will be no fault tolerance without distribution. If the data is located in the same data center, the application cannot be called fault-tolerant.
For example, AWS operates by regions and zones. A region is a collection of data centers housed together. An accessibility zone is a specific data center, necessarily remote from neighbors, having an independent connection to the Internet and power supply. This is real distribution and fault tolerance.
Lack of flexibility in performance. Their servers are either idle ¾ of the time, or there is not enough capacity at rush hour, and everything is lying.
Economy Mode vs Performance Mode: Either Peak Denial of Service or Overpayment The
ideal business situation is when both peaks and troughs are closed, and there is no overpayment. From the point of view of engineers - when, if necessary, there is an opportunity to raise a new environment and bang it as soon as the need disappears.
Ideal situation: capacities and costs for them change according to business tasks
It was just such an ideal situation that Amazon was striving for, where they invented clouds. The marketplace is characterized by seasonal loads: quieter nights, heavier days, plus Black Friday and the peak before Christmas. The site either drops during peak loads and loses customers, or pays for infrastructure all year as much as it earns at its peak. Over time, they found a way out - they began to rent out idle iron to compensate for the costs.
This is how Amazon Web Services (AWS) and a type of service called clouds came about.
So the clouds appeared, and they became in demand, because everyone - literally - got sick.
What is cloud
Suppose a company needs to organize its work with data, and management decides to create its own infrastructure. The company builds its own data center or leases space in a commercial data center. Buys and installs servers. Configures storage and networking. Further, he is engaged in virtualization, installation of operating systems, databases and other software necessary for the operation of end applications.
Now, imagine that management chooses IaaS (Infrastructure as a Service) instead. In this case, the cloud provider takes over part of the infrastructure organization tasks. The data center already exists, it has been built and meets the highest reliability requirements (at least in the case of large providers like AWS or Google Cloud). Storage, networks and virtualization are also configured by the provider.
Thus, the company has only to prepare the environment for working with data and its administration. Engineers have time for development, improvement, pumping, optimization. They become more involved in the development process, DevOps appears.
Distribution by responsibilities
Over time, cloud providers have evolved services and, in addition to IaaS, approaches such as Platform as a Service (platform as a service) and Software as a Service (application as a service) have emerged.
For example, suppose a company has developed an application and wants to launch it. In the case of IaaS, you need to log into the AWS web interface, create a virtual machine. The operating system will already be installed, but you will have to configure it yourself. You yourself will have to install the necessary software and understand how to work with data.
In the case of PaaS, everything you need is already installed, you just have to manage your application and data. Already easier, and fewer admins are needed for support. PaaS is not suitable for everyone, but it will definitely be useful for startups that need to quickly deploy and have no time to think about hardware.
And finally, Software as a Service is an application as a service, when you just use some service and don't care about anything.
Additional features
Features of the cloud: "damn magic!"
Let's consider the key features of clouds. Amazon Web Services (AWS) is often mentioned in the examples, because this company started it all, and now they are the leaders in the cloud services market.
On-Demand Self-Service - Self-service. The ability to get everything you need on demand, without contacting anyone. Without phone calls, without sending documents by mail, without waiting for several months - right away.
Fast Flexibility (0 → 100 → 10) - "fast" flexibility. Today the company does not rent a single server, tomorrow it will take one hundred machines, and the day after tomorrow - ten. Very fast and flexible.
Resource Pooling - resource sharing. Data from different companies can be stored on one iron server: the load profiles do not coincide, and they do not interfere with each other, but both pay less than for a dedicated server - the concept of an apartment building. However, not everyone loves apartment buildings. Then you can say "I want a dedicated server" and you will get it. Naturally, the price will be higher.
Elastic Scalability is flexible scalability. Today you need 10 servers, tomorrow you need 100, the day after tomorrow again 10. With clouds you don't have to think about where to get that much. You can simply use and close when no longer needed.
Measured Service is a measured service. Resource consumption is measured down to a bit and a microsecond so that the user only really pays for what was consumed. AWS was the first to introduce per-second billing for servers.
Pay as you Go - pay as you go. Most cloud providers charge ex post. But if the user knows that the server will be needed tomorrow and in a year, he can pay in another way: immediately for the entire period or in parts. There is a discount for this.
AWS has three ways to get a server: on-demand - pay as you go. reserved - payment for a year in advance, in whole or in parts; spot instances - payment by auction type. In the latter case, users set the price they are willing to pay for Spot Instance, and the server goes to the one who paid the most. The system is not the most reliable, but discounts can reach 90%. Spot Instance is suitable for delayed execution of tasks when processing speed is not as important as cost.
, , . -, , . -, , , . , . .
Global Availability / Distribution - global availability. AWS has 22 regions and 77 data centers worldwide, plus about 200 points of presence. Azure and Google Cloud have similar infrastructure. Thanks to this, clients of cloud services can use their applications, manage their data globally. If a company operates in one region, and customers live in another, then you can configure everything so that the connection from the client goes first to the internal AWS network and from there, at high speeds, goes to the company's region.
Programmable Access / Management - programmed access and programmed control. Previously, to order servers, you had to call suppliers, explain your tasks, wait for a commercial offer, call again ... Only a very large customer can contact AWS by phone. Time is expensive, so everything is done through a web interface, command line interface and software management.
If you need to optimize server capacities depending on traffic, then it is enough to configure the AutoScaling Group once - a group of automatic scaling for servers, monitoring and triggers. After that AWS will create and delete servers itself depending on the load. The client gets scaling with minimal overpayment and fully automatic control.
"Programmatic access, programming your infrastructure is really fucking magic, I just love it!"
AWS offers about 250 services for a wide variety of purposes. If we talk about infrastructure, then there are databases, and network storages, and in general everything. There is no industry for which they have not yet proposed something. Look at the screenshot, in the lower left corner there is a tab "Satellite" - control satellites. That is, if you want, you can work with satellites in AWS!
AWS Services
In fact, AWS has everything you need to develop. If you want to organize the work of microservices - please, there is a whole framework based on AWS. In modern applications, 99% of the code is repetition of what has already been written. In order not to repeat yourself, you can come to AWS and use it.
Cloud providers
AWS, Microsoft Azure and Google Cloud are the most popular cloud providers. But besides these, there are more than ten major cloud service providers.
Amazon landed first, and they continue to occupy about half of the market. Microsoft Azure joined with a long delay, but now every fifth customer uses them. Google Cloud Platform lags behind them a little, but in principle, not significantly.
Distribution of cloud providers in the market
Which one is better? I will not answer this question, because for the answer you need to take into account so many factors and nuances that twenty webinars and twenty specialists will not be enough (in the end, the specialists will fight and will not come to one solution). Google Cloud is considered cheaper than Amazon, and this is logical: if you enter a market that is already occupied, then with high prices you will not have a chance. As far as I know, Google Cloud is the fastest growing cloud right now.
Some companies prefer to use Hybrid Cloud and Multicloud. Multicloud means that resources are spread across multiple clouds. It is organized to save money and provide fault tolerance. If one type of processing is cheaper on Google Cloud, the second type on Amazon, then distributing the data is more profitable than keeping it in one service. However, it should be borne in mind that transferring large amounts of data between clouds can be expensive.
For fault tolerance, they duplicate data in the storages of two providers, so that if one falls down, the second will hedge. But there is a chance that this duplication will cost much more than the loss in case of downtime.
How to try
All providers have free access where you can try all the services. It will not work to create large storages, but for the test, free capacities will definitely be enough.
In the Cloud Administration course, we teach you how to work with AWS and how to deploy autoscaling systems. The video course was released in January 2021. It will be useful for developers, DevOps, and system administrators. Architects and technical management will also benefit: to save time, you can skip practice and focus on theory and demonstrations. They will help you understand how AWS works and how to use it in your company.