For many years in the server processor and chipset market, Intel and its xeons ruled unconditionally, but in 2017 AMD finally competed with them with the release of the first line of their new EPYC server processors. Since then, their position has only strengthened, and the lineup has been replenished with a new version of "epics". AMD supplies its modern processors to many well-known companies, including Dropbox, Hetzner, Linode, CERN, OVHcloud, Scaleway, several universities and smaller organizations, including data centers. Everyone praises the platforms at EPYC and recommends them for organizing and scaling a cluster of high-performance servers.
When the news began to arrive that AMD began to make processors that bypassed
Intel processors in terms of performance and price / quality ratio, we began to think about a separate service on this platform. The choice fell on Supermicro A + Server 2124BT-HNTR . The server consists of 4 nodes, each node has the following specification:
- Processors: 2x AMD EPYC 7742 64-Core Processor 2.25GHz;
- Memory: 16x64GB RAM 3200 MT / s;
- NVMe disks for the cluster: 4x INTEL SSDPE2KE076T8;
- System disk: 1x INTEL SSDSC2KB240G8;
- Network controller: AOC-MH25G-m2S2T.
All characteristics
There was no such server in Moscow and Russia, we had to order it directly from the
manufacturer. After waiting, which was complemented by some logistical problems associated with COVID-19, the server ended up with us in early June.
Assembly
At first glance, it seemed to us that there was nothing particularly epic about the server, since we have experience working with platforms of 12 nodes for the service of Hi-CPU servers . The same platform represents only four independent nodes, but we did not yet understand how much power they give in aggregate.
Tests
After assembly, of course, we wanted to load the server, we are dealing with a monster, in
total, which has 1024 cores, 4TB RAM on 4 nodes: Clickable picture The performance test results impressed us. The build quality from Supermicro engineers is also pleasing.
Installation
We mounted it in a rack and started assembling the cluster: The
cluster is assembled on these 4 nodes. In the future, the cluster can be scaled on similar
equipment or on similar from other vendors. The public network of the cluster is located on
interfaces 25Gbps (SFP28), the private network is also 25Gbps (SFP28).
Software revision
The fundamentals and principles laid down in the client service management software made it possible to quickly finalize the code for the new architecture, in mid-June the Epic Servers service was launched.
Benefits for us and customers
AMD EPYC allows you to create tariffs up to EPIC 128 x 3.4 GHz CPU / 512 GB RAM / 4000 GB NVMe, which no one can offer on a platform with processors from Intel. In addition, we use Only NVMe NAS with triple data replication.
During 2 months of working with this platform, we felt all its charms - customers can be provided with the same resources as in the case of Intel, but for less money. One node with 2 CPUs from AMD replaces 10 nodes with 2 CPUs Intel Scalable Silver.
A spoon of tar
Such a server is very expensive - the cost is about $ 150,000 and, of course, it was bought on full prepayment, in fact, we were buying a pig in a poke since we had not previously worked with AMD servers.
The second point, QEMU-KVM does not allow non-stop migrating virtual machines from AMD to Intel and vice versa, but using our backup functionality, you can switch from one type of server to another at any time without data loss.
The cluster showed us previously unknown performance indicators, which naturally
began to affect the temperature of processors. Even with a processor load of 30% on a node, the temperature was approaching 95 degrees. We have optimized the cooling somewhat in
rack, but it did not give any significant result. Then, in the form of a test, the cover on the server case was removed:
The temperature jump down was more than 10 degrees and did not exceed 80 degrees, even with a
load of more than 70%, which more than satisfies us. There are also plans to put additional cooling in the rack.
Outcome
As a result, the server more than met our expectations, the AMD EPYC 7742 processor seems to us unbreakable under load. The scheme based on such solutions will continue to
scale, over time we plan to launch this service in the Netherlands. We enjoyed working with EPYC and hope you will enjoy it too!
