Best in class: the history of the AES encryption standard

Since May 2020, official sales of WD My Book external hard drives that support AES hardware encryption with a 256-bit key have started in Russia. Due to legislative restrictions, previously such devices could only be purchased in foreign online electronics stores or on the "gray" market, but now anyone can get a protected drive with a branded 3-year warranty from Western Digital. In honor of this significant event, we decided to take a short excursion into history and figure out how the Advanced Encryption Standard came about and why it is so good compared to competing solutions.



For a long time, the official standard for symmetric encryption in the United States was DES (Data Encryption Standard), developed by IBM and included in the list of Federal Information Processing Standards in 1977 (FIPS 46-3). The algorithm is based on developments obtained during a research project codenamed Lucifer. When on May 15, 1973, the US National Bureau of Standards announced a competition to create an encryption standard for government agencies, the American corporation entered the cryptographic race with the third version of Lucifer, using the updated Feistel network. And along with other contestants, it suffered a fiasco: none of the algorithms presented for the first contest met the strict requirements formulated by the experts of the NBS.





Of course, IBM could not just come to terms with defeat: when the competition was restarted on August 27, 1974, the American corporation reapplied, presenting an improved version of Lucifer. This time, the jury did not have a single complaint: after competent work on the errors, IBM successfully eliminated all the shortcomings, so there was nothing to complain about. Having won a convincing victory, Lucifer changed his name to DES and was published in the Federal Register on March 17, 1975.



However, during open symposia organized in 1976 to discuss a new cryptographic standard, DES was heavily criticized by the expert community. The reason for this was the changes made to the algorithm by NSA specialists: in particular, the key length was reduced to 56 bits (initially Lucifer supported work with 64- and 128-bit keys), and the logic of the permutation blocks was changed. According to cryptographers, the "improvements" did not make sense and the only thing the National Security Agency was striving for, introducing modifications, was to be able to freely view encrypted documents.



In connection with the above accusations, a special commission was created under the US Senate to check the validity of the NSA's actions. In 1978, a report was published following the investigation, which reported the following:

• NSA representatives participated in the finalization of DES only indirectly, while their contribution concerned only changes in the operation of permutation blocks;
• the final version of DES was found to be more resistant to cracking and cryptographic analysis than the original, so the changes were justified;
• a key length of 56 bits is more than enough for the vast majority of applications, because breaking such a cipher will require a supercomputer worth at least tens of millions of dollars, and since ordinary attackers and even professional hackers do not have such resources, there is nothing to worry about.

The conclusions of the commission were partially confirmed in 1990, when Israeli cryptographers Eli Biham and Adi Shamir, working on the concept of differential cryptanalysis, conducted a large study of block algorithms, including DES. The scientists concluded that the new permutation model turned out to be much more resistant to attacks than the original, which means that the NSA really helped to eliminate several holes in the algorithm.





Adi Shamir



At the same time, the limitation on the key length turned out to be a problem, and a very serious one, which was convincingly proved by the Electronic Frontier Foundation (EFF) in 1998 as part of the DES Challenge II experiment conducted under the auspices of RSA Laboratory. A supercomputer, codenamed EFF DES Cracker, was built specifically for cracking DES, which was developed by John Gilmore, co-founder of EFF and project leader of the DES Challenge, and Paul Kocher, founder of Cryptography Research.





Processor EFF DES Cracker



The system they developed was able to successfully find the key to an encrypted sample by a simple brute-force method in just 56 hours, that is, in less than three days. To do this, DES Cracker needed to check about a quarter of all possible combinations, which means that even under the most unfavorable circumstances, it will take about 224 hours to crack, that is, no more than 10 days. At the same time, the cost of the supercomputer, taking into account the funds spent on its design, amounted to only 250 thousand dollars. It is easy to guess that today it is even easier and cheaper to crack such a cipher: not only has the hardware become much more powerful, but also thanks to the development of Internet technologies, a hacker does not have to buy or rent the necessary equipment - it is enough to create a botnet from PCs infected with a virus.



This experiment clearly demonstrated how obsolete DES is. And since at that time the algorithm was used in almost 50% of solutions in the field of data encryption (according to the same EFF), the question of finding an alternative became more acute than ever.

New challenges - new competition

In fairness, it should be said that the search for a replacement for the Data Encryption Standard began almost simultaneously with the preparation of the EFF DES Cracker: the National Institute of Standards and Technology (NIST) of the USA in 1997 announced the launch of an encryption algorithms competition aimed at identifying a new "gold standard" for crypto security. And if in the old days a similar event was held exclusively "for our own", then, keeping in mind the bad experience of 30 years ago, NIST decided to make the competition completely open: any company and any individual could take part in it, regardless of the location or citizenship.



This approach paid off even at the stage of selection of applicants: among the authors who applied for participation in the Advanced Encryption Standard competition were world-famous cryptologists (Ross Anderson, Eli Biham, Lars Knudsen), and small IT companies specializing in cybersecurity (Counterpane ), and large corporations (German Deutsche Telekom), and educational institutions (Catholic University of Leuven, Belgium), as well as start-ups and small firms that few people have heard of outside their countries (for example, Tecnologia Apropriada Internacional from Costa Rica).



Interestingly, this time NIST approved only two basic requirements for participating algorithms:

• the data block must have a fixed size of 128 bits;
• the algorithm must support at least three key sizes: 128, 192 and 256 bits.

It was relatively easy to achieve such a result, but, as they say, the devil is in the details: there were much more secondary requirements, and it was much more difficult to meet them. Meanwhile, it was on their basis that the NIST reviewers selected the contestants. Here are the criteria for the contenders to win:

1. the ability to resist any cryptanalytic attacks known at the time of the competition, including attacks via side channels;
2. the absence of weak and equivalent encryption keys (equivalent means those keys that, although they have significant differences from each other, lead to the receipt of identical ciphers);
3. encryption speed is stable and about the same on all current platforms (from 8 to 64-bit);
4. optimization for multiprocessor systems, support for parallelization of operations;
5. minimum requirements for the amount of RAM;
6. no restrictions for use in standard scenarios (as a basis for building hash functions, PRNGs, etc.);
7. the structure of the algorithm should be sound and easy to understand.

The last point may seem strange, but if you think about it, it makes sense, because a well-structured algorithm is much easier to analyze, and moreover, it is much more difficult to hide a "bookmark" in it, with which a developer could gain unlimited access to encrypted data.



The call for applications for the Advanced Encryption Standard competition lasted a year and a half. In total, 15 algorithms took part in it:

1. CAST-256, developed by the Canadian company Entrust Technologies, based on the CAST-128 created by Carlisle Adams and Stafford Tavares;
2. Crypton, Future Systems, ;
3. DEAL, , , ;
4. DFC, , (CNRS) France Telecom;
5. E2, Nippon Telegraph and Telephone;
6. FROG, - Tecnologia Apropriada Internacional;
7. HPC, ;
8. LOKI97, ;
9. Magenta, Deutsche Telekom AG;
10. MARS IBM, — Lucifer;
11. RC6, , AES;
12. Rijndael, ;
13. SAFER+, Cylink ;
14. Serpent, , ;
15. Twofish, Blowfish, 1993 .

According to the results of the first round, 5 finalists were determined, among which were Serpent, Twofish, MARS, RC6 and Rijndael. The jury found flaws in almost every of the listed algorithms, except one. Who was the winner? Let's extend the intrigue a bit and first consider the main advantages and disadvantages of each of the listed solutions.

MARS

In the case of the "god of war", experts noted the identity of the encryption and decryption procedures, but this was limited to its advantages. IBM's algorithm turned out to be surprisingly gluttonous, which made it unsuitable for operations with limited resources. There were also problems with parallelization of calculations. For efficient operation, MARS needed hardware support for 32-bit multiplication and rotation by a variable number of bits, which again imposed restrictions on the list of supported platforms.



MARS also proved to be quite vulnerable to attacks in time and power consumption, had problems with key expansion on the fly, and its excessive complexity made it difficult to analyze the architecture and created additional problems at the stage of practical implementation. In short, against the background of other finalists, MARS looked like a real outsider.

RC6

The algorithm inherited some of the transformations from its predecessor, RC5, which had been thoroughly researched earlier, which, combined with a simple and intuitive structure, made it completely transparent to experts and excluded the presence of "bookmarks". In addition, RC6 demonstrated record processing speeds on 32-bit platforms, and the encryption and decryption procedures were absolutely identical in it.



However, the algorithm had the same problems as the above-mentioned MARS: there is a vulnerability to side-channel attacks, and performance dependence on support for 32-bit operations, as well as problems with parallel computing, key expansion, and high demands on hardware resources. In this regard, he was in no way fit for the role of a winner.

Twofish

Twofish turned out to be quite nimble and well optimized for work on low-power devices, coped well with expanding keys and assumed several implementation options, which made it possible to fine-tune it for specific tasks. At the same time, the "two fish" proved to be vulnerable to attacks via side channels (in particular, in terms of time and power consumption), were not particularly friendly with multiprocessor systems and were extremely complex, which, incidentally, affected the key expansion speed.

Serpent

The algorithm had a simple and understandable structure, which greatly simplified its audit, was not particularly demanding on the power of the hardware platform, had support for expanding keys "on the fly" and was relatively easy to modify, which favorably differed from its opponents. Despite this, Serpent was, in principle, the slowest of the finalists, besides, the procedures for encrypting and decrypting information in it were radically different and required fundamentally different approaches to implementation.

Rijndael

Rijndael turned out to be extremely close to the ideal: the algorithm fully met the NIST requirements, while not inferior, and in terms of the totality of characteristics, it was noticeably superior to competitors. Reindal had only two weaknesses: vulnerability to power consumption attacks on the key expansion procedure, which is a very specific scenario, and certain problems with key expansion on the fly (this mechanism worked without restrictions only for two contestants - Serpent and Twofish). In addition, according to experts, Reindal had a slightly lower cryptographic strength than Serpent, Twofish and MARS, which, however, was more than compensated for by resistance to the vast majority of types of side-channel attacks and a wide range of implementation options.



In terms of the totality of characteristics, Reindahl was head and shoulders ahead of the competitors, so the result of the final vote was quite logical: the algorithm won a landslide victory, receiving 86 votes in favor and only 10 against. Serpent took the honorable second place with 59 votes, while Twofish came in third with 31 jury members. They were followed by RC6, winning 23 votes, and MARS naturally took the last line, receiving only 13 votes in favor and 83 against.



On October 2, 2000, Rijndael was declared the winner of the AES competition, traditionally changing its name to Advanced Encryption Standard, by which it is known today. The standardization procedure lasted about a year: on November 26, 2001, AES was included in the list of Federal Information Processing Standards, receiving the FIPS 197 index. The new algorithm was highly appreciated by the NSA, and since June 2003, the US National Security Agency even recognized AES with a 256-bit key encryption is strong enough to ensure the security of documents classified as "top secret".

WD My Book External Hard Drives with AES-256 Hardware Encryption

Thanks to its combination of high reliability and performance, Advanced Encryption Standard quickly gained worldwide recognition, becoming one of the most popular symmetric encryption algorithms in the world and being part of many cryptographic libraries (OpenSSL, GnuTLS, Linux's Crypto API, etc.). AES is now widely used in enterprise and consumer applications and is supported by a wide variety of devices. In particular, it is the hardware AES-256 encryption that is used in Western Digital external drives of the My Book family to ensure the protection of stored data. Let's take a closer look at these devices.





The WD My Book line of desktop hard drives is available in six capacities of 4, 6, 8, 10, 12 and 14 terabytes, so you can choose the one that best suits your needs. By default, external HDDs use the exFAT file system, which provides compatibility with a wide range of operating systems, including Microsoft Windows 7, 8, 8.1, and 10, as well as Apple macOS version 10.13 (High Sierra) and higher. Linux users can mount a hard drive using the exfat-nofuse driver.



The My Book connects to your computer using Hi-Speed ​​USB 3.0, which is backward compatible with USB 2.0. On the one hand, it allows you to transfer files at the highest possible speed, because the USB SuperSpeed ​​bandwidth is 5 Gb / s (that is, 640 MB / s), which turns out to be more than enough. At the same time, the backward compatibility feature provides support for almost any device released in the last 10 years.





Although the My Book does not require additional software installation due to its plug and play technology to automatically detect and configure peripheral devices, we still recommend using the proprietary WD Discovery software package included with each device.





The set includes the following applications:

WD Drive Utilities

The program allows you to get up-to-date information about the current state of the drive based on SMART data and check the hard drive for bad sectors. In addition, Drive Utilities can quickly erase all data stored on your My Book by not only erasing files, but also completely overwriting them multiple times, so they cannot be recovered once the procedure is complete.

WD Backup

Using this utility, you can set up scheduled backups. It should be said that WD Backup supports work with Google Drive and Dropbox, while allowing you to select any possible source-target combinations when creating a backup. Thus, you can configure the automatic transfer of data from My Book to the cloud, or import the necessary files and folders from the listed services both to an external hard drive and to a local machine. In addition, you can sync with your Facebook account, which allows you to automatically back up photos and videos from your profile.

WD Security

It is with this utility that you can restrict access to the drive with a password and manage data encryption. All that is required for this is to specify a password (its maximum length can be up to 25 characters), after which all information on the disk will be encrypted, and only those who know the passphrase will be able to access the saved files. For added convenience, WD Security lets you create a list of trusted devices that will automatically unlock your My Book when connected.



We emphasize that WD Security only provides a convenient visual interface for managing cryptographic protection, while data encryption is performed by the external drive itself at the hardware level. This approach provides a number of important benefits, namely:

• , , ;
• , , ;
• ;
• , « », .

All of the above guarantees data security and allows you to almost completely eliminate the possibility of theft of confidential information. Taking into account the additional capabilities of the drive, this makes My Book one of the best secure storage available on the Russian market.