The story of one "broken" test case, or be careful with the OpenSSL versions ...

Disclaimer. I am not a "real welder", but due to the search for interesting work in the field of information security, lately I have been regularly solving various CTFs and machines on HackTheBox. Therefore, when I was sent a link to one of the CTF-style test items, I could not pass by ...







The meaning of the test item is quite simple. A traffic dump is given, in which the encryption key, some garbage and an encrypted flag are hidden. We need to extract them and decipher the flag. Also shown is the OpenSSL command with which this flag was encrypted. The traffic is quite interesting, but after 10 lines of python code, I had an encryption key, garbage and an encrypted flag in front of me. It would seem, what could go wrong?



The assignment says that the flag was encrypted with approximately the following command (I omitted some insignificant parameters and changed the encryption algorithm).



echo "FLAG_xxxx…xxxxxx" | openssl enc -e -base64 -aes-256-cbc -nosalt -k $password 
      
      





I inserted the parameters obtained from the traffic into the command, started it and ... got garbage! I tried again. Garbage again. I tried to rebuild the traffic in different ways. No, apparently, traffic can only be collected unambiguously. But the encryption output is garbage again !!! At the same time, OpenSSL honestly warns that getting a key from a password in 1 pass is a bad idea ...



echo "ENCRYPTED_FLAG" | openssl enc -d -base64 -aes-256-cbc -nosalt -k $key 
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
      
      





The next day was spent trying to crack this madness. I reasonably reasoned that apparently I had incorrectly isolated the "garbage" specified in the conditions and wrote several options for dividing the resulting string into "password" and "encrypted flag" for subsequent "brute force". It didn’t help ... I began to deeply understand each of the parameters.



As we know, AES requires an encryption key and IV (initialization vector) to work. The -k option allows us to use a text phrase, from which OpenSSL itself gets the required key and IV. You can see them using the -p parameter.



echo "FLAG_123" | openssl enc -e -base64 -aes-256-cbc -nosalt -p -k "password"
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
key=5E884898DA28047151D0E56F8DC6292773603D0D6AABBDD62A11EF721D1542D8
iv =3B02902846FFD32E92FF168B3F5D16B0
C11kA+GcqkU4ocOvZAVr3g==
      
      





This knowledge also gave me nothing. Then I nevertheless decided to return to the craziest idea that came to me. Namely: the problem is not with me, but something has changed in OpenSSL ... The

traffic was dated 2016, so I took Ubuntu 14.04 and, without much hope of success, just inserted the initial data into it. And suddenly, instead of garbage, I got a FLAG! The evening ceased to be languid ... Moreover, the same command with the same password and the -p parameter produced completely different encryption keys and IV!



NEW SYSTEM (openssl 1.1.1h)



echo "FLAG_123" | openssl enc -e -base64 -aes-256-cbc -nosalt -p -k "password"
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
key=5E884898DA28047151D0E56F8DC6292773603D0D6AABBDD62A11EF721D1542D8
iv =3B02902846FFD32E92FF168B3F5D16B0
C11kA+GcqkU4ocOvZAVr3g==

      
      





OLD SYSTEM (openssl 1.0.1f)



echo "FLAG_123" | openssl enc -e -base64 -aes-256-cbc -nosalt -p -k "password"
key=5F4DCC3B5AA765D61D8327DEB882CF992B95990A9151374ABD8FF8C5A7A0FE08
iv =B7B4372CDFBCB3D16A2631B59B509E94
R3N+5v3zOz9QcNt08cwqcA==
      
      





It became clear that the fears were confirmed. The algorithm for generating Key and IV from a passphrase has changed, which completely broke the ability to solve CTF head-on on modern versions of OpenSSL. In the process of searching for the implementation nuances, I came across a very interesting work " Password-based OpenSSL Encryption Analysis of Key Derivation Protocol " and everything fell into place. In short, version 1.1.0 added a new protocol for generating keys from a password PBKDF2, but more importantly, the old PBKDF1 algorithm changed the default hashing algorithm from MD5 to SHA-256! Thus, the same password produces different Key and IV. In order to decrypt previously encrypted, in newer versions you need to use the -md md5 parameter .



“-Md messagedigest: specify the message digest used for key derivation from md2, md5, sha, or sha1



After adding this parameter, it became possible to get the flag on the new OpenSSL. I don’t know if this "nuance" was really taken into account by the developers of the test task, or they simply did not test it on modern systems, but the fact remains that I had to dive deeply into some of the subtleties of OpenSSL.



PS Through my acquaintances, I have already informed the developers of the test test about the problem I found, otherwise they are suddenly very surprised that these are people who do not go to them ...



All Articles