Ansible-vault decrypt: dispensing with Ansible

Initial data

Given :





  • CI / CD pipeline, implemented, for example, in GitLab. To work correctly, he needs, as is very often the case, some secrets - API tokens, log / password pairs, private SSH keys - everything you can think of;





  • this assembly line works, as it often happens, on the basis of containers. Accordingly, the smaller the images are, the better, the less stuff they contain, the better.





Requires a console utility that:





  • takes up a minimum of space;





  • knows how to decrypt encrypted secrets ansible-vault



    ;





  • does not require any external dependencies;





  • can read a key from a file.





I think that the people involved in building assembly lines will appreciate each of these requirements. Well, what did I get as a result - read on.





Just in case, I remind you right away that, according to the current legislation, the development of cryptographic information protection tools in the Russian Federation is a licensed activity . In other words, without a license, you cannot just take and sell the resulting solution.





Regarding the admissibility of the full texts of decryptors in articles like this, I hope that readers competent in this matter will be able to make their own clarifications in the comments.





Start over

, , Linux- CentOS 7 Ansible, , 2.9 Python 3.6. , , virtualenv



"/opt/ansible



". - YaML-, ansible-vault



:





ansible-vault encrypt vaulted.yml --vault-password-file=.password
      
      



, , vaulted.yml



, .password



.





, ansible-vault



? - -, :





vaulted.yml
$ANSIBLE_VAULT;1.1;AES256
61373536353963313739366536643661313861663266373130373730666634343337356536333664
3365393033623439356364663537353365386464623836640a356464633264626330383232353362
63613135373638393665663962303530323061376432333931306161303966633338303565666337
6465393837636665300a633732313730626265636538363339383237306264633830653665343639
30353863633137313866393566643661323536633666343837623130363966613363373962343630
34386234633236363363326436666630643937313630346230386538613735366431363934316364
37346337323833333165386534353432386663343465333836643131643237313262386634396534
38316630356530626430316238383364376561393637363262613666373836346262666536613164
66316638343162626631623535323666643863303231396432666365626536393062386531623165
63613934323836303536613532623864303839313038336232616134626433353166383837643165
643439363835643731316238316439633039
      
      



" " - .





/opt/ansible/lib/python3.6/site-packages/ansible/parsing/vault/__init__.py



, encrypt



VaultLib



:





VaultLib.encrypt
 ...
 b_ciphertext = this_cipher.encrypt(b_plaintext, secret)
 ...
      
      



encrypt



. - -, , VaultAES256



.





encrypt



:





VaultAES256.encrypt
@classmethod
def encrypt(cls, b_plaintext, secret):
    if secret is None:
        raise AnsibleVaultError('The secret passed to encrypt() was None')
    b_salt = os.urandom(32)
    b_password = secret.bytes
    b_key1, b_key2, b_iv = cls._gen_key_initctr(b_password, b_salt)

    if HAS_CRYPTOGRAPHY:
        b_hmac, b_ciphertext = cls._encrypt_cryptography(b_plaintext, b_key1, b_key2, b_iv)
    elif HAS_PYCRYPTO:
        b_hmac, b_ciphertext = cls._encrypt_pycrypto(b_plaintext, b_key1, b_key2, b_iv)
    else:
        raise AnsibleError(NEED_CRYPTO_LIBRARY + '(Detected in encrypt)')

    b_vaulttext = b'\n'.join([hexlify(b_salt), b_hmac, b_ciphertext])
    # Unnecessary but getting rid of it is a backwards incompatible vault
    # format change
    b_vaulttext = hexlify(b_vaulttext)
    return b_vaulttext
      
      



- "" 32 . "" _gen_key_initctr



(b_key1



, b_key2



) (b_iv



).





_gen_key_initctr



?





_gen_key_initctr:
@classmethod
def _gen_key_initctr(cls, b_password, b_salt):
    # 16 for AES 128, 32 for AES256
    key_length = 32

    if HAS_CRYPTOGRAPHY:
        # AES is a 128-bit block cipher, so IVs and counter nonces are 16 bytes
        iv_length = algorithms.AES.block_size // 8

        b_derivedkey = cls._create_key_cryptography(b_password, b_salt, key_length, iv_length)
        b_iv = b_derivedkey[(key_length * 2):(key_length * 2) + iv_length]
    elif HAS_PYCRYPTO:
        # match the size used for counter.new to avoid extra work
        iv_length = 16

        b_derivedkey = cls._create_key_pycrypto(b_password, b_salt, key_length, iv_length)
        b_iv = hexlify(b_derivedkey[(key_length * 2):(key_length * 2) + iv_length])
    else:
        raise AnsibleError(NEED_CRYPTO_LIBRARY + '(Detected in initctr)')

    b_key1 = b_derivedkey[:key_length]
    b_key2 = b_derivedkey[key_length:(key_length * 2)]

    return b_key1, b_key2, b_iv
      
      



, _create_key_cryptography



, "", ( 10 ). , b_key1



, b_key2



b_iv



.





. _create_key_cryptography



?





_create_key_cryptography:
@staticmethod
def _create_key_cryptography(b_password, b_salt, key_length, iv_length):
    kdf = PBKDF2HMAC(
        algorithm=hashes.SHA256(),
        length=2 * key_length + iv_length,
        salt=b_salt,
        iterations=10000,
        backend=CRYPTOGRAPHY_BACKEND)
    b_derivedkey = kdf.derive(b_password)

    return b_derivedkey
      
      



. , OpenSSL PBKDF2HMAC



. , , , /opt/ansible/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py.







, , , , b_key1



, b_key2



, b_iv



.





. _encrypt_cryptography



, :





_encrypt_cryptography
@staticmethod
def _encrypt_cryptography(b_plaintext, b_key1, b_key2, b_iv):
    cipher = C_Cipher(algorithms.AES(b_key1), modes.CTR(b_iv), CRYPTOGRAPHY_BACKEND)
    encryptor = cipher.encryptor()
    padder = padding.PKCS7(algorithms.AES.block_size).padder()
    b_ciphertext = encryptor.update(padder.update(b_plaintext) + padder.finalize())
    b_ciphertext += encryptor.finalize()

    # COMBINE SALT, DIGEST AND DATA
    hmac = HMAC(b_key2, hashes.SHA256(), CRYPTOGRAPHY_BACKEND)
    hmac.update(b_ciphertext)
    b_hmac = hmac.finalize()

    return to_bytes(hexlify(b_hmac), errors='surrogate_or_strict'), hexlify(b_ciphertext)
      
      



, : b_iv



, b_key1



, b_key2



.





hexlify



. (. 14 )





16-20 VaultAES256.encrypt: , "", , , ( , - ).





(, - $ANSIBLE_VAULT;1.1;AES256)



, , -, .





, , - , .





, Python , : ansible-vault . , Ansible - - " " , .





, FreePascal. , , : , -, , - - .





, : FreePascal 3.0.4 ( - , CentOS 7), DCPCrypt 2.1 ( GitHub). , (fpc



) rpm- fp



.





, "" fpc



- . , , - .





, ( PBKDF2), , "kdf".





:





kdf.pas
{$MODE OBJFPC}

// ALL CREDITS FOR THIS CODE TO https://keit.co/p/dcpcrypt-hmac-rfc2104/

unit kdf;

interface
uses dcpcrypt2,math;
function PBKDF2(pass, salt: ansistring; count, kLen: Integer; hash: TDCP_hashclass): ansistring;
function CalcHMAC(message, key: string; hash: TDCP_hashclass): string;

implementation
function RPad(x: string; c: Char; s: Integer): string;
var
  i: Integer;
begin
  Result := x;
  if Length(x) < s then
    for i := 1 to s-Length(x) do
      Result := Result + c;
end;

function XorBlock(s, x: ansistring): ansistring; inline;
var
  i: Integer;
begin
  SetLength(Result, Length(s));
  for i := 1 to Length(s) do
    Result[i] := Char(Byte(s[i]) xor Byte(x[i]));
end;

function CalcDigest(text: string; dig: TDCP_hashclass): string;
var
  x: TDCP_hash;
begin
  x := dig.Create(nil);
  try
    x.Init;
    x.UpdateStr(text);
    SetLength(Result, x.GetHashSize div 8);
    x.Final(Result[1]);
  finally
    x.Free;
  end;
end;

function CalcHMAC(message, key: string; hash: TDCP_hashclass): string;
const
  blocksize = 64;
begin
  // Definition RFC 2104
  if Length(key) > blocksize then
    key := CalcDigest(key, hash);
  key := RPad(key, #0, blocksize);
  Result := CalcDigest(XorBlock(key, RPad('', #$36, blocksize)) + message, hash);
  Result := CalcDigest(XorBlock(key, RPad('', #$5c, blocksize)) + result, hash);
end;

function PBKDF1(pass, salt: ansistring; count: Integer; hash: TDCP_hashclass): ansistring;
var
  i: Integer;
begin
  Result := pass+salt;
  for i := 0 to count-1 do
    Result := CalcDigest(Result, hash);
end;

function PBKDF2(pass, salt: ansistring; count, kLen: Integer; hash: TDCP_hashclass): ansistring;

  function IntX(i: Integer): ansistring; inline;
  begin
    Result := Char(i shr 24) + Char(i shr 16) + Char(i shr 8) + Char(i);
  end;

var
  D, I, J: Integer;
  T, F, U: ansistring;
begin
  T := '';
  D := Ceil(kLen / (hash.GetHashSize div 8));
  for i := 1 to D do
  begin
    F := CalcHMAC(salt + IntX(i), pass, hash);
    U := F;
    for j := 2 to count do
    begin
      U := CalcHMAC(U, pass, hash);
      F := XorBlock(F, U);
    end;
    T := T + F;
  end;
  Result := Copy(T, 1, kLen);
end;

end.
      
      



- , Pascal , Python, C.





, "" /, interface



. " " - API. , - , ( /, "_" "__").





, , .





("", header)
program devault;
uses
  math, sysutils, strutils, getopts, DCPcrypt2, DCPsha256, DCPrijndael, kdf;
      
      



- hexlify



unhexlify



(, , " "). Python - , - , .





hexlify/unhexlify
function unhexlify(s:AnsiString):AnsiString;
var i:integer;
    tmpstr:AnsiString;
begin
  tmpstr:='';
  for i:=0 to (length(s) div 2)-1 do
    tmpstr:=tmpstr+char(Hex2Dec(Copy(s,i*2+1,2)));
  unhexlify:=tmpstr;
end;

function hexlify(s:AnsiString):AnsiString;
var i:integer;
    tmpstr:AnsiString;
begin
  tmpstr:='';
  for i:=1 to (length(s)) do
    tmpstr:=tmpstr+IntToHex(ord(s[i]),2);
  hexlify:=tmpstr;
end;
      
      



showbanner()



, showlicense()



showhelp()



, .





showbanner() / showlicense() / showhelp()
showbanner()
procedure showbanner();
begin
  WriteLn(stderr, 'DeVault v1.0');
  Writeln(stderr, '(C) 2021, Sergey Pechenko. All rights reserved');
  Writeln(stderr, 'Run with "-l" option to see license');
end;
      
      



showlicense()
procedure showlicense();
begin
  WriteLn(stderr,'Redistribution and use in source and binary forms, with or without modification,');
  WriteLn(stderr,'are permitted provided that the following conditions are met:');
  WriteLn(stderr,'* Redistributions of source code must retain the above copyright notice, this');
  WriteLn(stderr,'   list of conditions and the following disclaimer;');
  WriteLn(stderr,'* Redistributions in binary form must reproduce the above copyright notice, ');
  WriteLn(stderr,'   this list of conditions and the following disclaimer in the documentation');
  WriteLn(stderr,'   and/or other materials provided with the distribution.');
  WriteLn(stderr,'* Sergey Pechenko''s name may not be used to endorse or promote products');
  WriteLn(stderr,'   derived from this software without specific prior written permission.');
  WriteLn(stderr,'THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"');
  WriteLn(stderr,'AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,');
  WriteLn(stderr,'THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE');
  WriteLn(stderr,'ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE');
  WriteLn(stderr,'FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES');
  WriteLn(stderr,'(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;');
  WriteLn(stderr,'LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON');
  WriteLn(stderr,'ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT');
  WriteLn(stderr,'(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,');
  WriteLn(stderr,'EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.');
  WriteLn(stderr,'Commercial license can be obtained from author');
end;
      
      



showhelp()
procedure showhelp();
begin
  WriteLn(stderr,'Usage:');
  WriteLn(stderr,Format('%s <-p password | -w vault_password_file> [-f secret_file]',[ParamStr(0)]));
  WriteLn(stderr,#09'"password" is a text string which was used to encrypt your secured content');
  WriteLn(stderr,#09'"vault_password_file" is a file with password');
  WriteLn(stderr,#09'"secret_file" is a file with encrypted content');
  WriteLn(stderr,'When "-f" argument is absent, stdin is read by default');
end;
      
      



, . , .





var secretfile, passwordfile, pass, salt, b_derived_key, b_key1, b_key2, b_iv,
    hmac_new, cphrtxt, fullfile, header, tmpstr, hmac:Ansistring;
    Cipher: TDCP_rijndael;
    key, vector, data, crypt: RawByteString;
    fulllist: TStringArray;
    F: Text;
    c: char;
    opt_idx: LongInt;
    options: array of TOption;
const KEYLENGTH=32; // for AES256
const IV_LENGTH=128 div 8;
const CONST_HEADER='$ANSIBLE_VAULT;1.1;AES256';
      
      



, - , . - , vars



.





preparecliparams()
procedure preparecliparams();
begin
  SetLength(options, 6);
  with options[1] do
    begin
      name:='password';
      has_arg:=Required_Argument;
      flag:=nil;
      value:=#0;
    end;
  with options[2] do
    begin
      name:='file';
      has_arg:=Required_Argument;
      flag:=nil;
      value:=#0;
    end;
  with options[3] do
    begin
      name:='passwordfile';
      has_arg:=Required_Argument;
      flag:=nil;
      value:=#0;
    end;
  with options[4] do
    begin
      name:='version';
      has_arg:=No_Argument;
      flag:=nil;
      value:=#0;
    end;
  with options[5] do
    begin
      name:='license';
      has_arg:=No_Argument;
      flag:=nil;
      value:=#0;
    end;
  with options[6] do
    begin
      name:='help';
      has_arg:=No_Argument;
      flag:=nil;
      value:=#0;
    end;
end;
      
      



:





begin
  repeat
    c:=getlongopts('p:f:w:lh?',@options[1],opt_idx);
    case c of
      'h','?' : begin showhelp(); halt(0); end;
      'p' : pass:=optarg;
      'f' : secretfile:=optarg;
      'w' : passwordfile:=optarg;
      'v' : begin showbanner(); halt(0); end;
      'l' : begin showlicense(); halt(0); end;
      ':' : writeln ('Error with opt : ',optopt); // not a mistake - defined in getops unit
     end;
  until c=endofoptions;
  if pass = '' then // option -p not set
    if passwordfile <> '' then
      try
        Assign(F,passwordfile);
        Reset(F);
        Readln(F,pass);
        Close(F);
      except
        on E: EInOutError do
        begin
          Close(F);
          writeln(stderr, 'Password not set and password file cannot be read, exiting');
          halt(1);
        end;
      end
    else
      begin // options -p and -w are both not set
          writeln(stderr, 'Password not set, password file not set, exiting');
          showhelp();
          halt(1);
      end;
  try
    Assign(F,secretfile);
    Reset(F);
  except
    on E: EInOutError do
    begin
      writeln(stderr, Format('File %s not found, exiting',[secretfile]));
      halt(1);
    end;
  end;
  readln(F,header);
  if header<>CONST_HEADER then
    begin
      writeln(stderr, 'Header mismatch');
      halt(1);
    end;
  fullfile:='';
  while not EOF(F) do
    begin
    Readln(F,tmpstr);
    fullfile:=fullfile+tmpstr;
    end;
  Close(F);
  fulllist:=unhexlify(fullfile).Split([#10],3);
  salt:=fulllist[0];
  hmac:=fulllist[1];
  cphrtxt:=fulllist[2];
  salt:=unhexlify(salt);
  cphrtxt:=unhexlify(cphrtxt);
  b_derived_key:=PBKDF2(pass, salt, 10000, 2*32+16, TDCP_sha256);
  b_key1:=Copy(b_derived_key,1,KEYLENGTH);
  b_key2:=Copy(b_derived_key,KEYLENGTH+1,KEYLENGTH);
  b_iv:=Copy(b_derived_key,KEYLENGTH*2+1,IV_LENGTH);
  hmac_new:=lowercase(hexlify(CalcHMAC(cphrtxt, b_key2, TDCP_sha256)));
  if hmac_new<>hmac then
    begin
    writeln(stderr, 'Digest mismatch - file has been tampered with, or an error has occured');
    Halt(1);
    end;
  SetLength(data, Length(crypt));
  Cipher := TDCP_rijndael.Create(nil);
  try
    Cipher.Init(b_key1[1], 256, @b_iv[1]);
    Cipher.DecryptCTR(cphrtxt[1], data[1], Length(data));
    Cipher.Burn;
  finally
    Cipher.Free;
  end;
  Writeln(data);
end.
      
      



, , , - .





.













2-13





;









14-34





, - , - ;









35-44





, ;





: ( secretfile) ; Assign(F, secretfile)



36 F stdin







45-50





$ANSIBLE_VAULT;1.1;AES256



;









51-57





;









58-63





: "", , - ; unhexlify ( VaultAES256.encrypt?)









64-73





; ; ; ;









74-83





; ; ; stdout









, , Python 3.10 - case (PEP-634)? , BDFL, 14 , PyCon 2007 PEP-3103 .





, , :





[root@ansible devault]# time fpc devault.pas -Fudcpcrypt_2.1:dcpcrypt_2.1/Ciphers:dcpcrypt_2.1/Hashes -MOBJFPC







, - .





Free Pascal Compiler version 3.0.4 [2017/10/02] for x86_64
Copyright (c) 1993-2017 by Florian Klaempfl and others
Target OS: Linux for x86-64
Compiling devault.pas
Compiling ./dcpcrypt_2.1/DCPcrypt2.pas
Compiling ./dcpcrypt_2.1/DCPbase64.pas
Compiling ./dcpcrypt_2.1/Hashes/DCPsha256.pas
Compiling ./dcpcrypt_2.1/DCPconst.pas
Compiling ./dcpcrypt_2.1/Ciphers/DCPrijndael.pas
Compiling ./dcpcrypt_2.1/DCPblockciphers.pas
Compiling kdf.pas
Linking devault
/usr/bin/ld: warning: link.res contains output sections; did you forget -T?
3784 lines compiled, 0.5 sec

real    0m0.543s
user    0m0.457s
sys     0m0.084s
      
      



: 3,8 0.6 . - , . - . , : 875. , ..





, ! , ".password":





[root@ansible devault]# ./devault -w .password -f vaulted.yml
---
collections:
- name: community.general
  scm: git
  src: https://github.com/ansible-collections/community.general.git
  version: 1.0.0
      
      



YaML .





.





Ansible? (, !)

- , .





Ansible - Ansible, Telegram.








All Articles