Using JtR to crack SHA1 with prefix and suffix

27 September 2014

Introduction

This post is about cracking hashes that have been generated using the format: sha1(<prefix> $password <suffix>). For instance, sha1("example.org,my_password,0"). Although John the Ripper does not offer such function by default, it is relatively easy to implement it using a dynamic format. We will be using the bleeding-jumbo flavour of JtR.

Dynamic format

To create a new hashing schema, you may define a 'dynamic' section into your JtR configuration. Using john.local.conf, you will avoid any merging conflict when updating. Here is the header of definition for our hashing:

[List.Generic:dynamic_8180]
Expression=sha1(CONST1.$p.CONST2)

We are going to use constants for both prefix and suffix (for now, let's assume these are common for all hashes):

CONST1=prefix,
CONST2=,suffix

Then comes the body of our hashing:

Func=DynamicFunc__clean_input
Func=DynamicFunc__append_input1_from_CONST1
Func=DynamicFunc__append_keys
Func=DynamicFunc__append_input1_from_CONST2
Func=DynamicFunc__SHA1_crypt_input1_to_output1_FINAL

To validate that our hashing operates as expected, JtR requires to set up some test cases.

Test=$dynamic_8180$2160a6a444745e385cad5283e28900ebc7d6f90b:password

You could use something similar to the following to generate such tests.

$ echo -n "prefix,password,suffix" | sha1sum

Now if you run such dynamic you will get errors. Two options are necessary to make this schema working.

SHA1 length

The dynamic format has initially been designed to handle MD5 composites. It will expect a default hash size of 16 bytes for the tests. Since SHA1 produces 20 bytes long hashes, we need to use the following flag:

Flag=MGF_INPUT_20_BYTE

It should be noted that, only the first 16 bytes will be compared with our test case. As mentioned in the documentation: "This 'should' be more than adequate to know you have cracked the hash."

x86 vs SSE2

If your version of JtR is built for basic x86, you shouldn't have to do anything more. If using SSE2 or later (AVX, XOP or x86_64), an error will still be produced when run. To understand what is happening, let's add a bit of debugging to dynamic_fmt.c (line 1613):

#if 0
// Dump state (for debugging help)
printf ("\nState after function: %s\n", dynamic_Find_Function_Name(curdat.dynamic_FUNCTIONS[i]));
// dump input 1
#ifdef MMX_COEF
dump_stuff_mmx_msg("input_buf[0]", input_buf[0].c, 64, 0);
dump_stuff_mmx_msg("input_buf[1]", input_buf[0].c, 64, 1);
dump_stuff_mmx_msg("input_buf[2]", input_buf[0].c, 64, 2);
dump_stuff_mmx_msg("input_buf[3]", input_buf[0].c, 64, 3);
#endif
printf ("input_buf86[0] : %*.*s\n", total_len_X86[0],total_len_X86[0],input_buf_X86[0].x1.b);
printf ("input_buf86[1] : %*.*s\n", total_len_X86[1],total_len_X86[1],input_buf_X86[1].x1.b);
printf ("input_buf86[2] : %*.*s\n", total_len_X86[2],total_len_X86[2],input_buf_X86[2].x1.b);
printf ("input_buf86[3] : %*.*s\n", total_len_X86[3],total_len_X86[3],input_buf_X86[3].x1.b);

Replace the #if 0 to #if 1 and recompile (you'll need to disable OpenMP too). When executed:

$ ./john --test -form=dynamic_8180
Benchmarking: dynamic_8180 [sha1(CONST1.$p.CONST2) 128/128 AVX 10x4x2]... 
[...]
State after function: DynamicFunc__SHA1_crypt_input1_to_output1_FINAL
input_buf[0] : 70726566 69782c70 61737377 6f72642c 73756666 69788000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
input_buf[1] : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
input_buf[2] : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
input_buf[3] : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
input_buf86[0] : 
input_buf86[1] : 
input_buf86[2] : 
input_buf86[3] : 
crypt_key[0] : 00000000 00000000 00000000 00000000 
crypt_key[1] : 00000000 00000000 00000000 00000000 
crypt_key[2] : 00000000 00000000 00000000 00000000 
crypt_key[3] : 00000000 00000000 00000000 00000000 
crypt_key_X86[0] : eea339da 0d4b6b5e efbf5532 90186095 
crypt_key_X86[1] : eea339da 0d4b6b5e efbf5532 90186095 
crypt_key_X86[2] : eea339da 0d4b6b5e efbf5532 90186095 
crypt_key_X86[3] : eea339da 0d4b6b5e efbf5532 90186095

The input buffer contains what we would expect but not the crypt_key_X86 array. It turns out that this value is:

$ echo -n "" | sha1sum 
da39a3ee5e6b4b0d3255bfef95601890afd80709  -

The answer for this mystery is in the final crypt function:

void DynamicFunc__SHA1_crypt_input1_to_output1_FINAL(DYNA_OMP_PARAMS){
  int i, til;

#ifdef _OPENMP
  i = first;
  til = last;
#else
  i = 0;
  til = m_count;
#endif
  for (; i < til; i += sha1_inc) {
#ifdef SHA1_SSE_PARA
  int len[SHA1_LOOPS], j;
  for (j = 0; j < SHA1_LOOPS; ++j)
    len[j] = total_len_X86[i+j];
  DoSHA1_crypt_f_sse(input_buf_X86[i>>MD5_X2].x1.b, len, crypt_key_X86[i>>MD5_X2].x1.b);
#else
#if (MD5_X2)
    if (i & 1)
      DoSHA1_crypt_f(input_buf_X86[i>>MD5_X2].x2.b2, total_len_X86[i], crypt_key_X86[i>>MD5_X2].x2.b2);
    else
#endif
    DoSHA1_crypt_f(input_buf_X86[i>>MD5_X2].x1.b, total_len_X86[i], crypt_key_X86[i>>MD5_X2].x1.b);
#endif
  }
}

Since 4dbaae, the SHA family hashes have been modified to use SSE as much as possible. By default, algorithm that uses SSE are expecting to fill up different buffers (input_buf) rather than the traditional x86 version (input_buf_X86). As we've seen above, SHA1 is actually using the old x86 buffers even when run for SSE. As a result, we are filling up the wrong buffer and hashing nothing.

To avoid this, we need to add the following flag to our format:

Flag=MGF_FLAT_BUFFERS

Maximum length

It is important to keep in mind that John has a hardcoded maximum for the input buffers. When using the x86 buffers, it is 125 bytes. This is not much of an issue for the passwords themselves but could be tricky for long prefix or suffix. Let's add this information to our format:

MaxInputLen=110 # Must be 125 - len(CONST1) - len(CONST2)

Final format

All together, the final format is:

[List.Generic:dynamic_8180]
Expression=sha1(CONST1.$p.CONST2)
MaxInputLen=110 # Must be 125 - len(CONST1) - len(CONST2)
CONST1=prefix,
CONST2=,suffix
Flag=MGF_INPUT_20_BYTE
Flag=MGF_FLAT_BUFFERS
Func=DynamicFunc__clean_input
Func=DynamicFunc__append_input1_from_CONST1
Func=DynamicFunc__append_keys
Func=DynamicFunc__append_input1_from_CONST2
Func=DynamicFunc__SHA1_crypt_input1_to_output1_FINAL
Test=$dynamic_8180$2160a6a444745e385cad5283e28900ebc7d6f90b:password
Test=$dynamic_8180$87fa43f93f272d53f84958efccee2b2438548b66:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

Using salts

An alternate solution would be to use the prefix as a first salt and the suffix as a second salt. This is particularly adapted to hashes that have different prefixes and suffixes.

[List.Generic:dynamic_8181]
Expression=sha1($s.$p.$s2)
Flag=MGF_INPUT_20_BYTE
Flag=MGF_FLAT_BUFFERS
Flag=MGF_SALTED
Flag=MGF_SALTED2
Func=DynamicFunc__clean_input
Func=DynamicFunc__append_salt
Func=DynamicFunc__append_keys
Func=DynamicFunc__append_2nd_salt
Func=DynamicFunc__SHA1_crypt_input1_to_output1_FINAL
Test=$dynamic_8181$644dfffd8ee97b8cb8a7019a8a53cb0c4eec5efe$prefix,$$2,suffix:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA