cryptography on Stephen Swanson

Cracking RSA: The Hard Way

stephen@stephenswanson.xyz (Stephen Swanson) — Sat, 01 Jan 2022 14:09:30 -0700

Ever since I finished my OSCP in November of 2021, I haven’t done much in the field of cybersecurity. Since I’m awfully rusty, I decided to start from the very basics and play around with the HTB beginner track. One of the more interesting challenges I finished has been the Weak RSA challenge, which involved generating a private key from a public key with a known factorization. RSA is incredibly common, in fact, the connection to this very website has been secured by RSA (check the certificate details), but I never really explored the underlying math.

The Challenge

The challenge consists of a zip archive containing two files, key.pub and flag.enc. flag.enc is the flag encrypted using RSA, and key.pub is a RSA public key in PEM format.

# key.pub
-----BEGIN PUBLIC KEY-----
MIIBHzANBgkqhkiG9w0BAQEFAAOCAQwAMIIBBwKBgQMwO3kPsUnaNAbUlaubn7ip
4pNEXjvUOxjvLwUhtybr6Ng4undLtSQPCPf7ygoUKh1KYeqXMpTmhKjRos3xioTy
23CZuOl3WIsLiRKSVYyqBc9d8rxjNMXuUIOiNO38ealcR4p44zfHI66INPuKmTG3
RQP/6p5hv1PYcWmErEeDewKBgGEXxgRIsTlFGrW2C2JXoSvakMCWD60eAH0W2PpD
qlqqOFD8JA5UFK0roQkOjhLWSVu8c6DLpWJQQlXHPqP702qIg/gx2o0bm4EzrCEJ
4gYo6Ax+U7q6TOWhQpiBHnC0ojE8kUoqMhfALpUaruTJ6zmj8IA1e1M6bMqVF8sr
lb/N
-----END PUBLIC KEY-----

TL;DR

After finishing the challenge, I found several people who posted writeups. (See here and here), but most of them focused on using a tool to automate the attack. Instead, I wanted to do it by hand. For completeness sake though, after extracting the files, this command will solve the challenge:

$ podman run -it --rm -v $PWD:/data docker.io/razaborg/rsactftool --uncipher /data/flag.enc --publickey /data/key.pub

I’m using podman here, since it’s more secure by design than docker, but you can simply swap podman with docker and the command should work as is, since they’re drop-in replacements of each other.

After running the command, the program spits out the flag:

RSA: The Nitty Gritty

I wanted to know how the tool got the answer though, so I started digging into how RSA works under the hood.

RSA is an asymmetric encryption algorithm, meaning that there are two keys used, one for encryption, and one for decryption. The security of RSA depends on the idea that it’s hard to factor the product of two prime numbers, but it’s easy to multiply those numbers.

A public RSA key has two parts, a modulus and a public exponent. typically noted as n and e, respectively. A private key only needs to contain a private exponent, d. This formula, where m is the plaintext message and c is the encrypted message, is used for encryption:

$$ c = m^e\text{ mod } n $$

while:

$$ m = c^d \text{ mod } n $$

is used for decryption.

As an example ¹ imagine we wanted to encrypt the message “The attack will come at dawn” First, we would pick two random prime numbers that are roughly the same size, for example, p = 79 and q = 97.

n, our modulus, is

$$n = np = 79\cdot97 = 7663$$

Next, we pick a random public exponent. Typically, this number is prime, since we must ensure that the exponent is not a factor of

$$(p -1)(q-1) = 7488$$

and picking a prime number simplifies that calculation. For this example, we pick e = 23.

Lastly, we find d using this equation:

Now, we have our public key: 23,7663, and our private key: 2279, 7663. We share the public key with the person sending the message. They then convert the message into a number using a tool like CyberChef²:

$$\text{The attack will come at dawn} = 84 104…97 119 110$$

Since the message must be shorter than the modulus, we break it into 33 three digit chunks:

Now, we raise each chunk to the power of e modulus n

Python handles this math really easily using the pow() function. To perform the encryption, you can simply do c1 = pow(841, 23, 7663), where 841 is the number, 23 is the exponent, and 7663 is the modulus. For small numbers using the modulus operator % would work, but for large numbers, using the extra argument to the pow function makes it much easier.

This leaves us with our encrypted string ³! To decrypt, simply repeat the process, but use the private exponent d instead of e:

With the successful decryption, we can now securely communicate over an insecure channel. As long as d is kept secret, and we are unable to factor the modulus, n, this message will stay secure.

Breaking RSA, the old fashioned way

Now that we have an understanding of how RSA works, we can begin attacking this key. In order to decrypt the message, we need to find d. The first step is to see the components of the key. I used the Python library PyCryptodome. To see the components, we just import the key and PyCryptodome will expose the variables:

from Crypto.PublicKey import RSA

with open('key.pub', 'r') as keyfile:
    key = RSA.import_key(keyfile.read())

print ("e:", key.e)
print ("n:", key.n)

Running the program:

Since we know the modulus, we can check a database to see if this particular number has known factors. For this, I’ll use factordb.com. After pasting the modulus into factordb, we see that the two factors of this number are known:

We can copy those into our python code as p and q from the equations above. Now, we know e (which is part of the public key), p, and q, so we can calculate d, the secret key, using one of the above equations!

from Crypto.PublicKey import RSA

with open('key.pub', 'r') as keyfile:
    key = RSA.import_key(keyfile.read())

p = 20423438101489158688419303567277343858734758547418158024698288475832952556286241362315755217906372987360487170945062468605428809604025093949866146482515539
q = 28064707897434668850640509471577294090270496538072109622258544167653888581330848582140666982973481448008792075646342219560082338772652988896389532152684857
e = key.e
n = key.n

d = pow(e,-1,((p-1) * (q-1))) # e^-1 mod (p-1)(q-1)

print("d:", d)

Now that we have the secret key, we simply have to use our decryption formula! Since the message is much smaller than the key, we don’t have to worry about chunking.

from Crypto.PublicKey import RSA

with open('key.pub', 'r') as keyfile:
    key = RSA.import_key(keyfile.read())

p = 20423438101489158688419303567277343858734758547418158024698288475832952556286241362315755217906372987360487170945062468605428809604025093949866146482515539
q = 28064707897434668850640509471577294090270496538072109622258544167653888581330848582140666982973481448008792075646342219560082338772652988896389532152684857
e = key.e
n = key.n

d = pow(e,-1,((p-1) * (q-1))) # e^-1 mod (p-1)(q-1)

# Thanks to https://stackoverflow.com/a/62986942 for the decryption code.
with open('flag.enc', 'rb') as flagfile:
    encflag = int.from_bytes(flagfile.read(), 'big')
    pt_int = pow(encflag,d,n) # m^d mod n
    plaintext = pt_int.to_bytes(128, 'big').lstrip(b'\x00')
    print(plaintext)

Finally, we decrypt the flag:

Conclusion

RSA is an incredible invention. It’s strong enough to be used to secure countless SSH or HTTPS connections, but underneath is a small bit of elegant math that college freshman can understand. It’s honestly amazing to me that the simple exponentiation that I was bored with in my college algebra class is used as the foundation of internet security.

As always, if this was interesting or if you learned a trick or two. Please drop me a line. I always appreciate feedback!

Thanks to Wikipedia and Bruce Schneier’s book “Applied Cryptography” for all of this information and these examples. ↩︎
The number is shortened for readability. This is the full chunked number: 841 041 013 297 116 116 979 910 732 119 105 108 108 329 911 110 910 132 971 163 210 097 119 110 ↩︎
Again, here’s the full encrypted number: 579 4278 4986 6386 1497 1497 1024 7201 1444 5022 885 432 432 2695 7351 2367 7201 4329 292 1236 3680 2037 5022 2367 ↩︎

Breaking Santa's Encryption for Fun and Profit

stephen@stephenswanson.xyz (Stephen Swanson) — Wed, 29 Apr 2020 13:53:30 -0700

Recently, I’ve had more time on my hands, so I decided to take a walk down memory lane and rediscover my favorite challenge of the 2019 SANS Holiday Hack Challenge, number 10: the Elfscrow Crypto challenge.

In order to solve this challenge, I relied heavily on this talk by Ron Bowes, and most of this was based on his methods. I highly recommend watching the video, especially if anything here is confusing.

After downloading everything, we end up with three files:

elfscrow.exe, obviously the encryption program.
elfscrow.pdb, the debugging symbols.
ElfUResearchLabsSuperSledOMaticQuickStartGuideV1.2.pdf.enc, this looks like an encrypted PDF based on the filename.

We’re dealing with a Windows binary, so I decided to reverse engineer this application in a virtual machine. I used Flare VM, since it’s built for this, however, you could probably get away with wine or any other Windows system. A quick note about using Flare: although it’s recommended to detach the VM from the internet, this program needs to connect back to the North Pole API servers in order to run, so you’ll have to connect your VM to the network.

Let’s open up our VM and just play with the program. This is one of the most important steps, at least for me; I just run the program with different options and different inputs, and just take note of the “huh” moments. When I was playing around with the program, at least at the beginning, I used an ASCII text file. I figured it’d be easier to understand the mechanism with a human readable file.

I played around for a little bit, encrypted the same file a few times in quick succession, and ended up with output that looked something like this. I’ve taken the liberty of highlighting the parts of the output that were interesting:

Quick note before moving on, you might notice an HTTP warning. I had to use –insecure to force HTTP, since, as of this writing, there’s a cert error that causes the program to fail.

I’ve highlighted two things. First, a major red flag is that the seed only changed slightly, from 1588005546 to 1588005549. This indicates that the random seed is based on the time, and not some other random data. It even looks similar to a timestamp, which is quickly confirmed by decoding one of the seeds into a date:

$ date -d @1588005546
Mon 27 Apr 2020 09:39:06 AM PDT

In fact, the seed for the encryption is nothing more than the current time!

Secondly, the key and key length are interesting to us. Instead of creating their own encryption algorithm, developers often rely on libraries that securely implement other algorithms, like AES. This is much more secure, since there’s less room for error on the programmers end, however, if we can identify the algorithm, it can make it easier for us to defeat it, since we can take advantage of libraries too.

One way we can identify the algorithm is by looking at the key length. AES typically uses keys of around 16-32 bytes, while Blowfish can use between 4-56 bytes¹, and DES uses 8 bytes.

Looking at the picture, we can see that the key is 8 bytes long, so it could be Blowfish, DES, or some other algorithm. I’m investigating DES first, since it’s more common.

In order to determine the algorithm, I’m using a website that can encrypt and decrypt DES, Blowfish, and AES. I like http://des.online-domain-tools.com because we can play with the options easily. Eventually, we figure out that the algorithm is indeed DES in CBC mode:

Everything looks good, except for the first 8 bytes. Let’s change the initial vector to null bytes:

I stumbled into a little rabbit hole here. I knew that DES is considered weak and is deprecated, so I tried to figure out if there was a simple attack against it. I found out, though, that it would still take days to crack, so it’d be unfeasible for a CTF challenge.

Based on what we know so far, we can draw a diagram that represents the algorithm behind Santa’s encryption:

Now we can finally make a plan for approaching the actual challenge. I like breaking it down into small components that can stand on their own. I.e, that I can test before moving on, so that I minimize the amount of code I have to debug at once.

Looking at the diagram, there are three things our decryption program has to do:

It has to decrypt DES using a key.
It has to generate that key using a timestamp.
It has to brute force the timestamp, since we only know roughly when the file was encrypted (December 6, 2019, between 7pm and 9pm UTC)

In essence, we’re stepping backwards through the diagram, reversing each element as we come across it.

We can finally start a python script to decrypt our file. I’m using PyCryptodome for a DES implementation. We can pull in our known options to make a little function like this:

from Crypto.Cipher import DES

def decrypt(key, filebytes):
    initialValue = bytes.fromhex("0000000000000000")
    santasCipher = DES.new(key, DES.MODE_CBC, iv=initialValue)

    return santasCipher.decrypt(filebytes)

The reason why we we split it up into separate functions is that we can test it easily, so let’s do that:

import sys

with open("12bugs1.enc",'rb') as encryptedfile:
    key = bytes.fromhex('04b3fd0a56885f80')
    sys.stdout.buffer.write(decrypt(key, encryptedfile.read()))

Here, we convert the key which we got from the testing of the Elfscrow binary to a byte object. Then we decrypt the file and write it to the screen. When we run this, we get the plaintext of our file.

Since we’re using python3, the only way to write raw bytes without decoding them or including the \x prefix is to write directly to stdout. That’s why I’m using sys.stdout.buffer.write() instead of print(). Problems with encoding and decoding cost me a few hours when I was first solving this.

That’s the easy part. Now we have to figure out how to get the hex key from the seed. For that, we need to look at the disassembly.

I like using Ghidra, it comes preinstalled with Flare VM, but java has to be installed manually. In addition, since we’re working with a .pdb file, we have to register the DIA SDK.

After we have the Elfscrow binary open in Ghidra, we can start poking around for interesting functions. Since we have a plan that we’re following, we can safely ignore much of main, and instead look for the functions that involve cryptography. I’m not great at reading assembly, so my reversing is partially based on the decompiler. After a little bit of investigation, I found:

generate_key, which appears to take a reference to an array, and fills that array with a generated key.
super_secure_srand, which looks to be a custom implementation of the srand function. It’s called with an integer to seed the random number generator.
super_secure_random, just returns a semi-random integer and updates the seed.

We need to implement all three of these functions in our decoder script in order to solve the challenge. Let’s start with super_secure_srand because it seems to be the foundation for the other two functions:

 super_secure_srand(int param_1)
    00401d9r   PUSH   EBP
    00401d91   MOV    EBP,ESP
    00401d93   MOV    EAX,dword ptr [EBP + param_1] // Loading the first parameter from the function call
    00401d96   PUSH   EAX
    00401d97   PUSH   s_Seed_=_%d_004042e8 // Formatting for fprintf
            
    00401d9c   CALL   dword ptr
            
    00401da2   ADD    EAX,0x40
    00401da5   PUSH   EAX                                          
    00401da6   CALL   dword ptr [->MSVCR90.DLL::fprintf]
            
    00401dac   ADD    ESP,0xc
    00401daf   MOV    ECX,dword ptr [EBP + param_1] // Putting the first parameter into ECX
    00401db2   MOV    dword ptr [DAT_0040602c],ECX // Moving the first parameter into the variable [DAT_0040602c]
            
    00401db8   POP    EBP
    00401db9   RET
    00401dba   align  align(6)

We can see that this function prints the new seed, and then sets it in a global variable. This is easily recreated in python with:

seed = 0

def super_secure_srand(newSeed):
    #print("Seed = " + newSeed)
    global seed
    seed = newSeed

Next, lets take a peak at super_secure_random:

 super_secure_random(void)
    00401dc0    PUSH   EBP
    00401dc1    MOV    EBP,ESP
    00401dc3    MOV    EAX,[DAT_0040602c]
             
    00401dc8    IMUL   EAX,EAX,0x343fd
             
    00401dce    ADD    EAX,0x269ec3
             
    00401dd3    MOV    [DAT_0040602c],EAX
             
    00401dd8    MOV    EAX,[DAT_0040602c]
             
    00401ddd    SAR    EAX,0x10
    00401de0    AND    EAX,0x7fff
             
    00401de5    POP    EBP
    00401de6    RET
    00401de7    align  align(9)

This function is a little bit more complicated, but we can take each instruction piece by piece and understand it.

First we make a local copy of the global seed in the EAX register. Then we multiply our local copy by 0x343fd and add 0x269ec3 to it. Next, we save our local seed as the global seed. Lastly, we shift our local copy to the right by 0x10 and perform an AND against 0x7fff. We then return our local copy.

It’s confusing to think about in assembler, but implemented in python, it looks like this:

def super_secure_random():
    # Mutating the original seed
    global seed
    newSeed = seed
    newSeed = newSeed * 0x343fd
    newSeed = newSeed + 0x269ec3
    # Saving a changed seed
    seed = newSeed

    # Mixing it up a little more
    newSeed = newSeed >> 0x10
    newSeed = newSeed & 0x7fff
    return newSeed

The arithmetic can be greatly simplified, but it’s easiest for me to track the binary operations if it’s all broken up into as many steps as possible.

generate_key is a little more complicated. The function graph looks like this:

If we break it up into separate pieces, however, and take each piece individually, we can still figure out what it does. In the first block, we can skip most of it until the call to time, since that’s the first place we see a value we’re interested in. We then head into a small set of instructions that call super_secure_srand with the time:

  00401e0e   CALL  time
  00401e13   ADD   ESP,0x4
  00401e16   PUSH  EAX
  00401e17   CALL  super_secure_srand
          
  00401e1c   ADD   ESP,0x4
  00401e1f   MOV   dword ptr [EBP + local_8],0x0
          
  00401e26   JMP   LAB_00401e31

Then we initialize the variable local_8 to 0. If we look back at the flow graph, we seem to be in a for loop. Although the picture doesn’t show it, every loop the value is compared to 0x8. In C++, it would look like this:

for (int i = 0; i <8; i++) {
    // loop interior
}

Now we can look at the interior of the loop:

  00401e37   CALL   super_secure_random
          
  00401e3c   MOVZX  ECX,AL
  00401e3f   AND    ECX,0xff
          
  00401e45   MOV    EDX,dword ptr [EBP + param_1]
  00401e48   ADD    EDX,dword ptr [EBP + local_8]
  00401e4b   MOV    byte ptr [EDX],CL
  00401e4d   JMP    LAB_00401e28

We call super_secure_random, but only take the last 8 bits, which we AND with 0xff. Once we have the new value, we make it the ith element in an 8 character long list.

Basically, we generate a byte for each element in our 8 byte key.

All told, the key generation algorithm looks like this in python:

def generate_key(randomSeed):
    keystring = ""
    super_secure_srand(randomSeed)
    for i in range(8):
        keyElement = super_secure_random().to_bytes(10,byteorder="little")[0]
        keyElement = keyElement & 0xff
        keystring = keystring + format(keyElement, '02x')
    return keystring

If we wanted to be 100% true to the program, we’d set randomSeed equal to the current timestamp, but since we’re attempting to decrypt it, we need to have control over the variable.

Let’s test what we have so far. If we put in the seed 1588005546, and everything works right, we should get the key of the first run, 04b3fd0a56885f80:

print(generate_key(1588005546))
# output: 04b3fd0a56885f80

And sure enough, we get the right hex value! Now, given the time the program was run, we can decrypt any document.

We just have one more step: bruteforce the seed used for our PDF.

There’s a slight problem though, between 7 and 9pm, there are 7200 seconds, which means 7200 different binary blobs, and looking for our PDF is akin to looking for a needle in a haystack, unless there’s a way to tell them apart.

Luckily, at the beginning of most files, there’s an identifier. These are the “magic bytes” of a file, and it’s how the file utility can tell a PDF from a JPG. The bytes we’re looking for are 25 50 44 46, or %PDF when they’re decoded.

What we need to do is attempt decryption with every timestamp in the range, and check the first 4 bytes for the PDF magic number. First, lets get the two timestamps we’re interested in:

$ date -d "2019-12-06 7:00 PM UTC" "+%s"
1575658800
$ date -d "2019-12-06 9:00 PM UTC" "+%s"
1575666000

Now we just need to loop over every number between them and attempt to decrypt our file. Here’s the python code:

def bruteforceFileSeed(filebytes, firstTimestamp, secondTimestamp):
    #Using tqdm to display a fancy progress bar as we loop through every time between our two timestamps
    for seed in tqdm(range(firstTimestamp, secondTimestamp)):
        key = generate_key(seed)
        decryptedFilebytes = decrypt(bytes.fromhex(key), filebytes)

        #Checking for PDF magic number
        if decryptedFilebytes[0:4] == bytes.fromhex("25504446"):
            print("Success with key:", key, "and seed", seed)
            return decryptedFilebytes

    #If we haven't found anything in the range, exit with an error
    print("Failure!")
    return None

with open(sys.argv[1], 'rb') as encryptedfile:
    #Begin the process
    decryptedFilebytes = bruteforceFileSeed(encryptedfile.read(), 1575658800, 1575666000)

    #If we've succeeded, write our decrypted bytes to the second file
    if decryptedFilebytes != None:
        with open(sys.argv[2], 'wb') as decryptedfile:
            decryptedfile.write(decryptedFilebytes)

Once we have all of that in our script, we’re finally ready to attempt to decrypt our file:

After about two minutes, our loop breaks with the success message. The file was encrypted with the key b5ad6a321240fbec at 1575663650. Even better than that, we have a decrypted file waiting for us! Let’s open up ElfUResearchLabsSuperSledOMaticQuickStartGuideV1.2.pdf and see if it worked:

We’ve completed the challenge! We have the plaintext of the document, and if we look at the middle line of the first page, we see that the flag for this challenge was “Machine Learning Sleigh Route Finder”. The completed python script can be found here.

If this write up was interesting or if you learned something, please drop me a line! It’s always great learning where I can improve or answering any questions about my methods.