Reverse a decryption algorithm with a given .exe GUI

user_1157173 0
Viewed 254

I am using a Keygen application (.exe). There are two input fields in it's GUI:

  1. p1 - at least 1 digit, 10 digits max - ^[0-9]{1,10}$
  2. p2 - 12 chars max - uppercase letters/digits/underscores - ^[A-Z0-9_]{0,12}$

Pressing generate button produce a key x.

x - 20 digits exactly - ^[0-9]{20}$

For each pair (p1,p2), there is only one x (in other words: f(p1,p2) = x is a function)

I am interested in it's encryption algorithm.

Is there any way of reverse engineering the algorithm?

I thought of two ways:

  1. decompiling. I used snowman, but the output is too polluted. The decompiled code probably contains non-relevant parts, such as the GUI.
  2. analyzing of input and output. I wonder if there any option to determine the used encryption algorithm by analyzing a set of f(p1,p2) = x results.
analyzer decompiling cryptography reverse-engineering
3 Answers

As you mentioned, using snowman or some other decompiling tools is probably the way to go.

I doubt you would be able to determine the algorithm just by looking at the input output combinations, since it is possible to write any kind of arbitrary algorithm, that can behave in any way.

Perhaps you could just ask the author what algorithm they're using ?

user_12661090

Unless it's something really simple, I'd rule out your option 2 of trying to figure it out by looking at input and output pairs.

For decompiling / reverse engineering a static binary, you should first determine whether it's a .NET application or something else. If it's written in .NET you can try this for decompilation:

https://www.jetbrains.com/decompiler/

It's really easy to use, unless the binary has been obfuscated.

If the application is not a .NET application, you can try Ghidra and/or Cutter which both has pretty impressive decompilers built in:

https://ghidra-sre.org/

https://cutter.re/

If static code analysis is not enough, you can add a debugger to it. Ghidra and x64dbg work really well together, and can be synced via a plugin installed in both.

If you're new to this, I can recommend both that you look into basic assembler for the x86 platform so you have a general idea of how the CPU works. Another way to get started is "crackme" style challenges from CTF competitions. Often there great write-ups with the solution, so you have both the question and answer available.

Good luck!

user_146322760

Type in p1 and p2. Scan the process for that byte string. Then put a hardware breakpoint for memory access on it. Generate the key, it will hit that hardware breakpoint. Then you have the address which accesses it and start reversing from there in Ghidra(Don't forget to use BASE + OFFSET) since ghidra's output won't have the same base as the running application. The relevant code HAS to access the inputs. So you know where the algorithm is. Since it either directly accesses it, or somewhere within that call chain is accessed relatively fast. Nobody can know without actually seeing the executable.

user_147445470
Related