Hex string produces semi-legible output. How do I find how the rest is encoded and what it means?

user_7336582 0
Viewed 124

I chanced upon a hex string and am wondering what it's about and why it is encoded the way it is.

The hex string is from a cookie from an unknown application (which in itself was found after dumping the already-encoded contents of session.db). It reads:

HEX=0x80037d710028580a0000005f7065726d616e656e74710188580a00000061646d696e5f61757468710288580700000076657273696f6e71035805000000372e342e30710458050000007469746c6571055811000000e5ae9de5a1944c696e7578e99da2e69dbf71065804000000636f6465710788752e

A trivial xxd -r -p reveals some of its contents (actually I hexdump'd the xxd output, the better to show the ASCII part and the remaining hex part):

echo $HEX | xxd -r -p | hexdump -C

00000000  80 03 7d 71 00 28 58 0a  00 00 00 5f 70 65 72 6d  |..}q.(X...._perm|
00000010  61 6e 65 6e 74 71 01 88  58 0a 00 00 00 61 64 6d  |anentq..X....adm|
00000020  69 6e 5f 61 75 74 68 71  02 88 58 07 00 00 00 76  |in_authq..X....v|
00000030  65 72 73 69 6f 6e 71 03  58 05 00 00 00 37 2e 34  |ersionq.X....7.4|
00000040  2e 30 71 04 58 05 00 00  00 74 69 74 6c 65 71 05  |.0q.X....titleq.|
00000050  58 11 00 00 00 e5 ae 9d  e5 a1 94 4c 69 6e 75 78  |X..........Linux|
00000060  e9 9d a2 e6 9d bf 71 06  58 04 00 00 00 63 6f 64  |......q.X....cod|
00000070  65 71 07 88 75 2e                                 |eq..u.|
00000076

Why are some parts legible and some obviously further encoded?

How would I go about identifying how the garbled parts are encoded and what they mean?

reverse-engineering protocols
1 Answers

It looks like an object serialized with Python 3's Pickle, an object serialization library:

import pickle
f = open("/mnt/z/Temp/download.dat", "rb")
data = pickle.Unpickler(f).load()
print(data)

Output:

{'_permanent': True, 'admin_auth': True, 'version': '7.4.0', 'title': '宝塔Linux面板', 'code': True}

How I knew this: Well, it's possible to recognize it from the pattern of how it looks, if you know it. Otherwise, googling the first bytes 80 03 7d 71 00 will also bring up some results where people are talking about files created by Python, and looking at their code will reveal that pickle is library used to write those files.

Probably the web application from which this session object originates is written in Django or some other Python-based web framework.

So, it's not that some parts are "further encoded", it's just that it's a binary format that encodes structured data, and the "readable" parts are the string parts while the the "garbled" parts are just parts of the format that are not directly readable to humans but make sense in the protocol (e.g. defining what type of value something is, how long the next part is, etc.).

By the way, "宝塔Linux面板" ("Pagoda Linux Panel") seems to be an easy-to-use server management panel.

user_18710330
Related