Hex digest for Python's built-in hash function

user_1698431 0
Viewed 11552

I need to create an identifier token from a set of nested configuration values. The token can be part of a URL, so – to make processing easier – it should contain only hexadecimal digits (or something similar). The config values are nested tuples with elements of hashable types like int, bool, str etc.

My idea was to use the built-in hash() function, as this will continue to work even if the config structure changes. This is my first attempt:

def token(config):
    h = hash(config)
    return '{:X}'.format(h)

This will produce tokens of variable length, but that doesn't matter. What bothers me, though, is that the token might contain a leading minus sign, since the return value of hash() is a signed integer.

As a way to avoid the sign, I thought of the following work-around, which is adding a constant to the hash value. This constant should be half the size of the range the value of hash() can take (which is platform-dependent, eg. different for 32-/64-bit systems):

HALF_HASH_RANGE = 2**(sys.hash_info.width-1)

Is this a sane and portable solution? Or will I shoot myself in the foot with this?

I also saw suggestions for using struct.pack() (which returns a bytes object, on which one can call the .hex() method), but it also requires knowing the range of the hash value in advance (for the choice of the right format character).

Addendum:
Encryption strength or collisions by chance are not an issue. The drawback of the hashlib library in this scenario is that it requires writing a converter that traverses the input structure and converts everything into a bytes representation, which is cumbersome.

hexdump hash python
3 Answers

I need to create an identifier token from a set of nested configuration values

I came across this question while trying to solve the same problem, and realizing that some of the calls to hash return negative integers.

Here's how I would implement your token function:

import sys


def token(config) -> str:
    """Generates a hex token that identifies a hashable config."""
    # `sign_mask` is used to make `hash` return unsigned values
    sign_mask = (1 << sys.hash_info.width) - 1
    # Get the hash as a positive hex value with consistent padding without '0x'
    return f'{hash(config) & sign_mask:#0{sys.hash_info.width//4}x}'[2:]

In my case I needed it to work with a broad range of inputs for the config. It did not need to be particularly performant (it was not on a hot path), and it was acceptable if it occasionally had collisions (more than what would normally be expected from hash). All it really needed to do is produce short (e.g. 16 chars long) consistent outputs for consistent inputs. So for my case I used the above function with a small modification to ensure the provided config is hashable, at the cost of increased collision risk and processing time:

import sys


def token(config) -> str:
    """Generates a hex token that identifies a config."""
    # `sign_mask` is used to make `hash` return unsigned values
    sign_mask = (1 << sys.hash_info.width) - 1
    # Use `json.dumps` with `repr` to ensure the config is hashable
    json_config = json.dumps(config, default=repr)
    # Get the hash as a positive hex value with consistent padding without '0x'
    return f'{hash(json_config) & sign_mask:#0{sys.hash_info.width//4}x}'[2:]
user_59469210

I'd reccomend using hashlib

cast the token to a string, and then cast the hexdigest to a hex integer. Bellow is an example with the sha256 algorithm but you can use any hashing algorithm hashlib supports

import hashlib as hl
def shasum(token):
    return int(hl.sha256(str(token).encode('utf-8')).hexdigest(), 16)
user_121784320
Related