XXHash as a Mysql User Defined Function

Viewed 334

We store representations of millions of chemical compounds as BLOBs in a MySQL database. We also keep hashes of these BLOBs when we need to query among these compounds and comparing these hashes in the queries.

Since we found out that standard hash functions(such as CRC) provided by MySQL library collides frequently for our use-case, we used a custom hash function specific to our data, wrapped it as a MySQL plugin and created a User Defined Function with this plugin as below:

CREATE FUNCTION customhash RETURNS INTEGER SONAME 'customhash.so'

Unfortunately, we need to move our MySQL installation to another managed data centre and because of the security reasons & data centre policy, we are not allowed to customize MySQL by adding plugins.

We've recently heard about the XXHash library, we made a few tests on it and we found out it has great performance and it doesn't generate collisions in our data. Also, it turns out it has already been used by MySQL standard distribution internally.

I wonder if it is possible to configure MySQL server to call XXH64_digest function in our MySQL routines without compiling it as a plugin.

1 Answers

I've checked MySQL source code and built-in functions and I could not find any way to run XXHash in MySQL routines. It seems XXHash is used by MySQL internally and it is not user-visible.

In order to run XXHash in MySQL routines, I have developed a plugin in case anyone needs to use XXHash algorithm in MySQL server.

This plug in can be found here: Github repository for xxhash_mysql_plugin.

After installing plug in you can run the xxhash function in your select statements as below:

enter image description here

Related