HBase - value filter with offset?

Question

Let say I have three rows with following values:

+------+----------+
| row  | f1:c1    |
+------+----------+
| row1 | \x00\x00 |
| row2 | \x01\x00 |
| row3 | \x03\x01 |
+------+----------+

Is it possible to select rows with \x00 as second byte of the value (eg. row1 and row2)?

Further explanation

I have an immutable object that consists of couple UUIDs as a part of my entity. Since UUID has fixed length the most efficient way of storing it is to concat all parts into a single byte array and store it in a single column.

However, I must be able to select rows based on specific field of said object. Which in theory is pretty simple: all I need to do is take my column value at specific offset and compare next 16 bytes against search value.

In fact ByteArrayComparable already kinda works that way. It takes an offset that points to the start of the value, and it seems that all I need is to add additional offset on top of that. But I cannot figure out how to do that.

All in all, this seems to me like a widely applicable use case. So there must be a way to do it, unless I am missing something.

P.S. I know that I can probably achieve what I want with RegexStringComparator but this seems wildly inefficient.

UPDATE

HBase supports custom filters which is perfect for my situation. Unfortunately all documentation I can find seems outdated since hbase.dynamic.jars.dir and hbase.use.dynamic.jar are not present in my configuration (my version is 2.0.1).

UPDATE 2

I managed to solve this with custom filter. It appears that they removed hbase.dynamic.jars.dir and hbase.use.dynamic.jar, but simply placing filter on the classpath works fine.

If anyone willing to write an answer on how to implement and use custom filter I will gladly award the bounty.

user_2137378 · Answer

Firstly, let's know more about custom filters to answer this sentence:

P.S. I know that I can probably achieve what I want with RegexStringComparator but this seems wildly inefficient.

custom filters can be used within a scan operation in HBase. When such a scan operation executes from your application for example which is on spark the executor uses an RPC connection to connect to the underlying region server and the region server uses the same type of connection for fetching data from data-node. But the question is that where the custom filters are applied? on your application? Of course NO. custom filters are applied on rows at region servers and only the matched ones are coming up to your application. Furthermore, it means that using these kinds of filters would help much in solving performance issues.

Secondly, if there is a need to select some rows based on their values you can use different kinds of filters but SingleColumnValueFilter may be much more useful in working with values. A complete list of custom filters is presented here. Additionally, RegexStringComparator can be used as SingleColumnValueFilter comparator and here is an example:

RegexStringComparator regexStringComparator=
                new RegexStringComparator(regexPattern);
SingleColumnValueFilter singleColumnValueFilter=
                new SingleColumnValueFilter(family, qualifier, 
                               CompareOp.EQUAL, regexStringComparator);

HBase - value filter with offset?

1 Answers