How to check if a Java String will fit into Cassandra TEXT column before writing it?

Question

We support an application that has some bad design. This application stores data in a Cassandra cluster in a TEXT column and sometimes writes quite large Strings in this column and we get a WriteFailureException.

Cassandra has a limit on the write size (16mb by default: https://docs.datastax.com/en/dse/6.7/dse-admin/datastax_enterprise/config/configCassandra_yaml.html#configCassandra_yaml__max_mutation_size_in_kb) which is great. We would like to notify the user that they are trying to write a large chunk of data in case such limit is reached. As I understand there is no way to distinguish whether this exception occurred because of this limit or due to any other errors inside the Cassandra cluster.

It would be even better to check if the size of the date exceeds the limit before trying to write it in Cassandra. Java String is UTF-16, Cassandra's TEXT is UTF-8, so my naive approach is to convert a String to UTF-8 and check it's size like that: s.getBytes(StandardCharsets.UTF_8).lenght()

However this seems quite expensive to convert a String to UTF-8 just to throw it away. Is there a sane way to do it? How do people check if their data fits in Cassandra before writing it?

Java 8, Cassandra 3.11

Alex_Ott_18627 · Answer

The better way is to check the size not of the individual strings, but the size of the request, because it's also dependent on the protocol version. If you're using prepared statements, then you can bind values and then call requestSizeInBytes on the bound statement (for driver 3.x), like this (source code)

int stmtSize = boundStatement.requestSizeInBytes(protocolVersion, codecRegistry);

For driver 4.x it's the computeSizeInBytes function (doc)

But take into account that it's approximate size, but it could be quite good approximation anyway

How to check if a Java String will fit into Cassandra TEXT column before writing it?

1 Answers