Compress string in Sql server and decompress in javascript

Viewed 2862

I want to transfer a long string from my database to my webpage. So, I want to try the approach of compressing my string in my server and decompress it in client side.

So far I have this in my sql server codes:

select compress('this is just a sample string')

which returns this:

0x1F8B08000000000004002BC9C82C5600A2ACD2E212854485E2C4DC829C5485E292A2CCBC7400206D53921C000000

Now I need a function in my javascript to revert the compression operation :

var str = "0x1F8B08000000000004002BC9C82C5600A2ACD2E212854485E2C4DC829C5485E292A2CCBC7400206D53921C000000";
alert(decompressed(str));

which should alert "this is just a sample string".

4 Answers

First, let me explain the difference between how VARBINARY(MAX) is stored and displayed in SSMS.
String "0x1F8B0800000000000..." is a hexadecimal representation created by SSMS of varbinary data stored in the database. The underlain binary value is mostly unprintable characters. So if you try to use this string in your java code it just can't work directly.

I created an example here, if you press "STEP" button, you will see real value: HexToBin

That is why you second example will not work, you need to read byte array from database and then use this byte array in zlib.

There is a good example how to use deflate in java: Java Decompress a string compressed with zlib deflate

Maybe you want something like this: https://jsfiddle.net/58mgsy9a/

const parseString = str => {
    const strWithoutprefix = str.slice(2);
    const array = strWithoutprefix.match(/.{1,2}/g);
    return array.map(pair => parseInt(pair, 16));
};

const decompressed = gzipArray => {
    const gunzip = new Zlib.Gunzip(gzipArray);
    const plain = gunzip.decompress();
    return String.fromCharCode(...plain);
};

var str = "0x1F8B08000000000004002BC9C82C5600A2ACD2E212854485E2C4DC829C5485E292A2CCBC7400206D53921C000000";
alert(decompressed(parseString(str)));

The parseString function takes your input string and converts it to an array of numbers, because zlib expects an array of numbers as the input. Then the decompressed function uses zlib to unzip the array. The decompress function of zlib returns an array of number, so we convert it back to a string.

When You run this query: select compress('this is just a sample string') (for instance using MSSQL Server Management Studio) You will see a text representation in hexadecimal format of the binary compressed text.

I did a test with a lorem ipsum long text casted as varchar and nvarchar as well (You may notice that inside that text there aren't double-byte characters, but it doesn't matter for this example):

This compressed long text is stored inside a varbinary(max) column inside MSSQL:

--------VARCHAR----------    --------NVARCHAR--------
Original     Hex   Binary    Original    Hex   Binary
   13046    4024     2011       13046   4748     2373

So, the best choice here is to transfer from the back-end the binary data, not the hexadecimal representation.

Tell the browser that the response is gzipped:

SERVER SIDE: Here is an example endpoint in PHP :

<?php  /* getcompressed.php */
    header("Content-Encoding: gzip");
    header("Vary: Accept-Encoding");
    header("Content-type: text/html; charset=UTF-16"); /* nvarchar */
    $serverName = 'testserver';
    $connectionOptions = array('Database'=>'testdatabase');
    $conn = sqlsrv_connect($serverName, $connectionOptions);  
    $sql = 'SELECT compressed FROM tb_compression WHERE id = 1';
    $qry = sqlsrv_query($conn, $sql);
    $row = sqlsrv_fetch_array($qry);
    $compressed = $row[0];
    sqlsrv_close($conn);
    echo $compressed;
?>

CLIENT SIDE: here is a working snippet in JavaScript:

var request = new XMLHttpRequest();
request.onload = function(e) {
    var text = e.currentTarget.responseText;
    console.log(text); /* Here You go */
};
request.responseType = 'text';
request.open('GET', 'getcompressed.php', true);
request.send(null);

This works for me in Chrome and FF as well. Try it out.

Using a gunzip javaScript Library:

SERVER SIDE: Here is an example endpoint in PHP :

<?php  /* getcompressed.php */
    header('Content-Type: application/octet-stream');
    $serverName = 'testserver';
    $connectionOptions = array('Database'=>'testdatabase');
    $conn = sqlsrv_connect($serverName, $connectionOptions);  
    $sql = 'SELECT compressed FROM tb_compression WHERE id = 1';
    $qry = sqlsrv_query($conn, $sql);
    $row = sqlsrv_fetch_array($qry);
    $compressed = $row[0];
    sqlsrv_close($conn);
    echo $compressed;
?>

CLIENT SIDE: here is a working snippet in JavaScript:

CREDIT: The library used here for the decompression is gunzip.min.js from Imaya Yuta.

var request = new XMLHttpRequest();
request.onload = function(e) {
    var response = new Uint8Array(e.currentTarget.response);
    console.log(response.byteLength);
    /* Strip out the response termination */
    var compressed = response.subarray(0, response.byteLength - 4);
    var gunzip = new Zlib.Gunzip(compressed);
    var decompressed = gunzip.decompress();
    var encoding = 'utf-8'; /* For varchar text (ansi) */
    //var encoding = 'utf-16'; /* For nvarchar text (double-byte) */
    var text = new TextDecoder(encoding).decode(decompressed);
    console.log(text); /* Here You go */
};
request.responseType = 'arraybuffer';
request.open('GET', 'getcompressed.php', true);
request.send(null);

If You look at the response.byteLength, for that example text of 13046 Bytes, You will check that there are effectively transmitted over the network just only 2015 Bytes.

As @Karl-Johan Sjögren stated the sql function compress() returns a GZIP value, you wont be able to decompress that on javascript without the use of a external library such as zlib.

Related