Verify size and type of data stored in IndexedDB (Int8Array)

user_515054 0
Viewed 181

I'm setting up a virtual file system based on IndexedDB in the browser client. My assumption is that it will be efficient storing typed arrays as values; in particular, I am saving 8 kB chunks of data via Int8Array. What means do I have to verify the effectiveness of this - actual space used, actual data representation?

For example, in Chromium I can see that the Int8Array seem to be properly preserved. But the 'Clear Storage' page shows a suspiciously high storage size - 91 kB - although the first test file I was writing is only 40 kB in size (40096 bytes spread across 5 keys, plus 24 bytes for a meta-data key). The keys are quite small arrays containing a short string path and a number. So it looks like the storage takes around twice as much as predicted:

chromium screenshot saying my usage is 90.9 kB total

chromium screenshot showing IDB contents as 5 values of type Int8Array and one of type ArrayBuffer

In constrast, I cannot find any information in Firefox about the usage amount, but the storage browser shows only type 'Array' for the values, and they are represented very inefficiently as JSON objects, although that may be a display issue only:

Firefox screenshot showing storage inspector and JSON string representations of the values

A related question is, is there a difference between storing the ArrayBuffer objects as opposed to an Int8Array view on it? I tried both, and there is a mimimal difference in size (Chromium uses 90.8 kB instead of 90.9 kB if I use ArrayBuffer instead of Int8Array as value passed to IDB).

indexeddb browser javascript
1 Answers

First: Of course, writing only a small file can distort the picture, as overhead for the database itself is discounted. So I wrote instead a 40 MB file (now using ArrayBuffer directly), and now Chromium reports just under 41 MB of storage usage, confirming that the data is stored relatively compact. Indead I could see that the live updated storage usage was temporarily higher before going back to 41 MB, indicating that there is a compaction/cleaner algorithm running as well.

Since Firefox cannot show data usage for file://, I ran it through a web server for testing, and here too 41 MB of space is used.

Another surprising result was that cleverly storing an Int8Array which is a subarray actually seems to store the entire backing ArrayBuffer content as well. So, while the values are reported Int8Array(8192) in Chromium, the storage was much larger due to the underlying ArrayBuffer size (64K in my case). In this sense, better store the ArrayBuffer instance directly to avoid surprises.

BTW, Firefox was around 3 times faster than Chromium in this task. Still both perform abysmally slow (3 and 10 seconds respectively to perform the asynchronous I/O of storing 41 MB in chunks of 8 KB).

user_5150540
Related