blown up .sav file size using haven::write_sav()

user_6105723 0
Viewed 836

I am writing SPSS .sav files from R using the package haven, which works very well for me in general. However I have noticed that the .sav file size written on disk using write_sav() seems to be much bigger than nescessary. Whenever I open and save a .sav file written by write_sav() in SPSS, the file size is reduced by a factor of up to ~10!

This matters to me as I am writing rather big data to SPSS for others and sometimes SPSS refuses to open a very big file. Maybe this would problem would not arise if write_sav() would store more efficiently in a "real" native SPSS way?

Does anyone know this issue and maybe has a helpful comment on it? SPSS installation is needed to replicate this issue

r-haven spss filesize r
4 Answers
user_118202350

If the output size is a problem, you could have a look at my package readspss. Using compression and zsav you should be able to get the best available compression. Compression in sav files depends on how the file is written. SPSS has different compression methods to store numeric information. Numerics can be stored only as doubles (no compression) or in a mix of doubles and int8_t (compression 1). Zsav used zlib to compress whatever the initial input was (compression 2). Eight integers take the size of a double hence the difference in the file size.

user_123400290

There are three variants of the SPSS (.sav) file format:

  1. Uncompressed (.sav). This is haven's default output, but is rarely used in my experience.
  2. Compressed (.sav). This is what most people use, and it has been the default save format for SPSS for many, many years.
  3. Zcompressed (.zsav, but sometimes .sav). Added a few years ago to SPSS, but doesn't seem used much. You can get this from haven by adding compress=TRUE to write_spss()

I have submitted a pull request to make the compressed (2) format the default.

user_2495480
Related