I've just built my first proper q/kdb+ database with splayed and partitioned tables. Everything is going fine, but I just noticed that my symbol s column file size is unusually large. Here is what I can see from the OS and from inside q:
# ls -latr 2017.10.30/ngbarx
total 532
-rw-r--r-- 1 root root 24992 Apr 17 20:53 vunadj
-rw-r--r-- 1 root root 24992 Apr 17 20:53 v
-rw-r--r-- 1 root root 300664 Apr 17 20:53 s
...
q)meta ngbarx
c | t f a
------| -----
date | d
s | s p
v | e
vunadj| e
...
q)get `:2017.10.30/ngbarx/s
`p#`sym$`A`AA`AACG`AADI`AADR`AAIC`AAIC-B`AAL`AAM-A`AAMC`AAME`AAOI`AAON`AAP`AA..
q)-22!get `:2017.10.30/ngbarx/v
24990
q)-22!get `:2017.10.30/ngbarx/s
28678
q)all (get `:2017.10.30/ngbarx/s) in sym
1b
q)count sym
62136
So comparing the real-type v column with the symbol-type s column, I see from ls that the symbol column is more than 10x the size, even though the internal size in bytes is similar and everything seems properly encoded in the sym file.
Is this expected behavior? Or am I doing something wrong that could be fixed?
UPDATE: I have not used compression, and have written the files using the magical function .Q.dcfgnt, which can be viewed here. Well, a slightly modified version, I noticed that this function as is also saved a date file in the directory, even though the column should be virtual, so I did some hacking in k (I'm not very good at it) and updated the inner function .Q.dpfgnt to this ...
k){[d;p;f;g;n;t]if[~&/qm'r:+en[d]t;'`unmappable];
{[d;g;t;i;x]@[d;x;g;t[x]i]}[d:par[d;p;n];g;r;<r f]'{x@&~x=`date}(!r);
@[;f;`p#]@[d;`.d;:;f,r@&~f=r:{x@&~x=`date}(!r)];n}