Access sub-folders in kdb+/Q

user_9155285 0
Viewed 277

The below script returns the list of files included in Main_Dir (which only includes folders)

path: `$":D:/Main_Dir/
files: key(path)

However, I would like to access the files (.csvs) which reside in the sub-folder for which the path being: D:/Main_Dir/sub_dir1,D:/Main_Dir/sub_dir2...sub_dir3

This code is a part of the larger sub-block, which intends to read csvs residing in the sub-folder.

path: `$":D:/Main_Dir/
files: key(path)
loadcsv: {[path;file]("SDNFFFFJJ";enlist csv) 0: ` sv path,file}
d2: raze loadcsv[path] each files
kdb
3 Answers

For the sake of variety, a recursive approach which would work for arbitrarily nested directories (similar to how MurrMacks answer would):

q){$[{x~key x}y;(();y)y like x;raze .z.s[x]each` sv'y,'key y]}["*.csv";`:.]
`:./dir1/f1.csv`:./dir3/f3.csv`:./dir3/subdir1/f3s.csv`:./dir3/subdir1/subsubdir2/f3ss.csv
user_38956970

I have made a small test directory

 λ tree
.
├── subdir1
│   └── hi.csv
├── subdir2
│   └── hi.csv
└── subdir3
    ├── hi.csv
    └── hi.txt

3 directories, 4 files

What you could do is make a list of all of the full (relative) paths of the files, and then filter out the ones that aren't csv using like.

q)paths: paths where (paths: raze {` sv' dir ,/: key dir: hsym x} each key `:.) like "*.csv"
`:subdir1/hi.csv`:subdir2/hi.csv`:subdir3/hi.csv

Then you could do your reading in of these files just using the each.

{[path] ("SDNFFFFJJ";enlist csv) 0: path} each paths

Would this work?

user_127041550

To add to Matthews answer and to give a different approach - you could use a system command with the linux 'find' utility. Using the same directory structure -

q)hsym `$system"find . -type f -name '*.csv'"
 `:subdir1/hi.csv`:subdir2/hi.csv`:subdir3/hi.csv

Matthews answer is much better and I would avoid using system commands in a production system but in this case its a simpler solution.

++ edit -noticing you are on windows you can use

q)hsym `$system"dir *.csv /b/s"
 `:subdir1/hi.csv`:subdir2/hi.csv`:subdir3/hi.csv
user_124366200
Related