Getting a list of substring based unique filenames in an array

user_6676240 0
Viewed 74

I have a directory my_dir with files having names like:

a_v5.json
a_v5.mapping.json
a_v5.settings.json
f_v39.json
f_v39.mapping.json
f_v39.settings.json
f_v40.json
f_v40.mapping.json
f_v40.settings.json
c_v1.json
c_v1.mapping.json
c_v1.settings.json

I'm looking for a way to get an array [a_v5, f_v40, c_v1] in bash. Here, array of file names with the latest version number is what I need.

Tried this: ls *.json | find . -type f -exec basename "{}" \; | cut -d. -f1, but it returns the results with files which are not of the .json extension.

shell bash
4 Answers

You can use the following command if your filenames don't contain whitespace and special symbols like * or ?:

array=($(
    find . -type f -iname \*.json |
    sed -E 's|(.*/)*(.*_v)([0-9]+)\..*|\2 \3|' |
    sort -Vr | sort -uk1,1 | tr -d ' '
))

It's ugly and unsafe. The following solution is longer but can handle all file names, even those with linebreaks in them.

maxversions() {
    find -type f -iname \*.json -print0 |
    gawk 'BEGIN { RS = "\0"; ORS = "\0" }
        match($0, /(.*\/)*(.*_v)([0-9]+)\..*/, group) {
            prefix = group[2];
            version = group[3];
            if (version > maxversion[prefix])
                maxversion[prefix] = version
        }
        END {
            for (prefix in maxversion)
                print prefix maxversion[prefix]
        }'
}
mapfile -d '' array < <(maxversions)

In both cases you can check the contents of array with declare -p array.

user_67703840

Arrays and bash string parsing.

declare -A tmp=()
for f in $SOURCE_DIR/*.json
do f=${f##*/}        # strip path
   tmp[${f%%.*}]=1   # strip extraneous data after . in filename
done
declare -a c=( $( printf "%s\n" "${!tmp[@]}" | cut -c 1 | sort -u ) ) # get just the first chars
declare -a lst=( $( for f in "${c[@]}"
                    do printf "%s\n" "${!tmp[@]}" |
                         grep "^${f}_"            |
                         sort -n                  |
                         tail -1; done ) )
echo "[ ${lst[@]} ]"
[ a_v5 c_v1 f_v40 ]

Or, if you'd rather,

declare -a arr=( $(
  for f in $SOURCE_DIR/*.json
  do d=${f%/*}         # get dir path
     f=${f##*/}        # strip path
     g=${f:0:2}        # get leading str
     ( cd $d && printf "%s\n" ${g}*.json | 
       sort -n | sed -n '$ { s/[.].*//; p; }' )
  done | sort -u ) )
echo "[ ${arr[@]} ]"
[ a_v5 c_v1 f_v40 ]
user_86565520

This is one possible way to accomplish this :

arr=( $(  { for name in $( ls {f,n,m}*.txt );  do echo ${name:0:1} ; done; } | sort | uniq ) )

Output :

$ echo ${arr[0]}
f

$ echo ${arr[1]}
m

$ echo ${arr[2]}
n

Regards!

user_77238820

AWK SOLUTION

This is not an elegant solution... my knowledge of awk is limited. You should find this functional.

I've updated this to remove the obsolete uniq as suggested by @socowi. I've also included the printf version as @socowi suggested.

ls *.json | cut -d. -f1 | sort -rn |  awk -v last="xx" '$1 !~ last{ print $1; last=substr($1,1,3)  }'

OR

printf %s\\n *.json | cut -d. -f1 | sort -rn | awk -v last="xx" '$1 !~ last{ print $1; last=substr($1,1,3)  }'

Old understanding below

Find files with name matching pattern.

Now take the second field since your results will likely be similar to ./

find . -type f -iname "*.json" | cut -d. -f2

To get the unique headings....

find . -type f -iname "*.json" | cut -d. -f2 | sort | uniq
user_29619160
Related