Does anyone have a particularly elegant command line (linux, OS X) way to identify "textually similar" files in a given directory?
By "textually similar", I mean that the files should only differ in N number of lines.
4 Answers
Using Terraform means having a lot of files that are copied from other files and only a few changes made. It's really frustrating to figure out where a file was copied from when you want to see what's special about it. I made a tool I call similarities.sh to help me identify how similar a file is to each file in a group of others.
#!/bin/bash
fileA="$1"
shift
for fileB in "$@"; do
(
# diff once grep twice with the help of tee and stderr
diff $fileA $fileB | \
tee >(grep -cE '^< ' >&2) | \
grep -cE '^> ' >&2
# recapture stderr
) 2>&1 | (
read -d '' diffA diffB;
printf "The files %s and %s have %s:%s diffs out of %s:%s lines.\n" \
$fileA $fileB $diffA $diffB $(wc -l < $fileA) $(wc -l < $fileB)
)
done | column -t
Here it is in action:
$ similarities.sh terraform.tfvars ../*/terraform.tfvars
The files terraform.tfvars and ../api_proxy/terraform.tfvars have 3:3 diffs out of 51:51 lines.
The files terraform.tfvars and ../cf-ip-location-lookup/terraform.tfvars have 4:12 diffs out of 51:59 lines.
The files terraform.tfvars and ../cf-region-cookie-setter/terraform.tfvars have 4:8 diffs out of 51:55 lines.
The files terraform.tfvars and ../cf-switch-region-origin/terraform.tfvars have 4:10 diffs out of 51:57 lines.
The files terraform.tfvars and ../reformat_devops_alerts/terraform.tfvars have 0:0 diffs out of 51:51 lines.
The files terraform.tfvars and ../restart_location/terraform.tfvars have 17:3 diffs out of 51:37 lines.
The files terraform.tfvars and ../warehouse-availability-etl/terraform.tfvars have 3:3 diffs out of 51:51 lines.
Related