How to measure similarity between two arbitrary files?

user_727238 0
Viewed 381

I have two arbitrary files. Git does not consider them as one renamed file. Is it possible to measure (in percent) similarity between them?

git
3 Answers

Here is a pure Perl5 solution based on @Adder's answer that adds a few optimizations in case you are dealing with large files.

#!/usr/bin/perl

use strict;
use warnings;

use Tie::File;
use Fcntl 'O_RDONLY';

sub flines ($) {
  my %opts = (
    mode   => O_RDONLY, # No need to cache or write,   
    memory => 0,        #   as we are not using read data.
    recsep => "\n",     # Use "\r\n" if Windows
  );
  tie @_, 'Tie::File', shift, %opts or die "tie: $!\n";
  $_ = scalar @_;
  untie @_;
  return $_
}

sub slines ($) {
  my $fh = $_[0];
  next for <$fh>;
  return $.
}

my ($f1, $f2) = @ARGV;

# count number of lines in unified diff
open my ($fh), "diff -u1 $f1 $f2 |" or die "open: $!\n";

my $sD = slines $fh;
my $s1 = flines $f1;
my $s2 = flines $f2;

printf "%8.3g\n", 1.0 - ( $sD / 2.0 ) / ( $s1 > $s2 ? $s1 : $s2 )

It is used and called the same as in his example:

$ similarity foo bar
   0.327
user_10543970
Related