Top N Values of Cosine Similarity Matrix in R

user_5893585 0
Viewed 176

How do I get the top pairs of a cosine similarity matrix like below:

southpark_matrix <- structure(c(0, 0.165272735625452, 0.386480286121192, 0.170696960480773, 
0.0869562860988618, 0.165272735625452, 0, 0.251690602341816, 
0.472701602991984, 0.137486001150133, 0.386480286121192, 0.251690602341816, 
0, 0.255849200006255, 0.0972813221214626, 0.170696960480773, 
0.472701602991984, 0.255849200006255, 0, 0.156449701347234, 0.0869562860988618, 
0.137486001150133, 0.0972813221214626, 0.156449701347234, 0), .Dim = c(5L, 
5L), .Dimnames = list(Docs = c("Mr. Garrison_2", "Cartman_3", 
"Mr. Garrison_3", "Cartman_4", "Jimbo_5"), Docs = c("Mr. Garrison_2", 
"Cartman_3", "Mr. Garrison_3", "Cartman_4", "Jimbo_5")))

southpark_matrix

                Docs
Docs             Mr. Garrison_2 Cartman_3 Mr. Garrison_3 Cartman_4    Jimbo_5
  Mr. Garrison_2     0.00000000 0.1652727     0.38648029 0.1706970 0.08695629
  Cartman_3          0.16527274 0.0000000     0.25169060 0.4727016 0.13748600
  Mr. Garrison_3     0.38648029 0.2516906     0.00000000 0.2558492 0.09728132
  Cartman_4          0.17069696 0.4727016     0.25584920 0.0000000 0.15644970
  Jimbo_5            0.08695629 0.1374860     0.09728132 0.1564497 0.00000000

How do I get the top 2 pairs?

In this example, the top 2 pairs would be. In my actual example, I have over 100 columns and rows.

Cartman_3 Cartman_4             0.4727016
Mr. Garrison_2 Mr. Garrison_3   0.38648029
cosine-similarity r matrix
2 Answers

The way I would do this would be to convert the matrix to a tibble. We can do this by following the steps here to convert the upper triangular part of the matrix to a dataframe in 2 columns (see here: Convert upper triangular part of a matrix to 3-column long format).

After this we can simply use the top_n(2, val) function weigthed by our values. Another method at this step would be to arrange the values in descending order using arrange(desc(val)) and then use the head(2) function to take the top 2 values.

I have produced a reprex of my methods below

library(tidyverse)

southpark_matrix <- structure(c(0, 0.165272735625452, 0.386480286121192, 0.170696960480773, 
                                0.0869562860988618, 0.165272735625452, 0, 0.251690602341816, 
                                0.472701602991984, 0.137486001150133, 0.386480286121192, 0.251690602341816, 
                                0, 0.255849200006255, 0.0972813221214626, 0.170696960480773, 
                                0.472701602991984, 0.255849200006255, 0, 0.156449701347234, 0.0869562860988618, 
                                0.137486001150133, 0.0972813221214626, 0.156449701347234, 0), .Dim = c(5L, 
                                                                                                       5L), .Dimnames = list(Docs = c("Mr. Garrison_2", "Cartman_3", 
                                                                                                                                      "Mr. Garrison_3", "Cartman_4", "Jimbo_5"), Docs = c("Mr. Garrison_2", 
                                                                                                                                                                                          "Cartman_3", "Mr. Garrison_3", "Cartman_4", "Jimbo_5")))

# Convert the matrix to an upper diagonal form
ind <- which(upper.tri(southpark_matrix, diag = TRUE), arr.ind = TRUE)
dimnam <- dimnames(southpark_matrix)
df <- data.frame(row = dimnam[[1]][ind[, 1]],
           col = dimnam[[2]][ind[, 2]],
           val = southpark_matrix[ind])
#top n method
df %>%
  tibble() %>% 
  top_n(2, val)
#> # A tibble: 2 x 3
#>   row            col              val
#>   <chr>          <chr>          <dbl>
#> 1 Mr. Garrison_2 Mr. Garrison_3 0.386
#> 2 Cartman_3      Cartman_4      0.473

#arrange and head method
df %>% 
  arrange(desc(val)) %>% 
  head(2)
#> # A tibble: 2 x 3
#>   row            col              val
#>   <chr>          <chr>          <dbl>
#> 1 Cartman_3      Cartman_4      0.473
#> 2 Mr. Garrison_2 Mr. Garrison_3 0.386

Created on 2021-04-04 by the reprex package (v2.0.0)

user_155096200

With lapply:

N <- 2
best <- head(sort(southpark_matrix[upper.tri(southpark_matrix)], decreasing = TRUE),N)
lapply(best, function(x) {
  list(similarity = x, names = rownames(which(southpark_matrix == x, arr.ind = TRUE)))
})
#> [[1]]
#> [[1]]$similarity
#> [1] 0.4727016
#> 
#> [[1]]$names
#> [1] "Cartman_4" "Cartman_3"
#> 
#> 
#> [[2]]
#> [[2]]$similarity
#> [1] 0.3864803
#> 
#> [[2]]$names
#> [1] "Mr. Garrison_3" "Mr. Garrison_2"
user_135133280
Related