Let's say I have two datasets df1 and df2 as below.
In the first step, I want to compute cosine similarity after matching first letter of names from df1 to first letter of professions in df2. For instance, the first letter of name arthur is 'a', so we pick professions with 'a' -- abbot, compute cosine similarity between the rnorm(10) values of arthur and abbot; then pick arthur again and compute cosine similarity between rnorm values of abel and architect. We ignore other professions as they don't have matching initial letters. Next, we pick arun and do the same (so on for each name). Thus, the last value we compute will be the cosine similarity between rnorm values of yatin and yeoman.
In the second step, I want to average the cosine similarities by name. Thus, we will have a single cosine value for each name.
library(tidyverse)
set.seed(1234)
df1 <- tibble(
arthur = rnorm(10),
arun= rnorm(10),
bob= rnorm(10),
bart=rnorm(10),
cody=rnorm(10),
ram=rnorm(10),
yatin=rnorm(10))
df2<- tibble(
abbot = rnorm(10),
architect=rnorm(10),
barista = rnorm(10),
beekeeper=rnorm(10),
baker=rnorm(10),
caterer=rnorm(10),
economist = rnorm(10),
engineer= rnorm(10),
lawyer = rnorm(10),
legislator=rnorm(10),
lifeguard= rnorm(10),
ranger = rnorm(10),
trader = rnorm(10),
yeoman= rnorm(10))
The output will be like this (I have just put some random values for cosine similarity):
Name Matching_Profession_Initial Average_Cosine_Similarity
arthur a 0.123
arun a 0.135
bob b 0.111
bart b 0.001
cody c 0.01
ram r 0.01
yatin y 0.09
Thanks so much.