Iteratively extract repeated word forms across speaking turns

user_8039978 0
Viewed 30

I'm working on speaking turns in conversation. My interest is in the words that get repeated from a prior turn to a next turn:

turnsX <- data.frame(
  speaker = c("A","B","A","B"),
  speech = c("let's have a look", 
             "yeah let's take a look",
             "yeah okay so where to start",
             "let's start here"), stringsAsFactors = F
)

I want to extract the repeated word forms. To this end I've run a for loop, iteratively defining each speech turn as a regex pattern for the next speech turn and str_extracting the words that get repeated from turn to turn:

library(stringr)
pattern <- c()
extracted <- c()
for(i in 1:nrow(turnsX)){
  pattern[i] <- paste0(unlist(str_split(turnsX$speech[i], " ")), collapse = "|")
  extracted[i+1] <- str_extract_all(turnsX$speech[i+1], pattern[i])
}

The result however is partly incorrect:

extracted
[[1]]
NULL

[[2]]
[1] "a"     "let's" "a"     "a"     "look" 

[[3]]
[1] "yeah" "a"    "a"   

[[4]]
[1] "start"

[[5]]
[1] NA

The correct result should be:

extracted
[[1]]
NULL

[[2]]
[1] "let's"    "a"     "look" 

[[3]]
[1] "yeah"   

[[4]]
[1] "start"

Where's the mistake? How can the code be mended, or what other approach is there, to get the correct result?

r for-loop regex
3 Answers

Maybe you can use Map and %in%.

x <- strsplit(turnsX$speech, " ")
Map(function(y,z) y[y %in% z], x[-length(x)], x[-1])
#[[1]]
#[1] "let's" "a"     "look"
#
#[[2]]
#[1] "yeah"
#
#[[3]]
#[1] "start"
user_104885040

Here's a base R approach using Map :

tmp <- strsplit(turnsX$speech, ' ')
c(NA, Map(intersect, tmp[-1], tmp[-length(tmp)]))

#[[1]]
#[1] NA

#[[2]]
#[1] "let's" "a"     "look" 

#[[3]]
#[1] "yeah"

#[[4]]
#[1] "start"
user_39629140

You want the word boundaries "\\b"

library(stringr)
pattern <- c()
extracted <- c()
for(i in 2:nrow(turnsX)){
  pattern[i - 1] <- paste0(unlist(str_split(turnsX$speech[i - 1], " ")), collapse = "|\\b")
  extracted[i] <- str_extract_all(turnsX$speech[i], pattern[i - 1])
}
# [[1]]
# NULL
# 
# [[2]]
# [1] "let's" "a"     "look" 
# 
# [[3]]
# [1] "yeah"
# 
# [[4]]
# [1] "start"
user_65740380
Related