Retrieve recurrent expressions from a text object in R

user_9419025 0
Viewed 29

I'm working with a set of texts and I want to discover the ocurrence of some terms that indicate that this text refers to a certain topic.

The term is "council" and I created an R object containing all the ocurrences of "council" + 5 following words / punctuation. I want now to "clean" this objects, retrieving only the terms that ocurr frequently.

This would be something like this:

terms <- c("council of the city of Chicago", "council of the city. The other", "council to request a approval of", "council of public policies to deal", "council of public policies in order")

# RESULT
# council
# council of the city
# council of public policies

It is important to mantion that my real object is way bigger than that and I don't know beforehand all the possibilities that might appear after the word "council".

Thanks for the help =)

r
1 Answers

First let's change that example to something with recurring strings:

terms <- c("council of the city of Chicago", "council of the city of Chicago", 
           "council of the city of Chicago", "council of the city of Chicago", 
           "council of the city. The other", 
           "council to request a approval of", "council to request a approval of",
           "council of public policies to deal", 
           "council of public policies in order")

Now we count the occurences

terms |> table()

and sort for the most frequent ones first

terms |> table() |> sort(decreasing = TRUE)

and now we look only at the two most frequent ones:

terms |> table() |> sort(decreasing = TRUE) |> head(2)

The result is a table object that contains the frequencies as a vector and the strings as dimnames:

> terms |> table() |> sort(decreasing = TRUE) |> head(2) |> dimnames()
$terms
[1] "council of the city of Chicago"   "council to request a approval of"

Possibly usefull plot to decide on where to set a cutoff:

terms |> table() |> sort(decreasing = TRUE) |> table() |> 
         plot(xlab = "string repetitions", ylab = "occurrences")
user_65031410
Related