Facet plot word proportions from 3 groups against each other in R (tidytext + ggplot2)

Question

In the first chapter of Text Mining with R Silge & Robinson propose the following code to produce the plots shown in https://www.tidytextmining.com/tidytext.html#word-frequencies, comparing word frequencies (proportions) from Austen's books, to Wells' and the Brontë Sisters'.

library(gutenbergr)
library(janeaustenr)
library(dplyr)
library(stringr)
library(tidytext)
library(tidyr)
library(scales)

original_books <- austen_books() %>%
  group_by(book) %>%
  mutate(linenumber = row_number(),
         chapter = cumsum(str_detect(text, 
                                     regex("^chapter [\divxlc]",
                                           ignore_case = TRUE)))) %>%
  ungroup()

tidy_books <- original_books %>%
  unnest_tokens(word, text) %>%
  anti_join(stop_words)

hgwells <- gutenberg_download(c(35, 36, 5230, 159))
tidy_hgwells <- hgwells %>%
  unnest_tokens(word, text) %>%
  anti_join(stop_words)

bronte <- gutenberg_download(c(1260, 768, 969, 9182, 767))
tidy_bronte <- bronte %>%
  unnest_tokens(word, text) %>%
  anti_join(stop_words)

frequency <- bind_rows(mutate(tidy_bronte, author = "Brontë Sisters"),
                       mutate(tidy_hgwells, author = "H.G. Wells"), 
                       mutate(tidy_books, author = "Jane Austen")) %>% 
  mutate(word = str_extract(word, "[a-z']+")) %>%
  count(author, word) %>%
  group_by(author) %>%
  mutate(proportion = n / sum(n)) %>% 
  select(-n) %>% 
  pivot_wider(names_from = author, values_from = proportion) %>%
  pivot_longer(`Brontë Sisters`:`H.G. Wells`,
               names_to = "author", values_to = "proportion")

ggplot(frequency, aes(x = proportion, y = `Jane Austen`, 
                      color = abs(`Jane Austen` - proportion))) +
  geom_abline(color = "gray40", lty = 2) +
  geom_jitter(alpha = 0.1, size = 2.5, width = 0.3, height = 0.3) +
  geom_text(aes(label = word), check_overlap = TRUE, vjust = 1.5) +
  scale_x_log10(labels = percent_format()) +
  scale_y_log10(labels = percent_format()) +
  scale_color_gradient(limits = c(0, 0.001), 
                       low = "darkslategray4", high = "gray75") +
  facet_wrap(~author, ncol = 2) +
  theme(legend.position="none") +
  labs(y = "Jane Austen", x = NULL)

This effectively produce 2 plots in a single facet (Austen against Wells, and Austen against the Brontës). But a plot is missing there: that of Wells vs. the Brontë sisters, and though it's fairly trivial to plot that as a single graph on its own, I'm having trouble plotting the 3 graphs together, ie., as a 3-column facet depicting each author's word frequencies vs. the others.

Any advice on achieving this visualization? Thank you!

Jon Spring · Answer

Here's a modified approach where the pivot_longer treats the authors on equal footing, then we join that frequency table to itself and just keep the inter-author comparisons. (You could keep the Austen vs. Austen but it would just be a line.)

  ...
  pivot_wider(names_from = author, values_from = proportion) %>%
  pivot_longer(-word, names_to = "author", values_to = "proportion")

frequency %>%
  left_join(frequency, by = "word") %>%
  filter(author.x != author.y) %>%
  replace_na(list(proportion.x = 0, proportion.y = 0)) %>%

ggplot(aes(x = proportion.x, y = proportion.y,
                      color = abs(proportion.y - proportion.x))) +
  geom_abline(color = "gray40", lty = 2) +
  geom_jitter(alpha = 0.1, size = 2.5, width = 0.3, height = 0.3) +
  geom_text(aes(label = word), check_overlap = TRUE, vjust = 1.5) +
  scale_x_log10(labels = percent_format()) +
  scale_y_log10(labels = percent_format()) +
  scale_color_gradient(limits = c(0, 0.001), 
                       low = "darkslategray4", high = "gray75") +
  facet_grid(author.x ~ author.y) +
  theme(legend.position="none") +
  labs(y = "Jane Austen", x = NULL)

You might consider adding a slice_sample(n = 10000) %>% or similar in between the prep and the ggplot in case it's too slow for your taste. In my experience ggplot2 starts to get too sluggish for my taste when n > 10k; in this case the combined data has 171k points so it's real slow.

Facet plot word proportions from 3 groups against each other in R (tidytext + ggplot2)

1 Answers