Must subset elements with a valid subscript vector during an dplyr::anti_join()

user_7510632 0
Viewed 4367

I have this piece of code that is giving me some trouble

stop_words_extra = c("extra","words")
stop_words_vector= append(stop_words_extra, stopwords(kind = "it"), after = length(stop_words_extra))
stop_words_ita = data.frame(word = stop_words_vector)

text_mining_tweets <- tweets_with_retweet_number_upd %>%
 unnest_tokens(word, text) %>%
 anti_join(stop_words_ita)

This return me an error:

Errore: Must subset elements with a valid subscript vector.
i Logical subscripts must match the size of the indexed input.
x Input has size 19030 but subscript has size 372882.

with this stack:

  1. +-`%>%`(...)
  2. | +-base::withVisible(eval(quote(`_fseq`(`_lhs`)), env, env))
  3. | \-base::eval(quote(`_fseq`(`_lhs`)), env, env)
  4. |   \-base::eval(quote(`_fseq`(`_lhs`)), env, env)
  5. |     \-`_fseq`(`_lhs`)
  6. |       \-magrittr::freduce(value, `_function_list`)
  7. |         +-base::withVisible(function_list[[k]](value))
  8. |         \-function_list[[k]](value)
  9. |           +-dplyr::anti_join(., stop_words_ita)
 10. |           \-dplyr:::anti_join.data.frame(., stop_words_ita)
 11. |             \-dplyr:::join_filter(x, y, by = by, type = "anti", na_matches = na_matches)
 12. |               +-dplyr::dplyr_row_slice(x, idx)
 13. |               \-dplyr:::dplyr_row_slice.rowwise_df(x, idx)
 14. |                 \-vctrs::vec_slice(group_keys(data), i)
 15. \-vctrs:::stop_indicator_size(...)

19030 are the rows in tweets_with_retweet_number_upd 372882 are the rows after I use unnest_tokens(word, text)

It works fine on dplyr 0.8.5, but not in the 1.0.0 version. I see that the new one had some changes about *_joins(), but I don't really know what I'm doing wrong here.

EDIT: Here's a reproducible example. I tried to shrink it down as much I as could: https://pastebin.com/YxfRsfHW

dplyr r
1 Answers

It seems this new version of dplyr has issues with NULL list columns of urls_expanded_url, if you remove that column it works as expected.

library(dplyr)
library(tidytext)

example_so %>%
  unnest_tokens(word, text) %>%
  dplyr::select(-urls_expanded_url) %>%
  anti_join(stop_words_ita)


#   status_id user_id  created_at          screen_name urls_url  retweet_counter is_affidabile word     
#   <chr>     <chr>    <dttm>              <chr>       <list>              <dbl> <lgl>         <chr>    
# 1 111111111 22222222 2018-07-18 14:00:10 NA          <lgl [1]>               1 TRUE          grazie   
# 2 111111111 22222222 2018-07-18 14:00:10 NA          <lgl [1]>               1 TRUE          de       
# 3 111111111 22222222 2018-07-18 14:00:10 NA          <lgl [1]>               1 TRUE          andré    
# 4 111111111 22222222 2018-07-18 14:00:10 NA          <lgl [1]>               1 TRUE          Porto    
# 5 111111111 22222222 2018-07-18 14:00:10 NA          <lgl [1]>               1 TRUE          musical   
# 6 111111111 22222222 2018-07-18 14:00:10 NA          <lgl [1]>               1 TRUE          Italians 
# 7 111111111 22222222 2018-07-18 14:00:10 NA          <lgl [1]>               1 TRUE          cina     
# 8 111111111 22222222 2018-07-18 14:00:10 NA          <lgl [1]>               1 TRUE          racconta 
# 9 111111111 22222222 2018-07-18 14:00:10 NA          <lgl [1]>               1 TRUE          Zhang    
#10 111111111 22222222 2018-07-18 14:00:10 NA          <lgl [1]>               1 TRUE          changxiao
# … with 15 more rows
user_39629140
Related