Convert dfmSparse from Quanteda package to Data Frame or Data Table in R

Viewed 2880

I have a dfmSparse object (large, with 2.1GB) which is tokenized and with ngrams (unigrams, bigrams, trigrams and fourgrams), and I want to convert it to a data frame or a data table object with the columns: Content and Frequency.

I tried to unlist... but didn't work. I'm new in NLP, and I don't know with method to use, I'm without ideas and didn't found a solution here or with Google.

Some info about the data:

>str(tokfreq)
Formal class 'dfmSparse' [package "quanteda"] with 11 slots
  ..@ settings    :List of 1
  .. ..$ : NULL
  ..@ weighting   : chr "frequency"
  ..@ smooth      : num 0
  ..@ ngrams      : int [1:4] 1 2 3 4
  ..@ concatenator: chr "_"
  ..@ Dim         : int [1:2] 167500 19765478
  ..@ Dimnames    :List of 2
  .. ..$ docs    : chr [1:167500] "character(0).content" "character(0).content" "character(0).content" "character(0).content" ...
  .. ..$ features: chr [1:19765478] "add" "lime" "juice" "tequila" ...
  ..@ i           : int [1:54488417] 0 75 91 178 247 258 272 327 371 391 ...
  ..@ p           : int [1:19765479] 0 3218 3453 4015 4146 4427 4637 140665 140736 142771 ...
  ..@ x           : num [1:54488417] 1 1 1 1 5 1 1 1 1 1 ...
  ..@ factors     : list()

>summary(tokfreq)
       Length         Class          Mode 
3310717565000     dfmSparse            S4

Thanks!

EDITED: This is how I created the dataset from a corpus:

# tokenize
tokenized <- tokenize(x = teste, ngrams = 1:4)
# Creating the dfm
tokfreq <- dfm(x = tokenized)
2 Answers
Related