Create new dummy variable columns from categorical variable

user_144537 0
Viewed 86100

I have a several data sets with 75,000 observations and a type variable that can take on a value 0-4. I want to add five new dummy variables to each data set for all types. The best way I could come up with to do this is as follows:

# For the 'binom' data set create dummy variables for all types in all data sets
binom.dummy.list<-list()
for(i in 0:4){
    binom.dummy.list[[i+1]]<-sapply(binom$type,function(t) ifelse(t==i,1,0))
}

# Add and merge data
binom.dummy.df<-as.data.frame(do.call("cbind",binom.dummy.list))
binom.dummy.df<-transform(binom.dummy.df,id=1:nrow(binom))
binom<-merge(binom,binom.dummy.df,by="id")

While this works, it is incredibly slow (the merge function has even crashed a few times). Is there a more efficient way to do this? Perhaps this functionality is part of a package that I am not familiar with?

r
9 Answers

If you're open to using the data.table package, mltools has a one_hot() method.

library(data.table)
library(mltools)

binom <- data.table(y=runif(1e5), x=runif(1e5), catVar=as.factor(sample(0:4,1e5,TRUE)))
one_hot(binom)

                 y          x catVar_0 catVar_1 catVar_2 catVar_3 catVar_4
     1: 0.90511891 0.83045050        0        0        1        0        0
     2: 0.91375984 0.73273830        0        0        0        1        0
     3: 0.01926608 0.10301409        0        0        1        0        0
     4: 0.48691138 0.24428157        0        1        0        0        0
     5: 0.60660396 0.09132816        0        0        1        0        0
    ---                                                                   
 99996: 0.12908356 0.26157731        0        1        0        0        0
 99997: 0.96397273 0.98959000        0        1        0        0        0
 99998: 0.16818414 0.37460941        1        0        0        0        0
 99999: 0.72610508 0.72055867        1        0        0        0        0
100000: 0.89710998 0.24155507        0        0        0        0        1

Usage

one_hot(dt, cols = "auto", sparsifyNAs = FALSE, 
        naCols = FALSE, dropCols = TRUE,
        dropUnusedLevels = FALSE)

Which column(s) should be one-hot-encoded? cols = "auto" encodes all unordered factor columns. Therefore, the command below is equivalent. This is only important when the data.table contains factors that should not be encoded.

one_hot(binom, cols="catVar")
user_21468940

I did not have good luck with model.matrix() function as it was omitting some factor levels for whatever reason. However, i have had good luck with this simple function from library(fastDummies):

Columns that are converted into binary dummy variables have to be categorical.

fastDummies::dummy_cols(fastDummies_example, select_columns = "numbers", remove_selected_columns = "numbers")

user_64310980

You can use the package called dummies

binom <- data.frame(y=runif(1e5), x=runif(1e5), catVar=as.factor(sample(0:4,1e5,TRUE)))
head(binom)

          y          x catVar
1 0.4143348 0.09721401      1
2 0.3140782 0.54340539      3
3 0.1262037 0.51820499      2
4 0.7159850 0.13167720      3
5 0.8203528 0.94116026      3
6 0.2169781 0.82020216      1

Solution:

library(dummies)
binom<-dummy.data.frame(binom)
head(binom)

          y          x catVar0 catVar1 catVar2 catVar3 catVar4
1 0.4143348 0.09721401       0       1       0       0       0
2 0.3140782 0.54340539       0       0       0       1       0
3 0.1262037 0.51820499       0       0       1       0       0
4 0.7159850 0.13167720       0       0       0       1       0
5 0.8203528 0.94116026       0       0       0       1       0
6 0.2169781 0.82020216       0       1       0       0       0
user_119836010
Related