I am trying to build a neural network using keras in R. I am trying to use a dataset comprised of certain variables from another dataset. I think I have made an error in data partition, but I am not too sure what is the error.
For context, I have used feature selection methods to identify the most important features. Then, I created a dataset comprised of the most important features. I am now trying to use this dataset to train a neural network.
I am getting the following error:
Error in py_call_impl(callable, dots$args, dots$keywords) :
ValueError: Data cardinality is ambiguous:
x sizes: 3505
y sizes: 626
Make sure all arrays contain the same number of samples.
I am using the following R code:
#Loading Libraries
library(dplyr)
library(faux)
library(caret)
library(randomForest)
library(ROSE)
library(e1071)
library(corrplot)
library(tidymodels)
library(tidyverse)
library(class)
library(caTools)
library(DataExplorer)
library(glmnet)
library(xgboost)
library(class)
library(pROC)
library(neuralnet)
library(keras)
hc <- data$hc
cwtfp <- data$cwtfp
irr <- data$irr
pl_c <- data$pl_c
pl_n <- data$pl_n
forestry_change <- data$forestry_change
agriculture_change <- data$agriculture_change
growthbucket_factor_imp <- data$growthbucket_factor
data_important <- data.frame(hc, cwtfp, irr, pl_c, pl_n, forestry_change, agriculture_change, growthbucket_factor_imp)
#Splitting the important dataset
set.seed(1)
sample <- sample.split(data_important$growthbucket_factor, SplitRatio = 0.7)
train_imp <- subset(data_important, sample == TRUE)
test_imp <- subset(data_important, sample == FALSE)
#Dense Neural Network
#data preparation
data_important_matrix <- as.matrix(data_important)
dimnames(data_important_matrix) <- NULL
#Partition
set.seed(1234)
ind <- sample(2, nrow(data_important_matrix), replace = T, prob = c(0.7,0.3))
training_nn <- data_important_matrix[ind==1, 1:7]
test_nn <- data_important_matrix[ind==2, 1:7]
trainingtarget_nn <- data_important_matrix[ind==1, 8]
testtarget_nn <- data_important_matrix[ind==2, 8]
#Data processing
training_nn <- as.numeric(unlist(training_nn))
test_nn <- as.numeric(unlist(training_nn))
#one hot encoding
trainnnLabels <- to_categorical(trainingtarget_nn)
testnnLabels <- to_categorical(testtarget_nn)
#Creating the model specification
model <- keras_model_sequential()
model %>%
layer_dense(units = 8, activation = 'relu', input_shape = c(8)) %>%
layer_dense(units = 4, activation = 'relu') %>%
layer_dense(units = 1, activation = 'softmax')
summary(model)
#Compile
model %>% compile(loss = 'binary_crossentropy',
optimizer = 'adam',
metrics = 'accuracy')
#Fitting the model
history <- model %>%
fit(training_nn,
trainnnLabels,
epoch = 200,
batch_size = 32,
validation_split = 0.2)