I needed to simulate data from a random guess binary classifier such that
$p(success) = 0.55$ and $p(failure) = 1-0.55$.
So I simulated the data from a Bernoulli distribution as follows and compared the results with the actual response data.
set.seed(123)
#predicted response
y_pred=replicate(50,rbinom(1,1,prob = 0.55))
#actual response
y=sample(rep(0:1,each=25))
table(y,y_pred)
y_pred
y 0 1
0 13 12
1 10 15
Did I do this correctly? Any guidance would be really helpful.
Looks reasonable for me. You don't need replicate, though. And personally I would set a global n:
set.seed(123)
## predicted response
n <- 50
y_pred <- rbinom(n, 1, prob=.55)
## calculate actual probability of predicted response
sum(y_pred) / length(y_pred)
# [1] 0.54
## actual response
y <- sample(rep(0:1, each=n/2))
## calculate actual probability of actual response
sum(y) / length(y)
# [1] 0.5
table(y, y_pred)
# y_pred
# y 0 1
# 0 13 12
# 1 10 15
However, the actual probability of your predicted response at such small n can have large random fluctuations (i.e. depends on the seed), particularly with small n. Let's put the code into a function for a minute to show that.
n <- 50
sfun <- function() {
y_pred <- rbinom(50, 1, prob=.55)
sum(y_pred) / length(y_pred)
}
set.seed(383159)
sfun()
# [1] 0.62 ## 13% off!
set.seed(82809)
sfun()
# [1] 0.44 ## 20% off!
What you could do is to use a repeat loop, that breaks if the result is within a set tolerance. (Note, that this will run forever, when tol is set too small!)
tol <- .01
set.seed(123)
n <- 50
repeat({
y_pred <- rbinom(n, 1, prob=.55)
pr1 <- sum(y_pred) / length(y_pred)
if (pr1 <= .55 + tol & pr1 >= .55 - tol)
break
})
y_pred
# [1] 1 0 1 0 0 1 1 0 0 1 0 1 0 0 1 0 1 1 1 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0
# [35] 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0
sum(y_pred) / length(y_pred)
# [1] 0.54 ## ok!