Set-up of Optimisation Problem in Connectionist Neuron problem

Viewed 75

I would like to set up an optimisation procedure identifying the hightest predicition accurancy by picking the value $\theta_{optimal} \in [-3, 3]$ according to a connectionist neuron model with binary identifyer. As there is no way to include latex code I provide an image of the model/formula instead:

enter image description here

Assume I already identified a certain weight vector $w=(w_1 w_2)^T$. I can therefore write the calculation as

w <- c(0.9396926, 0.3420201)
X <- as.matrix(t(ex[,1:2])) 

# with theta = 0
result <- as.data.frame(sign(t(w %*% X)))
result[result == -1] <- 0
# where result is a df with dimension 15x1, and df$V1 are the predicted y labels 

To shorten the code of the example, I can elaborate the euclidean distance between the true labels and the predicted labels (values) instead of calculating the predicition accurancy:

euclidean <- function(a, b){
  sqrt(sum((a - b)^2))
} 
euclidean(as.integer(ex$y) ,result$V1)

Since I never did something like this before, I have no idea how to set up such an iterative optimisation procedure of

  • first calculating the predicted values y for all possible values of theta,
  • and then, for each vector of predicted results res calculate the euclidean distance,
  • and finally choose the theta which gives the highest accurancy/lowest euclidean distance.

I would appreciate it if someone could give me hints, link to comparable set-ups, or guide me through some steps.


Example Data: dput(data[, 1:15])

ex <- structure(list(x.1 = c(0.365, 0.543, -0.401, 0.866, -0.386, -0.443, 
-0.519, 0.332, 0.211, -0.208), x.2 = c(0.708, -0.268, 0.643, 
-0.796, 0.742, 0.615, 0.818, -0.211, -0.237, -0.656), y = structure(c(1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = c("0", "1"), class = "factor")), row.names = c(NA, 
10L), class = "data.frame")

x.1 and y.1 are the respective coordinates and y is the known true label.

1 Answers

We can try the code below using optimize

f <- function(theta) sqrt(sum((as.integer(ex$y) - (t(w %*% X) >= theta))^2))
xmin <- optimize(f, c(-3, 3))

which gives

> xmin
$minimum
[1] -1.583516

$objective
[1] 0
Related