Performing residual bootstrap using kernel regression in R

Question

Kernel regression is a non-parametric technique that wants to estimate the conditional expectation of a random variable. It uses local averaging of the response value, Y, in order to find some non-linear relationship between X and Y.

I am have used bootstrap for kernel density estimation and now want to use it for kernel regression as well. I have been told to use residual bootstrapping for kernel regression and have read a couple of papers on this. I am however unsure how to perform this. Programming has been done in R using the FKSUM package. I have made an attempt to use standard resampling on kernel regression:

library(FKSUM)
set.seed(1)

n <- 5000
sample.size <- 500
B.replications <- 200
x <- rbeta(n, 2, 2) * 10
y <- 3 * sin(2 * x) + 10 * (x > 5) * (x - 5)
y <- y + rnorm(n) + (rgamma(n, 2, 2) - 1) * (abs(x - 5) + 3)

#taking x.y to be the population
x.y <- data.frame(x, y)

xs <- seq(min(x), max(x), length = 1000)
ftrue <- 3 * sin(2 * xs) + 10 * (xs > 5) * (xs - 5)

#Sample from the population
seqx<-seq(1,5000,by=1)
sample.ind <- sample(seqx, size = sample.size, replace = FALSE)
sample.reg<-x.y[sample.ind,]
x_s <- sample.reg$x
y_s <- sample.reg$y

fhat_loc_lin.pop <- fk_regression(x, y)
fhat_loc_lin.sample <- fk_regression(x = x_s, y = y_s)

plot(x, y, col = rgb(.7, .7, .7, .3), pch = 16, xlab = 'x',
     ylab = 'x', main = 'Local linear estimator with amise bandwidth')
lines(xs, ftrue, col = 2, lwd = 3)
lines(fhat_loc_lin, lty = 2, lwd = 2)

#Bootstrap
n.B.sample = sample.size  # sample bootstrap size

boot.reg.mat.X <- matrix(0,ncol=B.replications, nrow=n.B.sample)
boot.reg.mat.Y <- matrix(0,ncol=B.replications, nrow=n.B.sample)
fhat_loc_lin.boot <- matrix(0,ncol = B.replications, nrow=100)

Temp.reg.y <- matrix(0,ncol = B.replications,nrow = 1000)


for(i in 1:B.replications){
  sequence.x.boot <- seq(from=1,to=n.B.sample,by=1)
  sample.ind.boot <- sample(sequence.x.boot, size = sample.size, replace = TRUE)
  boot.reg.mat <- sample.reg[sample.ind.boot,]

  boot.reg.mat.X <- boot.reg.mat$x
  boot.reg.mat.Y <- boot.reg.mat$y

  fhat_loc_lin.boot <- fk_regression(x = boot.reg.mat.X ,
                                     y = boot.reg.mat.Y, 
                                     h = fhat_loc_lin.sample$h)

  lines(y=fhat_loc_lin.boot$y,x= fhat_loc_lin.sample$x, col =c(i) )

Temp.reg.y[,i] <- fhat_loc_lin.boot$y  

}

quan.reg.l <- vector()
quan.reg.u <- vector()

for(i in 1:length(xs)){
  quan.reg.l[i] <- quantile(x = Temp.reg.y[i,],probs = 0.025)
  quan.reg.u[i] <- quantile(x = Temp.reg.y[i,],probs = 0.975)
}

# Lower Bound
Temp.reg.2  <-  quan.reg.l 
lines(y=Temp.reg.2,x=fhat_loc_lin.boot$x ,col="red",lwd=4,lty=1)

# Upper Bound
Temp.reg.3  <-  quan.reg.u
lines(y=Temp.reg.3,x=fhat_loc_lin.boot$x ,col="navy",lwd=4,lty=1)

Asking the question on here now since I haven't received any response on CV. Any help would be greatly appreciated!

Performing residual bootstrap using kernel regression in R

0 Answers