# Implementation of the cross validiation

I'm attending a course in computational statistics, which should be an applied course. We study different methods, which are important in "reality". One of these topics is Cross Validation. I'm faced with the following problem coming from a homework. We are given a dataset and suppose that the model is of the form $$Y_i=m(X_i)+\epsilon_i$$ i.e. a nonparametric regression. We want to compute the generalized error using the leave one out Cross Validation score. This should be done by using kernel estimator ksmooth, local polynomials and smoothing splines. My first question is very general: Given the data how can I choose by eye a reasonable bandwith for the kernel estimator? See for example the following picture: Since this data looks rather "wild" it is for me not clear how to choose a bandwidth. My first attempt was just to run ksmooth playing around with different bandwidth. But as I said, here the data is wild, so it is (for me) hard to determine a reasonable bandwidth.

The second problem is more concrete about the problem described above. So far I have the following code:

   cv <- function(data,used.function)
{
n <- nrow(data)
cv.value <- rep(0,length(n))
for (i in 1:n){
new.data <- data[-i,]
cv.value[i] <- used.function(new.data[,1],new.data[,2],data[i,1])
}
## MSE
return(1/n*sum((new.data[,2]-cv.value)^2))
}

fcn1 <- function(reg.x, reg.y, x){
return(ksmooth(reg.x, reg.y, x.point = x, kernel = "normal", bandwidth = h)$y) } ### CV-score for kernel estimator: (cv.nw <- cv(real.data, fcn1))  the function cv should be general such that I can apply local polynomials and smoothing spline too. The variable real.data contains the data. It is a$n\times 2$matrix which store all the$x$values and$y$values. In the function body of cv I perform a leave one out cross validation. However using this code gives for cv.nw a NA. What is wrong with my code? I'm very thankful for your help. • If you're looking for a reasonable starting value, check out the function dpill from the KernSmooth library. Apr 18 '13 at 16:09 • @wcampbell thanks for your comment. But I'm interested in finding an appropriate bandwidth by eye – math Apr 19 '13 at 12:23 ## 1 Answer I realize this is a somewhat older question now but perhaps this can shed some light. There are several things wrong with your code. 1. The code as it stands is NOT reproducible. At no point is h defined which is the exact parameter you wish to tune. 2. You aren't iterating over different bandwidths. If you are using cross-validation to compare between different models, you need different models (i.e. different parameters). Modifying your code and utilizing the cars dataset, here is a reproducible example of Leave-One-Out Cross-validation on bandwidth whereby you pass a 'grid' of different bandwidths to tune the model. cv <- function(data, used.function, bandwidth.grid) { n <- nrow(data) mse <- matrix(, nrow=length(bandwidth.grid), ncol=2) for (b in 1:length(bandwidth.grid)){ cv.value <- rep(0, n-1) for (i in 1:(n-1)){ new.data <- data[-i,] funcargs <- list(reg.x=new.data[,1],reg.y=new.data[,2],x=data[i,1], h = bandwidth.grid[b]) cv.value[i] <- do.call(used.function, funcargs) } mse[b,] <- c(bandwidth.grid[b], 1/n*sum((new.data[,2]-cv.value)^2)) } ## MSE colnames(mse) <- c("bandwidth", "mse") return(mse) } ### kernel estimator usind nadaraya-watson: fcn1 <- function(reg.x, reg.y, x, h){ return(ksmooth(reg.x, reg.y, x.point = x, kernel = "normal", bandwidth = h)$y)
}

attach(cars)
### CV-score for kernel estimator:
cv(cbind(speed, dist), fcn1, seq(10))
> cv(cbind(speed, dist), fcn1, seq(10))
bandwidth      mse
[1,]         1 261.9555
[2,]         2 223.3542
[3,]         3 217.8303
[4,]         4 214.0923
[5,]         5 211.1874
[6,]         6 211.4104
[7,]         7 214.6941
[8,]         8 220.1501
[9,]         9 227.2262
[10,]        10 235.5479


Here we can see that a bandwidth = 5 would be best.