8
$\begingroup$

I'm attending a course in computational statistics, which should be an applied course. We study different methods, which are important in "reality". One of these topics is Cross Validation. I'm faced with the following problem coming from a homework. We are given a dataset and suppose that the model is of the form $$ Y_i=m(X_i)+\epsilon_i $$ i.e. a nonparametric regression. We want to compute the generalized error using the leave one out Cross Validation score. This should be done by using kernel estimator ksmooth, local polynomials and smoothing splines. My first question is very general: Given the data how can I choose by eye a reasonable bandwith for the kernel estimator? See for example the following picture:

enter image description here

Since this data looks rather "wild" it is for me not clear how to choose a bandwidth. My first attempt was just to run ksmooth playing around with different bandwidth. But as I said, here the data is wild, so it is (for me) hard to determine a reasonable bandwidth.

The second problem is more concrete about the problem described above. So far I have the following code:

   cv <- function(data,used.function)
{
  n <- nrow(data)
  cv.value <- rep(0,length(n))
  for (i in 1:n){
      new.data <- data[-i,]
      cv.value[i] <- used.function(new.data[,1],new.data[,2],data[i,1])
  }
  ## MSE
  return(1/n*sum((new.data[,2]-cv.value)^2))
}


### kernel estimator usind nadaraya-watson:
fcn1 <- function(reg.x, reg.y, x){
  return(ksmooth(reg.x, reg.y, x.point = x, kernel = "normal", bandwidth = h)$y)
}
### CV-score for kernel estimator:
(cv.nw <- cv(real.data, fcn1))

the function cv should be general such that I can apply local polynomials and smoothing spline too. The variable real.data contains the data. It is a $n\times 2$ matrix which store all the $x$ values and $y$ values. In the function body of cv I perform a leave one out cross validation. However using this code gives for cv.nw a NA. What is wrong with my code? I'm very thankful for your help.

$\endgroup$
2
  • $\begingroup$ If you're looking for a reasonable starting value, check out the function dpill from the KernSmooth library. $\endgroup$
    – wcampbell
    Apr 18 '13 at 16:09
  • $\begingroup$ @wcampbell thanks for your comment. But I'm interested in finding an appropriate bandwidth by eye $\endgroup$
    – math
    Apr 19 '13 at 12:23
3
$\begingroup$

I realize this is a somewhat older question now but perhaps this can shed some light. There are several things wrong with your code.

  1. The code as it stands is NOT reproducible. At no point is h defined which is the exact parameter you wish to tune.
  2. You aren't iterating over different bandwidths. If you are using cross-validation to compare between different models, you need different models (i.e. different parameters).

Modifying your code and utilizing the cars dataset, here is a reproducible example of Leave-One-Out Cross-validation on bandwidth whereby you pass a 'grid' of different bandwidths to tune the model.

cv <- function(data, used.function, bandwidth.grid)
{
  n <- nrow(data)
  mse <- matrix(, nrow=length(bandwidth.grid), ncol=2)
  for (b in 1:length(bandwidth.grid)){
    cv.value <- rep(0, n-1)
    for (i in 1:(n-1)){
      new.data <- data[-i,]
      funcargs <- list(reg.x=new.data[,1],reg.y=new.data[,2],x=data[i,1], h = bandwidth.grid[b])
      cv.value[i] <- do.call(used.function, funcargs)
    }
    mse[b,] <- c(bandwidth.grid[b], 1/n*sum((new.data[,2]-cv.value)^2))
  }

  ## MSE
  colnames(mse) <- c("bandwidth", "mse")
  return(mse)
}

### kernel estimator usind nadaraya-watson:
fcn1 <- function(reg.x, reg.y, x, h){
  return(ksmooth(reg.x, reg.y, x.point = x, kernel = "normal", bandwidth = h)$y)
}

attach(cars)
### CV-score for kernel estimator:
cv(cbind(speed, dist), fcn1, seq(10))
> cv(cbind(speed, dist), fcn1, seq(10))
      bandwidth      mse
 [1,]         1 261.9555
 [2,]         2 223.3542
 [3,]         3 217.8303
 [4,]         4 214.0923
 [5,]         5 211.1874
 [6,]         6 211.4104
 [7,]         7 214.6941
 [8,]         8 220.1501
 [9,]         9 227.2262
[10,]        10 235.5479

Here we can see that a bandwidth = 5 would be best.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.