1
$\begingroup$

I've implemented a comparison between the performance of 80%-forecast intervals is in the forecast package - see 1st part of the code below providing a number of hits This number states, how many times the forcast interval was right for the left-out data entries. Btw regarding variable names: the German "preis" means "price" and "absatz" means "sales", i.e. "preise" means "prices" and "absaetze" is the plural for "sales".

So, I compared the formula-based prediction interval to what I think bootstrapping is - see 2nd part of the code. But the number of hits in the 2nd case by no means resembles the 80% of the first case. The following actions did not help to reproduce the 80% : using less data in the given data frame, using median formulas for bootstrapping instead of the upper/lower computation in the loop, more samples resampling in the resampling.

I cannot imagine the bootstrapping approach performing so bad - what did I do wrong?

#given

# data frame

preis<-c(1:100)
absatz<-(-2*preis)+1000+rnorm(100)


jeansData<-data.frame(absaetze=absatz,preise=preis)

#### implementation ###


#leave-one-out cross-validation for formula, i.e. with the borders         given     above  
###### (1ST PART) ########

numberOfHits<-0

for(i in (1:100)){

preisCandidateToBeChecked<-preis[i]
absatzCandidateToBeChecked<-absatz[i]

absatzWithoutCandidate<-absatz[-i]
preisWithoutCandidate<-preis[-i]

jeansData<-data.frame            (absaetze=absatzWithoutCandidate,preise=preisWithoutCandidate)
fit<-lm((absaetze~preise), data=jeansData)

#check, if in interval and count as hit, if value is in interval

if(absatzCandidateToBeChecked <= (forecast(fit,     newdata=preisCandidateToBeChecked)$upper[1]) & (absatzCandidateToBeChecked >= (forecast(fit, newdata=preisCandidateToBeChecked)$lower[1])) )
{numberOfHits<-numberOfHits+1}

}

#execute code until here and inspect numberOfHits; the hit rate pretty much resembles the 80% assumed

#then execute the rest

#leave-one-ot cross-validation for bootstrapping (not using the bootstrap function)  ###### (2ND PART) ########


numberOfHits<-0

for(i in (1:100)){

preisCandidateToBeChecked<-preis[i]
absatzCandidateToBeChecked<-absatz[i]

absatzWithoutCandidate<-absatz[-i]
preisWithoutCandidate<-preis[-i]

jeansData<-data.frame(absaetze=absatzWithoutCandidate,preise=preisWithoutCandidate)

#ten or hundred or thousand regressions by bootstrapping

allPredictions<-c()

for(j in (1:10)){

fit<-lm((absaetze~preise), data=jeansData[sample(nrow(jeansData),10,replace=TRUE),])

allPredictions<-c(allPredictions,forecast(fit,     newdata=preisCandidateToBeChecked)$mean)

}

#build and name bootstrapped forecast interval from regressions

upper<-sort(allPredictions)[9]
lower<-sort(allPredictions)[2]

if((absatzCandidateToBeChecked <= upper) & (absatzCandidateToBeChecked >= lower) )
{numberOfHits<-numberOfHits+1}

} #inspect numberOfHitsAgain - it's around 40%. What is foul here?!
$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.