3
$\begingroup$

Given a simple linear model:

N <- 10
x <- rnorm(N)
y <- x + rnorm(N)
firstData <- data.frame(x, y)

interceptOnly <- lm(y ~ 1, firstData)
linearModel <- lm(y ~ x, firstData)

anova(interceptOnly, linearModel)
summary(linearModel)

It is possible to predict new values from the same model:

newX <- rnorm(N)
newY <- newX + rnorm(N)
newData <- data.frame(x=newX, y=newY)
newData$predictedY <- predict(linearModel, newData)

But how do you then evaluate the predicted values? Of course, you can put it in a new lm and say: newLinearModel <- lm(y ~ x, newData) But I would like to use the intercept and coefficient from the original model!

Does anyone know how to produce some kind of summary / anova based on the original intercept+coefficient but calculating the error from the new data?

$\endgroup$
7
  • 1
    $\begingroup$ Take the first model you created and calculate predictions using your newX. Then calculate the RMSE using newY. $\endgroup$
    – Steve S
    Sep 26 '15 at 8:28
  • $\begingroup$ Thanks for the reference to RMSE. How can I make a statement regarding RMSE? What is good/bad etc? I'm guessing I cannot get any p value here? $\endgroup$
    – trev
    Sep 28 '15 at 11:29
  • $\begingroup$ Would it be possible to do the anova "by hand", as if the coefficient and intercept from the first model were the "best fits" to the second model, and then use an F test as before? e.g. if firstData gave 5+6x fit then compare 5+6newX to the intercept only model? $\endgroup$
    – trev
    Sep 28 '15 at 11:34
  • $\begingroup$ Could you explain what you mean by "evaluate the predicted values" and "calculating the error from the new data"? $\endgroup$
    – whuber
    Oct 5 '15 at 13:38
  • $\begingroup$ Sure I'll try! :) Evaluate predicted values: Note that for the firstData I can evaluate the model fit. Using summary(), I can say that the coefficient x is significant, p < .05. In a more complex case where x is categorical, I could use anova() and compare to the intercept only baseline, and say the effect is significant (F-test, p value). I can get predicted values for new data with predict(). What I would like is a summary of the model for new data, so I can say that the linearModel from experiment 1 predicts the new data collected in experiment 2, p < .05. $\endgroup$
    – trev
    Oct 6 '15 at 8:45
2
$\begingroup$

I think you're using the predict function in the wrong way. When you use predict, it will look for independent variables with the same name as in the model. If there are no independent variables with the same name, it will just output the predicted values from your original x variable. Try this by typing predict(linearModel) and you'll see the same result as when you type predict(linearModel, newData). You need to name the newX variable x, just as your original x variable, for it to work.

newData$x <- newX
newData$predictedY <- predict(linearModel, newData)

Now you'll get the predicted values based on your new X variable and the intercept and coefficient for x from the model. You can calculate the residuals by newData$predictedY - newData$newY.

$\endgroup$
2
  • $\begingroup$ Sorry my example code was wrong. Well spotted! I've fixed it so that predict() uses the new x values now. $\endgroup$
    – trev
    Sep 28 '15 at 11:21
  • $\begingroup$ Now I would like to go from residuals to some kind of statement: the model fitted to data from exp1 was able to predict new data / was a poor predictor etc. $\endgroup$
    – trev
    Sep 28 '15 at 11:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.