# How can I assign a quality to a regression based on the number of data points?

Let's say I want to create a line of best fit to approximate the relationship between years of golf experience, and average golf score.

If I have only 4 data points, my line of best fit will have a lot of noise. Is there an equation I can use to say how good the line of best fit is based on the number of data points are used to create it?

I guess we could say quality = theNumberOfDataPoints, but it doesn't seem like a linear relationship to me... Is it maybe the square root of the number of data points?

You mention your line of best fit, so you are thinking graphically. You could also show the "quality" graphically.

In that case I would suggest plotting, along with the line of best fit itself, the upper and lower bounds of:

• the 95 % confidence interval (of the mean DV for different values of IV), and
• the 95 % prediction interval (of DVs predicted by the model from different values of IV).

There are some examples here: http://www.medcalc.org/manual/scatter_diagram_regression_line.php

...and here is a simple one for data just like yours, with

• 95 % confidence interval bounds in red
• 95 % prediction interval bounds in orange

R code:
(

a <- c(5,10,15,20)
score <- c(95,82,75,69)
plot(a,score)
model.lm <- lm(score ~ a)
abline(model.lm,col="grey30")
frame = data.frame(a,score)
newx <- seq(0,25)
prdConf <- predict(model.lm, newdata=data.frame(a=newx), interval = c("confidence"), level = 0.95, type="response")
prdPred <- predict(model.lm, newdata=data.frame(a=newx), interval = c("prediction"), level = 0.95, type="response")
lines(newx,prdConf[,2],col="red",lty=2)
lines(newx,prdConf[,3],col="red",lty=2)
lines(newx,prdPred[,2],col="orange",lty=2)
lines(newx,prdPred[,3],col="orange",lty=2)
# with help from [https://stat.ethz.ch/pipermail/r-help/2007-November/146285.html][3]


)

In my view plots like this should be actually be standard practice (especially the 95 % prediction interval) since it communicates the predictions made by the model so clearly, but I have only seen it rarely.

• Can you define DV and IV? I'm sorry if I missed that. – sooprise Aug 9 '13 at 15:47
• IV = independent variable (predictor variable (could refer to a manipulated variable or a covariate)); DV = dependent variable (outcome variable). I am sure I am forgetting other possible terms. Above, the IV is a (which is years of golf), and the DV is score. – A.M. Aug 9 '13 at 17:39

You could look at the variance in your residuals and see how long it takes for that to begin to converge:

x=NULL; y=NULL; N=1e2; v=NULL
for(i in 1:N){
new.x=runif(1)
x=c(x,new.x)
y=c(y,jitter(5*new.x))

m=lm(y~x)
v=c(v, var(m$residuals)) } plot(v, type='l', xlab='sample size', ylab='var(residuals)')  This isn't a measure of the quality of your model as much as it is a way of evaluating if you should expect your model to improve with more data. If you're just interested in model quality, for linear models the go-to evaluation of fit is generally the$R^2\$ statistic, but there are certainly others. As Justin mentioned, you can reference the F statistic and p-value.

When you run your model you should get an F statistic and p-value for the model overall (standard significance guidelines apply) and those also appear for each coefficient. They should take sample size into account.

Here's some more good information: http://blog.yhathq.com/posts/r-lm-summary.html