# Basic Linear Regression - Ensuring basic linear model beats naive model

I am new to regression and am working through some exercises.

I am trying to predict the dependent variable based off a time series of independent variables, which I believe are correlated.

date     dependent independent
january  0.655     15.0470029
february 0.911     15.27434021
march    1.15      15.46698656
april    1.38      15.62245827
may      1.62      15.86285276
june     1.95      16.12415496
july     2.09      16.29778048
august   2.35      16.47115632
sept     2.65      16.66489092
oct      predict?  16.94209902


Basic regression in Excel gives me an R^2 of 0.996487955

Linear formula from excel:

16.94209902*1.208521405+-17.54118113 = 2.933708174

I know the ACTUAL for oct is 2.88

How do I ensure that my independent variables actually have predictive signal? How can I apply a naive model to see if I am just capturing the linearity in the dependent data?

I am new to all of this, so please let me know if you need further clarity.

Thanks!

Do you only have 9 months of data? If so, that's way to little to make reasonable predictions. That doesn't say your October prediction might not be close, but that's more a matter of your time series being pretty featureless and having a (currently) steady growth rate.

Note how your independent variables have a sudden and huge jump in March, and how your dependent variable has a steady growth. These hint at the various issues that affect time series. (Note also that two increasing time series will have a reasonably high correlation even if they are totally unrelated.)

You are correct that values will be correlated. I'd suggest reading up on time series, and the term you're describing is called "autocorrelation". That is, May 2015 will be more similar to April 2015 than to Dec 2014 or even May 2014 (though see below).

You may also have "seasonality". That is, May 2013, May 2014 and May 2015 will tend to be similar.

Time series are tricky, and Ordinary Least Squares (OLS), which is probably what Excel is using, assumes that each data point is independent, which is not the case with most time series. You will fool yourself and get indications that are too optimistic. (The bottom line is your regression will think it has, say, 120 data points, but in reality because of autocorrelation and seasonality, you really have information equivalent a dozen or two points.)

The basic starting point for time series are ARIMA models, which you can also research. To be honest, time series are not easy, so you have some studying ahead of you, but it's a very interesting topic.

Linear regression with one independent and one dependent variable is nothing else just coming up with the a and b parameters in the formula y = a*x + b where x is the independent (explanatory) variable and y is the dependent variable.

In this particular example you don't use the month data, so you can safely ignore it (though you could use it in a more complicated model). Learn more about linear regression more on Wikipedia: https://en.wikipedia.org/wiki/Linear_regression

Since linear regression is an extremely simple model that happens to take place in 2 dimensions you can always plot it to make sure that a linear model makes sense.

This is your data and the fitted lin. reg. model plotted in 2D:

The blue line is the fitted model.

You can see that January and February are outliers (because all the other points seem to be along a straigt line) so in this particular case it would make to either ignore January and February or fit two models to the data, one for Jan-Feb and one for Mar-Sept.

I want to mention two things:

1. This is a very small data set, don't try to draw any conclusions from it.
2. The data seems to be synthetic (i.e. computer generated) because the points are on a straight line, this is rarely the case when you are working on a real world data set.

I hope this helps,

Peter

• Sorry, typos there. 15, not 5! – Jacob Jones Sep 14 '16 at 16:47