It is common to plot the line of best fit on a scatter plot when there is a linear association between two variables. One method of doing this is with the line of best fit found using the least-squares method. Another method would be to use a regression line that, which can be written as (y-mean(y))/SD(y) = r*(x-mean(x))/SD(x). What is the difference between these two models? I don't understand when to use one over the other. We also learned that the regression line always passes through the averages of the conditional y distributions of the data is football-shaped when plotted. Is this also the case for the least-squares line?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
Linear regression ends up being a lot more than this, but when you plot a “trend line” in Excel or do either of the methods you’ve mentioned, they’re all the same. The formula you give is a simple way of finding the regression equation that works in the particular case that you’re considering where there’s only one predictor variable. We use matrix algebra when the regression is more complicated with multiple predictors, and what you have is a special case of that full model.
I do not follow what you’ve written about a football, but the least squares regression line passes through $(\bar{x},\bar{y})$, yes.
-
$\begingroup$ When you see people post the equation $\hat{\beta}=(X^TX)^{-1}Xy$, that’s the full ordinary least squares regression equation using matrices. If that looks like “huh?” right now, that’s okay. You’ll get there if you continue studying statistics, but you’ll come across that plenty if you hang on on Cross Validated much. $\endgroup$– DaveOct 9, 2020 at 1:42
-
$\begingroup$ Right, and that equation assumes one solution under the condition of linear independence. I think his second statement is referring to a Maximum Likelihood probability for each x. $\endgroup$ Oct 9, 2020 at 1:51
-
$\begingroup$ What I mean by football shaped data is data that is normal in both X and Y. In that case, the equation (y-mean(y))/SD(y) = r*(x-mean(x))/SD(x) allows us to approximate the mean of y in a 'strip' of data in x. I am curious if the ordinary least squares line also passes through every mean value at every x value for the data if both x and y are normally distributed. $\endgroup$ Oct 9, 2020 at 1:52
-
1$\begingroup$ @FranklinV Ordinary least squares is a generalization of what you’re describing. What you call the equation is an ordinary least squares regression line in a particular case, so if you understand what it means to pass through the means for your equation, that also applies to OLS. (The real answer is that regression models a conditional or parameterized expected value. Again, you’ll get there, probably when you take a full-blown class on linear and generalized linear models (at least that’s when I learned it).) $\endgroup$– DaveOct 9, 2020 at 1:59