I'm in an AP statistics class, and several weeks ago we discussed regression lines. I understand that a regression line is the best fit for the the given data and that it can be used to predict variables. A least squares regression line can be used to determine the smallest possible value for squared residuals. Why is a line informative even when the data doesn't fall along the line?
5
-
2$\begingroup$ Are you asking, why squared residuals is more informative than some other measure such as absolute values or 4th powers? Or are you asking why is a line informative even when the data doesn't fall along the line? $\endgroup$– xanCommented Oct 29, 2016 at 15:35
-
$\begingroup$ @xan I'm asking why the line is informative when the data doesn't fall along the line. $\endgroup$– PhilipCommented Oct 29, 2016 at 15:36
-
$\begingroup$ That's a good question. Can you edit your post to clarify for folks that might not notice the comments? $\endgroup$– xanCommented Oct 29, 2016 at 15:39
-
$\begingroup$ @Philip Do you know what is measurement error? Can you give an example of measurement error? $\endgroup$– Jill CloverCommented Oct 29, 2016 at 17:05
-
$\begingroup$ @Philip The philosophy of regression analysis divides the measured data into two parts: the systematic part and the random part. For example, you car travelled at 55 miles/hour for 2 hours. 112(The measured distance) = 55*2 + 2(random error). $\endgroup$– Jill CloverCommented Oct 29, 2016 at 17:08
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
The objective of the least squares principle is to determine the parameters of the model such that the sum of the squared residuals is minimum. By residual we mean, the difference between the observed and the predicted value of the response variable. So, smallest squared residuals means the best possible prediction of the response variable. The regression model describes the existing functional relationship between the variables in the given set of observations.
Fitting a regression line to a given set of values means determining the parameter estimates of the model. These estimates are obtained to achieve some criterion. The criterion used in least squares principle is to obtain the parameter estimates such that the sum of the squared residuals is minimum. The regression line so obtained is the best fit in the sense of minimum sum of squared residuals.
Apart from describing the relationship between the variables, another use of the regression model is to predict the values of the response variable for some given values of the predictor variable.
$\begingroup$
$\endgroup$
It shed some light on unknown relationships in our life.
Suppose you are physics in 1600. You don't know what is free fall in Newtonian mechanics.
However, you are smart enough to figure out the falling distance $h$ has something to do with time $t$. But how?
You test many free fall at different heights, and get your data. You plot to see what's the relationship. It seems a linear model doesn't work.
$$h = \beta t$$
Then you tried
$$h = \beta t^2$$
This model matches your data much better. Then what is this $\beta$?
This $\beta$ can be guessed from your data. The best guess would the $\beta$ minimize the "distance" (forgive me) between the line and the data.
You find out $\beta = 5$ is the best answer. Then you will ask why 5? This is a magical number. Where does it come from? You begin your journey on finding reasonable explanation for this number. From now on, the question is no longer a statistics question. It is a physics question.
Our ultimate goal is to know what's behind what we see. However, science discovery is a long long journey. The fitted line could be a good starting point for us.
answered Oct 29, 2016 at 6:55