What is a loss function? How can we relate the slope of Linear Regression with Sum of Squared Errors?


A loss function is a (real-valued) function which typically takes a data set and a certain fitted model.

It is typically used as a measurement to determine goodness-of-fit of the model. Multiple techniques are based on finding the model that minimizes such loss function.

SSE is used in linear regression because it directly relates to the portion of the variance of outcome $Y$ that is not explained (cannot be contributed) to the difference is the values of (the) predictor(s) $X$. It is a measure of 'predicability' of the $X$'s for the value of $Y$.

The SSE directly relates to the slope of a linear regression model because it is the sum of the squared deviations of a given $(X,Y)$ from $(X,\hat{Y})$ where $\hat{Y}$ is the predicted value based on the model and the given $X$.

| cite | improve this answer | |
  • $\begingroup$ Your answer would be better if you were explicit about how SSE "relates" to unexplained variance and the slope of the regression line. $\endgroup$ – Peter Leopold Mar 14 '19 at 14:46
  • $\begingroup$ This answer seems opaque and lacking in historical context, hence my downvote. $\endgroup$ – James Phillips Mar 14 '19 at 15:13

Not the answer you're looking for? Browse other questions tagged or ask your own question.