2
$\begingroup$

Wikipedia gives the following definition for linear regression:

In statistics, linear regression is an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables (or independent variable) denoted X. The case of one explanatory variable is called simple linear regression.

What does the name actually mean? I.e. what do the 'linear' and 'regression' parts of the name actually stand for?

|cite|improve this question
$\endgroup$
  • $\begingroup$ Popular question without like 7+ upvotes, nice work guys I like this SE. This isn't a question that shows research effort. (I'm totally serious.) $\endgroup$ – Alec Teal Jul 28 '15 at 15:11
5
$\begingroup$

Linear means you are estimating a relationship by a straight line, or more correctly by linear multiples of the explanatory variables and a constant.

Regression is related to regression to the mean: the expected height of children of tall parents (adjusted for gender) estimated from linear regression is above average but lower than the heights of the parents; this has echoes of eugenics. Using Francis Dalton's data as illustrated in Wikipedia you get something like this

enter image description here

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Just to emphasize the linear part, linear regression is linear in the parameters not necessary the data. That is the model y= $\beta_{0} +\beta_{1}x^{100} +\epsilon$ is a linear model but the model y= $\beta_{0} +\beta_{1}^{2}x +\epsilon$ is not. $\endgroup$ – Nick Thieme Jul 28 '15 at 14:49
  • $\begingroup$ Can you explain your funky graph please? $\endgroup$ – Alec Teal Jul 28 '15 at 15:12
  • $\begingroup$ @AlecTeal: It is not my graph, but heights are in inches and the graph type is a sunflower plot with strokes coming out of each point (strictly speaking a range) representing the number of observations there. Look at cran.r-project.org/web/packages/HistData/HistData.pdf under "Galton" for some example code $\endgroup$ – Henry Jul 28 '15 at 18:50
1
$\begingroup$

"Linear" refers to the type of the model, $$ y=\beta_0 + \beta_1 x_1 + \beta_2 x_2 +\ldots + \beta_m x_m. $$ to understand the meaning of "linear" without going to higher dimensions, let's have just one $x$ and assume $\beta_0=0$. Then above formula reduces to,

$$ y= \beta_1 x_1 $$ as you see this is a line with slop of $\beta_1$ in 2D space. This models simply means, by knowing a coefficient, $\beta_1$ and having some values for $x_1$, then you can find (maybe predict with some uncertainty) the value of the left hand side (usually called response variable). This procedure is called regression in literature.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.