In R, if I write
lm(a ~ b + c + b*c)
would this still be a linear regression?
How to do other kinds of regression in R? I would appreciate any recommendation for textbooks or tutorials?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this communityIn R, if I write
lm(a ~ b + c + b*c)
would this still be a linear regression?
How to do other kinds of regression in R? I would appreciate any recommendation for textbooks or tutorials?
Linear refers to the relationship between the parameters that you are estimating (e.g., $\beta$) and the outcome (e.g., $y_i$). Hence, $y=e^x\beta+\epsilon$ is linear, but $y=e^\beta x + \epsilon$ is not. A linear model means that your estimate of your parameter vector can be written $\hat{\beta} = \sum_i{w_iy_i}$, where the $\{w_i\}$ are weights determined by your estimation procedure. Linear models can be solved algebraically in closed form, while many non-linear models need to be solved by numerical maximization using a computer.
This post at minitab.com provides a very clear explanation:
Response = constant + parameter * predictor + ... + parameter * predictor
I would be careful in asking this as an "R linear regression" question versus a "linear regression" question. Formulas in R have rules that you may or may not be aware of. For example:
http://wiener.math.csi.cuny.edu/st/stRmanual/ModelFormula.html
Assuming you're asking if the following equation is linear:
a = coeff0 + (coeff1 * b) + (coeff2 * c) + (coeff3 * (b*c))
The answer is yes, if you assemble a new independent variable such as:
newv = b * c
Substituting the above newv equation into the original equation probably looks like what you're expecting for a linear equation:
a = coeff0 + (coeff1 * b) + (coeff2 * c) + (coeff3 * newv)
As far as references go, Google "r regression", or whatever you think might work for you.
a
is a linear function of the four coefficients.
$\endgroup$
– whuber♦
Mar 24 '11 at 21:16
You can write out the linear regression as a (linear) matrix equation.
$ \left[ \matrix{a_1 \\a_2 \\a_3 \\a_4 \\a_5 \\ ... \\ a_n} \right] = \left[ \matrix{b_1 & c_1 & b_1*c_1 \\ b_2 & c_2 & b_2*c_2 \\b_3 & c_3 & b_3*c_3 \\b_4 & c_4 & b_4*c_4 \\b_5 & c_5 & b_5*c_5 \\ &...& \\ b_n & c_n & b_n*c_n } \right] \times \left[\matrix{\alpha_b & \alpha_c & \alpha_{b*c}} \right] + \left[ \matrix{\epsilon_1 \\\epsilon_2 \\\epsilon_3 \\\epsilon_4 \\\epsilon_5 \\ ... \\ \epsilon_n} \right] $
or if you collapse this:
$\mathbf{a} = \alpha_b \mathbf{b} + \alpha_c \mathbf{c} + \alpha_{b*c} \mathbf{b*c} + \mathbf{\epsilon} $
This linear regression is equivalent to finding the linear combination of vectors $\mathbf{b}$, $\mathbf{c}$ and $\mathbf{b*c}$ that is closest to the vector $\mathbf{a}$.
(This has also a geometrical interpretation as finding the projection of $\mathbf{a}$ on the span of the vectors $\mathbf{b}$, $\mathbf{c}$ and $\mathbf{b*c}$. For a problem with two column vectors with three measurements this can still be drawn as a figure for instance as shown here: http://www.math.brown.edu/~banchoff/gc/linalg/linalg.html )
Understanding this concept is also important in non-linear regression. For instance it is much easier to solve $y=a e^{ct} + b e^{dt}$ than $y=u(e^{c(t-v)}+e^{d(t-v)})$ because the first parameterization allows to solve the $a$ and $b$ coefficients with the techniques for linear regression.
lm()
stands for a linear regression. Your model includes three parameters (minus the intercept) forb
,c
, and their interactionb:c
, which stands forb + c + b:c
orb*c
for short (R follows Wilkinson's notation for statistical models). Fitting a Generalized Linear Model (i.e., where the link function is not identity, as is the case for the linear model expressed above) is requested throughglm()
. $\endgroup$ – chl Mar 30 '11 at 20:16