5
If this is the correct form, you could fit a linear regression with $E(Y|X) = b_0 + b_1 X_1 + b_2 X_2$ where $X_2$ is $X_1 \times X_1$.
This would present a relationship where $Y$ is quadratic in $X_1$ but linear in the beta parameters.
If you're not sure if a quadratic term is appropriate, and if you don't care much about interpreting the beta estimates ...
2
Your result is not generally true. For a counterexample, suppose a simple linear regression ($p=2$) and, for simplicity, $n$ even. Suppose that the vector $x$ of predictor values is $x=[-n/2,-n/2+1, \dotsc, n/2]^T$ (but omitting zero). To get the design matrix $\mathbf{X}$ augment a column of ones. Then we can calculate that
$$
\mathbf{X}^T \mathbf{X}=\...
2
Intuition:
Imagine that $X$ is only a single column relating to the intercept (estimating the mean of a distribution). Then it will be a vector of size $n $ with only ones and $X^tX=n $.
For $X $ a matrix with more columns it goes a bit the same. Each cell in the matrix $X^tX $ is the product of two vector of size $n $ and scales with $n $.
Sort of, it ...
1
In linear regression, you predict the target variable $y$ using a model that is based on a linear function of the predictors $X_1,X_2,\dots,X_k$
$$
y = \beta_0 + \beta_1 X_2 + \beta_2 X_2 + \dots + \beta_k X_k + \varepsilon
$$
In logistic regression, the target variable is binary, and you are trying to predict probability of "success" $p(y=1)$ using a ...
1
In multiple linear regression (MLR), you still have one dependent variable, and the relation is as follows:
$$y_i=\beta_0+\sum_{i=1}^n\beta_ix_i+\epsilon$$
In logistic regression (LR), what you predict is a probability, namely the class posterior, and is bounded unlike MLR output. They're similar in the sense that they both use linear combination of features....
1
First, you shouldn't use forward (or backward or stepwise) selection. They all have major problems, this has been discussed here many times. You seem to have a very large sample, this means that even weak relationships will be significant.
Second, in my opinion, whether you should transform Y by taking logs should depend on the substantive question, rather ...
1
1) The two approaches actually work in the same way, albeit in the first (multiple covariates) case the system of linear equations that are being solved is greater.
2) The outputs would be different but how different would depend on the correlation. The model with more covariates will invariably explain more variation, although this is not to say that it ...
Only top voted, non community-wiki answers of a minimum length are eligible
