New answers tagged

1

To see the residuals are the same consider that by definition of the residuals $$y = X_1\hat \beta_1 + X_2\hat\beta_2 + \hat u$$ which is the first regression and its residuals $\hat u$. Now multiply with $M_{X_1}$ to get $$M_{X_1}y = M_{X_1}X_2\hat\beta_2 + \hat u,$$ because $M_{X_1} X_1 = \mathbf 0$ and $M_{X_1}\hat u = \hat u - P_{X_1}\hat u = \hat u$ ...


0

OLS solves for BLUE (best linear unbiased estimator) only when all Gauss-Markov assumptions are met. You need a linear model, independence, identical distribution, exogeneity, and homoscedasticity. In scenarios, without linearity, we can still solve for a local minimum using gradient descent. Preferably, stochastic gradient descent with momentum. In terms ...


1

Assume you have $y\in \mathbf{R^n}$ and $X\in \mathbf{R^{n\times p}}$, $n>p$. The best subset selection solves the following minimization problem $\qquad \qquad \qquad \qquad \qquad \min_{\beta \in \mathbf{R^p}} ||y-X\beta||^2_n + ||\beta||_0 \qquad (1)$ where $||\beta||_0$ counts the total number of non-zero coefficients. You solve it by running all ...


1

Your OLS model: lm(formula = payment_amt ~ offset(years) + as.factor(gender) + age, data = pm) Is the same as: lm(formula = payment_amt - years ~ as.factor(gender) + age, data = pm) With a log-link, you can use offsets to model rates because of how math works with logarithms, but for an identity link there's really no point to using an ...


0

In an equation that you gave as example $Y_i=\beta_0+\beta_1(x_i-\bar x)+\varepsilon_i$, non invertibility of X'X means that $x_i$ is a constant, i.e. $x_i=\bar x$. The solution is then $\beta_0=\bar Y$ and $\beta_1$ any finite number (because $(x_i-\bar x)=0$). As you expect, it's a degenerated case with inifinite number of solutions.


0

Assuming there was autocorrelation you are dealing with the assumption that the error terms are not correlated with each other [they are independent of each other]. The impact of autocorrelation is to distort the statistical tests (by distorting the standard errors).


3

You've basically laid out the key facts, I think you just need a hint on how to fit them all together. Here's a quick-and-dirty overview. I think it's easier to see how to accomplish your goal if you build up from the Sherman-Morrison formula, which is just a special case of the Woodbury matrix identity. The Sherman-Morrison formula is a rank-1 update, ...


1

$$E(XY)=E(E(XY|X))=E(XE(Y|X))$$ $$E(X_i \mu_i)=E \left( E(X_i \mu_i|X_i) \right)=E \left(X_i E( \mu_i|X_i) \right)=E(X_i 0)=0$$


1

Assuming that you mean independent sampling when saying e = 100 random normal variables from N(0, 5), then there is no autocorrelation in your example. Assumptions such as independence in regression models apply to the error term, in your case e. What you do have in the example is a (strong) trend, so what you should be preoccupied with is if this ...


0

The following script fits a quadratic line in the plot (in blue): quadratic_model <- lm(D ~ GDPCAP + I(GDPCAP^2), data = mydata) order_id <- order(mydata$GDPCAP) lines(x = mydata$GDPCAP[order_id], y = fitted(quadratic_model)[order_id], col = "blue", lwd = 2) Coefficients of the quadratic equation are obtained as follows: ...


1

Your terminology is confused, you say gaussian link in GLM, but gaussian is the family, not the link function. In your case the link function is logarithmic. In your notation, you also left out the predictors $x$. So, with log link function and a linear predictor $\beta^T x$, the assumption is that $Y | X=x$ is distributed (independently) $\mathcal{N}(e^{\...


13

Ok. It's a bit long to include the whole proof here, so I will just sketch: Apply a first-order Taylor expansion around some, initially arbitrary point, $x_0$, $$y = m(x_0) + [x-x_0]'\nabla m(x_0,\theta) + R_1 + \epsilon.$$ where $R_1$ is the Taylor remainder. Set $$b_0 = m(x_0),\; b = \nabla m(x_0,\theta),\;\beta = (b_o, b)' $$ $$\tilde x = x-x_0,\; u =...


1

\begin{aligned} \min\limits_{\boldsymbol{b}} \boldsymbol{e}^T\boldsymbol{e} = (\boldsymbol{Y}-\boldsymbol{Xb})^T(\boldsymbol{Y}-\boldsymbol{Xb}) \\ \end{aligned} FONC: let $\boldsymbol{u} = \boldsymbol{Y}-\boldsymbol{Xb}$ \begin{aligned} \frac{\partial \boldsymbol{e}^T\boldsymbol{e}}{\partial{\boldsymbol{b}}} &= 0 \\ \frac{\partial [\boldsymbol{Y}^T-\...


2

Because OLS belongs in a subfield of mathematical optimization called convex optimization, and that field has some nice properties, such as every local minimum is a global minimum


0

The key is to use : W >= X^tX imply that W^-1 <= X^tX and you can multiply by X^tX (S.D.P) no problem the inequality still work .... Note that (A >= B imply that A - B is S.D.P)


2

Your time series is too short to fit that many parameters. Recall that, in a VAR(p) model \begin{equation*} x_t = d + A_1 x_{t-1} + A_2 x_{t-2} + \dots + A_p x_{t-p} + \epsilon_t, \end{equation*} each parameter matrix $A_i$ will have dimension $k\times k$, where $k$ is the number of parameters. Estimation simply procedes by OLS to each equation of the VAR ...


1

Because this problem is most instructive when you work out the answer yourself, I will suggest a logical sequence of attack but not provide a complete answer. Begin at the end: you want to find $R^2,$ which is defined as a ratio of variances. The numerator is the variance of the predicted values $\hat y_i$ while the denominator is the variance of the ...


Top 50 recent answers are included