# Tag Info

0

I'm not sure why you would start with the assumption that b doesn't exist. Regardless of whether or not the relationship between Y and the X variables is linear, it is usually possible to find a value for the vector b (exceptions might be related to multi-collinearity and non-invertibility of X'X). This is "just" linear algebra: we are finding a ...

0

Let $y=X\beta+\epsilon$ and let $\hat{\beta}=(X^TX)^{-1}X^Ty$ be the OLS estimator. Denote $P_X$ as the projection matrix of $X$, we can then write the residuals as: $$e=y-\hat{y}=y-X\hat{\beta}=y-X(X^TX)^{-1}X^Ty=(I_n-P_X)y=Qy$$ Where $I_n$ is $n$-size identity matrix and $Q=I_n-P_X$. Note that $Q$ is also a projection matrix and thus is idempotent. Easily, ...

0

$\newcommand{\Cov}{\mathrm{Cov}}$ Calculate the analytical covariance matrix of the estimator for the parameters and see if the corresponding off-diagonal elements are zero. Then you know at least if the parameters are correlated or not, but not necessarily independent. If a Gaussian distribution defines the estimator, then off-diagonal elements equal to ...

5


5

In most cases we don't have priors that would be conjugate to the likelihood function and result in a closed-form solution. In such a case, we are left with approximate solutions, such as grid approximation for simple problems, and for more realistic ones Laplace approximation, Markov Chain Monte Carlo (MCMC) (see mcmc) sampling, variational inference (see ...

1

No, but something similar works. In the one-slope case you need two points to define a line. With $p$ parameters to estimate you need $p+1$ points. So, for any subset of $p+1$ points you get a unique $p$-plane, which defines a set of coefficient estimates, and you take a weighted average of these to get the OLS estimate. The weight needs to be (proportional ...

0

In the high-data limit, the average prediction variance is the (Residual Square Sum) / (number of data) + 2 * (number of parameters) according to the Stone (1977) paper which links least squares and the Akaike information criterion.

Top 50 recent answers are included