New answers tagged

0

I'm not sure why you would start with the assumption that b doesn't exist. Regardless of whether or not the relationship between Y and the X variables is linear, it is usually possible to find a value for the vector b (exceptions might be related to multi-collinearity and non-invertibility of X'X). This is "just" linear algebra: we are finding a ...


0

Let $y=X\beta+\epsilon$ and let $\hat{\beta}=(X^TX)^{-1}X^Ty$ be the OLS estimator. Denote $P_X$ as the projection matrix of $X$, we can then write the residuals as: $$e=y-\hat{y}=y-X\hat{\beta}=y-X(X^TX)^{-1}X^Ty=(I_n-P_X)y=Qy$$ Where $I_n$ is $n$-size identity matrix and $Q=I_n-P_X$. Note that $Q$ is also a projection matrix and thus is idempotent. Easily, ...


0

$\newcommand{\Cov}{\mathrm{Cov}}$ Calculate the analytical covariance matrix of the estimator for the parameters and see if the corresponding off-diagonal elements are zero. Then you know at least if the parameters are correlated or not, but not necessarily independent. If a Gaussian distribution defines the estimator, then off-diagonal elements equal to ...


5

$\newcommand{\one}{\mathbf 1}\newcommand{\e}{\varepsilon}$I would just go for a linear algebra approach since then we get joint normality easily. You have $y = X\beta + \e$ with $X = (\one \mid x)$ and $\e\sim\mathcal N(\mathbf 0, \sigma^2 I)$. We know $$ \hat\beta = (X^TX)^{-1}X^Ty \sim \mathcal N(\beta, \sigma^2 (X^TX)^{-1}) $$ where $$ (X^TX)^{-1} = \...


3

No, the bias is not zero unless $c=0$. Bias is the difference between the expected value of the prediction (here $\hat\varphi_1 y_{t-1}$) and the deterministic part of the target (here $\varphi_1 y_{t-1}$). Given our estimator $\hat\varphi=\hat\varphi^{CLS}$ and the point of interest $y_{t-1}=c$, \begin{aligned} \text{Bias} &= \mathbb{E}[\hat\varphi_1^{...


2

This is largely covered elsewhere, e.g., in my answer to When can we speak of collinearity. Whether Pearson's $r$ is positive or negative makes no difference. I have never heard of your "minimum benchmark", and it doesn't make any sense to me. Consider that if you only had $4$ data, I gather your minimum benchmark would say that a pairwise ...


5

Let $y$ be income and $x$ be years of training. It sounds like you are thinking about $$ \log(y) = \beta_0 + \beta_1 x $$ Exponentiating both sides yields $$ y = B_0 \exp(\beta_1 x) $$ Where $B_0 = \exp(\beta_0)$ is the median income when individuals have 0 years of training (why median and not mean? I will get to that). So how do we interpret this ...


5

In most cases we don't have priors that would be conjugate to the likelihood function and result in a closed-form solution. In such a case, we are left with approximate solutions, such as grid approximation for simple problems, and for more realistic ones Laplace approximation, Markov Chain Monte Carlo (MCMC) (see mcmc) sampling, variational inference (see ...


1

No, but something similar works. In the one-slope case you need two points to define a line. With $p$ parameters to estimate you need $p+1$ points. So, for any subset of $p+1$ points you get a unique $p$-plane, which defines a set of coefficient estimates, and you take a weighted average of these to get the OLS estimate. The weight needs to be (proportional ...


0

In the high-data limit, the average prediction variance is the (Residual Square Sum) / (number of data) + 2 * (number of parameters) according to the Stone (1977) paper which links least squares and the Akaike information criterion.


Top 50 recent answers are included