Questions tagged [blue]
Best Linear Unbiased Estimator
48
questions
2
votes
1
answer
82
views
In linear models, why are we focused on BLUE rather than UMVUE?
It seems more natural to talk about UMVUE (following the basic statistical estimation theory). But when we turn to lm, we only care about BLUE, why? Are there any insurmountable difficulties here?
- 93
0
votes
0
answers
26
views
Gauss-Markov-Theorem not holding true when error terms not normally distributed? [duplicate]
I tried to simulate some stuff around the Gauss-Markov-Theorem when I stumbled over something I cannot explain. I tried to simulate the density of an OLS estimator and a LAD (least absolute deviations)...
0
votes
2
answers
179
views
Logistic Regression: Median Imputing vs Mean Imputing
Missing values need to be imputed when performing logistic regression. Missing values are often imputed by the median. The mean is the BLUE (best linear unbiased estimator). But Median is robust to ...
- 267
2
votes
1
answer
143
views
Why would bootstrap OLS standard errors differ from ML estimate?
Let's say I have a regression dataset (paired x and y) such that the response variable (y) has an unknown distribution (but definitely not Gaussian) and is large enough such that the central limit ...
- 157
0
votes
1
answer
79
views
Least squares estimator of normally distributed observations
I am trying to comprehend what the authors are asking me to show in this following exercise problem from Hogg, McKean and Craig's book titled "Introduction to Mathematical Statistics."
3.5....
1
vote
1
answer
54
views
Check $cov(X,e)=0$ using residuals [duplicate]
I know it is wrong but I am not sure why. We run a linear regression
$$ Y = a + bX + e$$
we get the residual
$$ \hat{e} = Y -(\hat{a}+ \hat{b} X)$$
Why can we not check $cov(X,e)=0$ using $corr(X,\...
- 351
2
votes
0
answers
38
views
Simplistic linear estimator for a probability vector
I am working on a problem where the unknown probabilities $p_i$ are related to observed rates/frequencies $\pi_\alpha$ as
$$
\pi_\alpha = \sum_iW_{\alpha i}p_i,
$$
where $W_{\alpha i}$ is known (...
- 3,933
1
vote
0
answers
32
views
How to use the effects output of a MANOVA model to calculate BLUEs? (without an intercept)
So my team wants a single best linear unbiased estimate (BLUE) using 2 response variables. I have asremlR and here is my code to produce the model:
...
0
votes
0
answers
19
views
matrix result needed to show consistency of equation
I am trying to show the following system of equation
Va+Xb=0 and
X'a=k,
is consistent when a'X=k', where V is nXn and X is nXp (p<n) matrix, a,b,k are vectors.
I could not find the reason how Va ...
- 1
1
vote
1
answer
43
views
BLUE from calculus
Let $p'\beta$ be an estimable LPF. Suppose
that $l'y$ is the candidate which must satisfy the unbiasedness
condition and the minimum-variance condition. Formulate this as an optimization problem with ...
- 184
8
votes
4
answers
2k
views
What is the benefit of regression with student-t residuals over OLS regression? [duplicate]
Sometimes I see advice to fit regressions with student-t residuals rather than using OLS (which is equivalent to assuming normally distributed residuals) if the distribution of the residuals is heavy-...
- 1,337
7
votes
0
answers
421
views
In Ordinary Least Square (OLS) estimation: is the slope actually an "Inverse-variance weighting" estimator?
I am suspecting the answer is yes, but I'd appreciate help in proving it (even though we know that the estimator is BLUE, so it should probably hold).
For context:
An Inverse-variance weighting is ...
- 21.6k
1
vote
0
answers
389
views
What is an auxiliary density/distribution?
I am currently reading an academic paper where, without definition, the concept of an "auxiliary distribution" has been invoked. Additional expressions used are "auxiliary density ...
- 145
2
votes
1
answer
196
views
Exercises concerning a linear regression model with parameter $\beta$
I'm trying to solve the following exercises. I am unsure of what I'm doing so I appreciate any help.
My attempt:
(a) We have that
\begin{align*}
\boldsymbol{\hat \beta} &= (\boldsymbol X^T \...
- 571
5
votes
0
answers
184
views
Generalized Least Square When Disturbance Covariance Matrix Is Rank Deficient
I cannot find any general results on the following Generalized Least Square (GLS) problem.
Let $Y = X\beta + E$, where $X$ is deterministic and of full column rank $k$, and $E$ is of zero mean, with a ...
- 491
1
vote
0
answers
287
views
GLS estimator - derivation
I'm stuck with the following question:
Given the model $$Y_t=\alpha+\beta X_t+u_t\,,$$ where the standard assumptions hold but $Eu_t^2=\sigma^2 X_t^2$, derive the GLS estimator.
Basically, all Gauss ...
- 257
77
votes
1
answer
4k
views
Impractical question: is it possible to find the regression line using a ruler and compass?
The ancient greeks famously sought to construct geometrical relationships using only a ruler and a compass. Given a set of points in a two dimensional plane, is it possible to find the OLS line using ...
- 818
6
votes
2
answers
9k
views
Prove that the OLS estimator of the intercept is BLUE
Consider the simple linear regression model
$$y_i = \alpha + \beta x_i + u_i$$
with classic Gauss-Markov assumptions. In proving that $\hat{\beta}$, the OLS estimator for $\beta$, is the best linear ...
- 349
0
votes
1
answer
256
views
Linear mixed model high AIC [closed]
I try to create a Linear mixed model by comparing different combinations of random effects. However my AIC score is much higher then I find in tutorials (best 6541.42). Can I still use such a model to ...
- 103
0
votes
0
answers
60
views
Does the Gauss-Markov Theorem state that OLS is the only BLUE estimator?
I was reading through the proof on wiki. Which is the following.
\begin{align}
\operatorname{E} \left[ \tilde\beta \right] &= \operatorname{E}[Cy] \\
&= \operatorname{E} \left [\left ((X'X)^{-...
1
vote
0
answers
700
views
Finding Best Linear Unbiased Estimator
I have the doubt that if Gauss Markov theorem is applicable here since the Variance is not constant in the model. Without Gauss Markov Theorem, how can we obtain BLUE?
- 43
2
votes
1
answer
266
views
Bias-variance tradeoff question
From https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff, the derivation for the bias-variance decomposition of the squared error is
$$
E[(y - \hat{f})^2] = Bias[\hat{f}]^2 + \sigma^2 + Var[\...
- 705
1
vote
1
answer
87
views
What is the expectation of the product of 2 random variables (Gauss-Markov assumptions)?
In the two variable (intercept and slope) model:
among other, one of the Gauss-Markov assumptions is (in the BLUE framework of OLS)
My coarse slides state this implies that there is no correlation ...
- 155
1
vote
0
answers
377
views
Please verify whether the calculating process of the WLS estimator and the variance is correct or not
There are two Heteroscedasticity regression models
1.
$$ y_i = \beta x_i + \epsilon_i, \quad i=1, \ldots, n $$
where $\epsilon_i$'s are independent and distributed as $\epsilon_i \sim N(0, \sigma^2 ...
- 45
0
votes
0
answers
146
views
Why is linearity (as in linearity in BLUE) a desirable property for OLS estimators?
By linearity, I do not mean the linearity of the estimators in the OLS regression equation, what I am trying to ask is why are the beta coefficients a linear function of y(i)'s. Why is linearity a ...
0
votes
0
answers
142
views
When to use variance stabilizing method?
Let's suppose we want to estimate $p$ from $m$ independant realisations of $X\sim Bin(n,p)$: $x_1,x_2,\dots,x_m$, with respective size $n_i$ $i$ in $\{1,\dots,m\}$. Let $p_i$ be $p_i:=x_i/n_i$. To ...
- 173
3
votes
2
answers
477
views
What is relation betwen linearity assumption in OLS and L in BLUE (i.e. OLS is BLUE)
The linearity assumption says that the dependent variable is linear in parameters. When Gauss-Markov assumptions hold, OLS is BLUE, meaning smallest variance amongst Linear Unbiased estimators, where ...
- 51
2
votes
2
answers
2k
views
Why use OLS when it is assumed there is heteroscedasticity?
So I'm slowly going through the Stock and Watson book and I'm a bit confused on how to deal with the issue of homoscedacity/heteroscedacity. Specifically, it is mentioned that economic theory tells ...
- 168
0
votes
0
answers
39
views
Estimability in Design model
consider the design model $y=\theta+e$
I know we can obtain the normal equations from observation to estimate the parameters. my question is- is the estimation BLUE?
Given normal equations are:
$...
- 175
2
votes
1
answer
885
views
In a mixed model (asreml), are coefficients and predictions the same?
I am using asreml-R to model genotypic effects of crop field trials and I am confused on how to get best linear unbiased estimates for crop varieties of the model.
I've found two different ways how ...
- 656
10
votes
1
answer
3k
views
Is the OLS estimator the UMVUE (assuming Normality)?
Suppose
$$
\mathbf{y} = \mathbf{X} \mathbf{b} + \mathbf{e} \, ,
\\
\mathbf{e} \sim \mathcal{N}(0,\mathbf{I}_P) \, .
$$
We know that $\mathbf{\hat{b}} = (\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T \...
- 852
0
votes
0
answers
1k
views
Proving OLS estimator is BLUE in simplified model (deriving the variance)
A model is given as $Y = \mu + u_i$ where $u_i $ $IID(0,\sigma^2)$ with a sample of $n$ observations.
I have to prove that the OLS estimator for $\mu = \frac{\sum Y_i}{n}$ is the BLUE estimator.
I ...
- 203
5
votes
2
answers
1k
views
What's the difference between "Optimal linear predictor" and "best unbiased linear estimator"?
Greene (econometric analysis 7th ed. p 53) states that OLS is the "optimal linear predictor":
Then on the next page, he states that OLS is also the BLUE estimator (Gauss-Markov Theorem):
I ...
- 2,809
1
vote
0
answers
42
views
deduce variance from precision for bathymetry dataset
I mean to merge 2 gridded dataset of bathymetry using
BLUE (Best Linear Unbiased Estimate).
$merged = \frac{(\sigma_1^2 * dataset2 + \sigma_2^2 * dataset1)}{(sigma_1^2 + sigma_2^2)}$
I know the ...
8
votes
2
answers
5k
views
What are the properties of MLE that make it more desirable than OLS?
This question seems fundamental enough that I'm convinced it has been answered here somewhere, but I haven't found it.
I understand that if the dependent variable in a regression is normally ...
- 2,146
3
votes
4
answers
14k
views
Is OLS estimator the only BLUE estimator?
Gauss–Markov_theorem states that OLS estimator is a BLUE estimator. My doubt is can there be any other linear estimator, other than OLS, which is also a BLUE estimator?
After going through the proof ...
- 1,562
12
votes
2
answers
8k
views
Why is bias equal to zero for OLS estimator with respect to linear regression?
I understand the concept of bias-variance tradeoff. Bias based on my understanding, represents the error because of using a simple classifer(eg: linear) to capture a complex non-linear decision ...
- 5,603
1
vote
1
answer
5k
views
Proof that an estimator is linear
can u guys give some hint on how to prove that tilde beta is a linear estimator and that it is unbiased?
$$\tilde\beta=\frac1n\sum_{i=1}^n\frac{y_i-\bar{y}}{x_i-\bar{x}}$$
i have attempted to ...
- 21
4
votes
1
answer
1k
views
Role of Gauss-Markov Theorem in Linear Regression
How does being BLUE matter in Linear Regression for the coefficients? What does Heteroscedasticity Consistent & Auto-correlation Consistent Dispersion matrix take care off in this regard?
4
votes
3
answers
2k
views
Restricted OLS have less variance than OLS?
According to Gauss-Markov Theorem, ordinary least squares (OLS) is the best linear unbiased estimator (BLUE). How then can restricted OLS have less variance?
Please tell me the reason.
- 41
17
votes
1
answer
5k
views
Other unbiased estimators than the BLUE (OLS solution) for linear models
For a linear model the OLS solution provides the best linear unbiased estimator for the parameters.
Of course we can trade in a bias for lower variance, e.g. ridge regression. But my question is ...
- 3,711
1
vote
0
answers
229
views
Combining variance reduction techniques
I'm looking for some reference on the combination of various variance reduction techniques, in particular a best linear unbiased estimator. The only reference I have is McLeish - Monte Carlo ...
- 175
2
votes
1
answer
2k
views
How to Estimate the Error Term in a Heteroscedastic Model with Regression Through the Origin
Suppose we have a NO INTERCEPT model, $$y_i=\beta x_i+e_i$$
where $e_{i}$ follows a N(0,$\sigma^2 x_i^h$), so $e_i$ is equal in distribution to $e_{0i} x_i^{\frac{h}{2}}$, where $e_{0i}$ follows a N(...
- 530
26
votes
1
answer
20k
views
Why do the estimated values from a Best Linear Unbiased Predictor (BLUP) differ from a Best Linear Unbiased Estimator (BLUE)?
I understand that the difference between them is related to whether the grouping variable in the model is estimated as a fixed or random effect, but it's not clear to me why they are not the same (if ...
- 17.9k
8
votes
3
answers
15k
views
Proving Linear Estimator (beta) is BLUE?
In the book Statistical Inference pg 570 of pdf, There's a derivation on how a linear estimator can be proven to be BLUE.
I got all the way up to 11.3.18 and then the next part stuck me.
After ...
- 869
4
votes
1
answer
4k
views
Proof for "Least squares estimator is BLUE"
I checked all the books and on-line materials I could find for the proof, but found all of them have a derivation problem, which I cannot understand.
To prove the least squares estimator is the $...
- 51
11
votes
2
answers
3k
views
Gauss-Markov theorem: BLUE and OLS
I'm reading up on the Guass-Markov theorem on wikipedia, and I was hoping somebody could help me figure out the main point of the theorem.
We assume a linear model, in matrix form, is given by:
$$ y =...
- 852
5
votes
2
answers
6k
views
Why doesn't the Cramér-Rao lower bound apply?
Let $X_1, X_2, \dots, X_n$ be a sample of i.i.d. random variables, with density $$f_\theta=\frac{2}{3\theta}\left(1-\frac{x}{3\theta}\right) $$ for $0 < x < 3\theta$. And
$f_\theta=0$ if $ x <...
