Questions tagged [unbiased-estimator]
Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.
777
questions
0
votes
0
answers
18
views
Unbiased estimator of the reversed regression
Suppose I have random variables $X$ and $Y$. I am interested in an unbiased estimator of $\mathbb{E}[Y | X]$, but I performed regression the other way and have $\mathbb{E}[X |Y]$. Under what data ...
- 151
1
vote
0
answers
29
views
Motivation behind this exercise problem on complete sufficient statistic
This is from Hogg and McKean's "Introduction to Mathematical Statistics" Chapter 7 (Sufficiency), section 7.4 (Completeness and Uniqueness).
Exercise 7.4.10.
Let $Y_1 < Y_2 < \cdots &...
4
votes
2
answers
136
views
Deriving MSE($\hat{\beta}$) under Linear regression
I was able to derive the MSE, but there's a part of the derivation which I don't really get. Here's what I got:
Facts:
$\mathbb{E}(\hat{\beta})=\hat{\beta}\space$ (unbiased estimator)
$\text{Cov}(\...
- 55
2
votes
1
answer
39
views
Unbiased estimate of success probability
Typically we assume independence to estimate the probability of success i.e. probability of head in a coin tossing example. Means we toss a coin $n$ times and see how many times we get ...
- 21
1
vote
1
answer
134
views
Prove that $T$ is a complete statistic and find a UMVUE for $p$
While preparing for my prelims, I came across this problem:
Let $X_1, X_2,\cdots, X_n$ be a sequence of Bernoulli trials, $n \geq 4.$ It is given that, $X_1,X_2,X_3 \stackrel{\text{i.i.d.}}{\sim} Ber(\...
- 23
0
votes
0
answers
18
views
An example problem of converting a maximum likelihood problem into a restricted maximum likelihood problem
I have a question about this derivation. What is an example value of the actual matrix $A'$ such that $A'X=0$, $A'A=I$, and $\frac{1}{n}\Sigma((A'Y_{i}-mean(A'Y))^{2}=\frac{1}{(n-1)}\Sigma((Y_{i}-...
- 425
0
votes
0
answers
46
views
Unbiased Estimator of Nugget Effect
Question: I am trying the measure the nugget effect, which is parameterized by $(1-\lambda)$ in the following variance-covariance used to describe the multivariate normal distribution of my n-...
- 425
2
votes
0
answers
59
views
Unbiased estimator of mean divided by square root of second moment [closed]
Let $X$ be some random variable. Assume that
$$\mu = \mathbb{E}X,\,\delta = \sqrt{\mathbb{E}X^2}$$
are well defined and finite (in other words $X$ has first two moments). Now suppose that $X_1,...,X_n$...
- 121
2
votes
0
answers
191
views
What estimator and R package can be used for staggered difference-in-difference with (non-panel) cross-sectional data, controls and interactions
I am trying to run a difference-in-difference analysis in R. My data is non-panel, so I am reliant on a TWFE model where I have groups of individuals who are ...
- 65
1
vote
0
answers
45
views
Question on nonlinear least squares
Consider the following equation for $Y>0$:
$$
(1) \quad \log(Y)=\log(\gamma)+\log(\alpha+\beta X)+\epsilon.
$$
Assume that $E(\epsilon| X)=c\neq 0$. What are the consequences of this assumption on ...
- 891
0
votes
1
answer
104
views
Proving an Estimator of the sample variance to be MVUE
Question: Prove that $\hat{\sigma}_x^2=\displaystyle\frac{1}{N-1}\sum_{i=1}^N(X_i-\overline{X})^2$, with $\overline{X}=\frac{1}{N}\sum_{i=1}^N X_i$ is an unbiased, minimum variance estimator of the ...
- 103
1
vote
0
answers
62
views
Degrees of freedom for biased sample autocorrelation function
I want to find the expression for the a biased estimate of the autocorrelation function for a time series $X$, and am doing this from the biased estimated autocovariance function for lag $k$, divided ...
- 704
0
votes
0
answers
48
views
How to compute the bias of the auto-normalized importance sampling estimator
A preceding post has compared auto-normalized importance sampling with ordinal importance sampling. Beginner readers shall be directed there, but I will remind the readers of just enough elements for ...
0
votes
1
answer
223
views
Why weighted importance sampling is a biased estimator?
By simple math, we can have
$$
E_P[f(X)] = \sum_X f(x)p(x) = \sum_X f(x)\frac{p(x)}{q(x)}q(x) = E_Q[f(X)\frac{P(X)}{Q(X)}],
$$
which can be approximated by Monte Carlo sampling in two ways.
1. Normal (...
6
votes
1
answer
159
views
When to calculate the bias corrected geometric mean
Most sources give a simple equation to compute the geometric mean (GeoMean) of data samples from a lognormal distribution.
GeoMean = exp(m)
where m is the mean of ...
- 20.9k
7
votes
1
answer
73
views
On unbiasedness of an optimal forecast
Diebold "Forecasting in Economics, Business, Finance and Beyond" (v. 1 August 2017) section 10.1 lists absolute standards for point forecasts, with the first one being unbiasedness: Optimal ...
- 68.9k
0
votes
1
answer
61
views
Unbiased estimator for mean
The question: Given a random sample $X_1,...,X_n$ show that $\frac{1}{n}\sum_{i=1}^n X_i$ is an unbiased estimator for $E(X_1)$.
My confusion: Given a statistical model $(\Omega,\Sigma,p_{\theta})$, ...
- 135
1
vote
2
answers
107
views
Covariance of Best Linear Unbiased Estimators and arbitrary LUE
I'm working on a problem involving two linear unbiased estimators $T$ and $T'$ of a parameter $\theta$, defined from a sample $\{X_1, \dots, X_n\}$ with mean $\theta$ and finite variance. I aim to ...
- 13
4
votes
1
answer
151
views
Why does not this underlying hypergeometric distribution lead to unbiased estimators?
This example is take from Lippman's "Elements of probability and statistics".
Let N be the number of fish in a lake the warden wants to estimate. He catches 100 fish, tags them and releases ...
- 275
0
votes
0
answers
38
views
Do you change the mean / standard deviation when calculating the unbiased normalised autocorrelation function?
I am trying to calculate the unbiased normalised autocorrelation function. I think this field is a little complicated as different sources appear to use different nomenclature to describe the same ...
- 206
2
votes
0
answers
35
views
How to estimate the age of players correctly?
I have the data of players active on a gaming console and the playtime hours corresponding to the games they have played and their age. I want to analyze the top (say 10) games that the people between ...
- 313
9
votes
1
answer
96
views
Adjusted R2 and bias
Consider the population $R^2$:
\begin{equation}
\rho^2 = 1- \frac{\sigma^{2}_u}{\sigma^{2}_y}
\end{equation}
This equation describes the proportion of the variation in $y$ in the population explained ...
- 285
1
vote
0
answers
43
views
What are the uniformly minimum variance unbiased estimators (UMVUE) for the minimum and maximum parameters of a PERT distribution?
I believe the answers to this question are the sample minimum and the sample maximum, but I have not been able to find a reference or proof of this.
- 11
0
votes
0
answers
22
views
Taking a random sample from a non radom sample and checking it is unbiased
I have a large population and I want select a sample based on some characteristics hence the selected sample is not random since not all members of the population have an equal chance of being ...
- 21
1
vote
0
answers
42
views
Showing that the estimator of the log posterior used in stochastic gradient MCMC is unbiased
In the SGLD paper as well as in this paper it is claimed (paraphrasing) that
the following estimator:
$$\widetilde{U}(\theta) = -\dfrac{|\mathcal{S}|}{|\widetilde{\mathcal{S}}|} \sum_{{x}\in \...
- 33
0
votes
1
answer
124
views
Unbiased estimator for parameter of random variables following a uniform distribution [duplicate]
Suppose $X_i$ are i.i.d. and have density $f_\theta(x) = \frac{1}{\theta}$ if $x \in (\theta, 2\theta)$ for positive $\theta$.
$(\min_iX_i, \max_iX_i)$ is a sufficient statistic for $\theta$?
To ...
- 345
4
votes
1
answer
150
views
Unbiased estimator of $\sigma^4$
In the post [here], the user asked the question
$\{X_i\}_1^n$ is random sample from $N(\mu, \sigma^2)$ with unknown parameters. Find an unbiased estimator of $\sigma^4$.
The solution uses a property ...
- 1,636
0
votes
0
answers
64
views
Misunderstanding on the use of Popoviciu and von Szokefalvi Nagy's inequalities on the variance of a unbiased estimator
Let $X_1,\cdots,X_n$ be (discrete in my case) i.i.d. and bounded between $m$ and $M$. I'm interested in bounding the variance of an unbiased estimator:
$$\mathbb{V}\left[\frac1n\sum_{i=1}^nX_i\right]$$...
- 103
3
votes
1
answer
165
views
Is the sample mean an unbiased estimator of population mean in the presence of autocorrelation?
I've seen previous questions here that the sample mean can be considered an unbiased estimator of the population mean. e.g.1, 2.
While the examples seem to refer to independent sample points, it seems ...
- 539
0
votes
0
answers
26
views
How to prove that the MLE of a uniform distribution is biased using the formula given below? [duplicate]
I've calculated the MLE of the uniform distribution on [0,theta] as maxi{Xi} but don't know how to prove it is biased. The formula I have learned to prove it is unbiased is E(θ^)-θ=0.
Was stuck on how ...
- 1
2
votes
0
answers
54
views
Unbiased estimate of log-likelihood of Markov bridge
Note: I have cross-posted this question to MathSE.
I have the following problem I am trying to solve. I have a parametric family of "transition" distributions $p_\theta(x_{i+1}\mid x_i)$ and ...
3
votes
1
answer
125
views
Partially Endogenous Regressors
If I have a linear model
$$ Y = X_1 \beta_1 + X_2\beta_2 + e$$
where $X_1$ is endogenous to $e$ but $X_2$ is not, then simply performing OLS will yield an unbiased estimate for $\beta_2$ but not $\...
- 85
4
votes
1
answer
210
views
Unbiased estimators and moment of moments
Following section 7.4 of Rose and Smith "Mathematical Statistics with Mathematica" (book available online here), I'm trying to use the Fundamental Expectation Result (eq 7.15) and other ...
- 1,636
1
vote
0
answers
53
views
Proof of attenuation bias in multiple linear regression model
Consider the case of measurement error with a single explanatory variable measured with error
\begin{equation}
y=\beta_0 + \beta_1 x_1 + \beta_2 x_2 + ... + \beta_k x^{\ast}_k + \nu
\label{...
- 309
1
vote
0
answers
53
views
Unbiasness of OLS estimates under Stochastic Regressor
I found although the Gauss-Markov Theorms are so widely used, it has so many different versions. Appreciate it if anyone could help me clarify this specific question I have.
Given the OLS estimators:
$...
- 13
0
votes
0
answers
242
views
UMVUE for a Uniform distribution [duplicate]
How did we derive the PDF and CDF highlighted in green?
Thanks
0
votes
1
answer
85
views
The correct condition for OLS estimates to be unbiased?
For the ordinary least square (OLS) estimates of regression ($\vec{y} =\mathbf{X} \cdot \vec{\beta} + \vec{\epsilon}$) to be unbiased (without considering the efficiency), which one of the three ...
- 13
0
votes
4
answers
199
views
Unbiased estimator in no-intercept regression model
On an assignment I've been tasked with finding whether
$$\hat{\beta}_1=\frac{\sum_{i=1}^n x_iy_i}{\sum_{i=1}^n x_i^2},$$
the estimator for the slope of a no-intercept regression model $Y_i=\beta_1 X_i+...
- 103
1
vote
2
answers
42
views
How to obtain the local unbiased condition for an estimator from global unbiased condition?
A standard problem in classical statistics is to find a good estimator that minimizes a given cost function under certain conditions. Normally we want to require the estimator $\hat\theta$ to be ...
- 187
3
votes
1
answer
161
views
Estimating ratio of regression coefficients
What is the best method of estimating a ratio of regression coefficients $\beta_1/\beta_2$ under the usual assumptions / in practice? I have two relatively well approximated signals $X_1, X_2$ and ...
3
votes
1
answer
73
views
Tossing Until First Heads Outcome, and Repeating, as a Method for Estimating Probability of Heads
Consider the problem of estimating the heads probability $p$ of a coin
by tossing it until the first heads outcome is observed. Say we get $k_1$
tosses, then $U_1 = \frac{1}{k_1}$ is an estimate for $...
- 33
1
vote
0
answers
32
views
How to tell if a function is biased [closed]
I have a function that I want to see if its calculations are biased (I think, but an not certain, in the sense of an unbiased estimator).
So, I have (can generate) a set of numbers (drawn from a ...
4
votes
1
answer
157
views
Combine two estimates of same variable
Suppose I know that $ A = 0.3 \cdot B + 0.7 \cdot C $
And I have these estimates:
$$ \hat A = A + \varepsilon $$
$$ \hat B = B + \zeta $$
$$ \hat C = C + \eta $$
For the sake of the argument, $ \...
- 63
4
votes
1
answer
320
views
Cramer-Rao lower bound for the variance of unbiased estimators of $\theta = \frac{\mu}{\sigma}$
Let $X_1, \cdots, X_n$ be a sample from the $N(\mu, \sigma^2)$ density, where $\mu, \sigma^2$ are unknown.
I want to find a lower bound $L_n$ which is valid for all sample-sizes $n$ for the variance ...
- 63
0
votes
0
answers
43
views
How to esimate the mean and variance of data from a Pareto distribution
I have large sample of data that is approximately from a Pareto distribution with unknown parameters. Unfortunately the distribution is sufficiently heavy tailed that just taking the sample mean is ...
- 2,067
14
votes
3
answers
862
views
Best estimator of the mean of a normal distribution based only on box-plot statistics
Suppose $X_1,\ldots,X_n\sim\operatorname N(\mu,\sigma^2)$ and you can observe only the sample size $n,$ the two extreme values, and the first, second, and third quantiles of the sample. Among unbiased ...
- 10.7k
3
votes
2
answers
584
views
Confusion about the notation in Horvitz-Thompson estimator
I am a bit confused about the terminology used in the context of sampling of populations. The Horvitz-Thompson estimator, as well as the Hansen-Hurwitz estimator, for example, are examples of ...
- 33
1
vote
2
answers
51
views
estimator for standard error [closed]
I have searched extensively and I fail to find an answer. If you are not confident, please don't answer or modify the question, since it will confuse readers even more.
Agree on some definitions
Let $...
- 11
1
vote
1
answer
163
views
Best estimator for a binomial distribution
i have a data set that is being generate by a Bernoulli distribution let say $\mathbf{X} \sim x \in \{ 0,1 \}$, where x=1 with probability p. I dont have any information what is the 'p' parameter of ...
- 13
0
votes
1
answer
60
views
Statistical Properties of OLS Estimators (Unbiasedness)
In deriving the unbiasedness of OLS Estimators,
$$\hat{\beta_1} = \beta_1 + \frac {\sum_{i=1}^{n} (x_i - \bar{x})u_i}{\sum_{i=1}^n (x_i - \bar{x})^2}$$
My professor changes the above to:
$$\hat{\...
- 79