Questions tagged [unbiased-estimator]

Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

Fisher Information with respect to the Standard deviation of Normal distribution

Let $X\sim\mathcal{N}(0,\sigma^2)$ be given. I computed the Fisher Information to be $I(\sigma)=\frac{2}{\sigma^2}$. Note that the Fisher Information for the variance is given by $I(\sigma^2)=\frac{1}{...
0
votes
0answers
4 views

Using probability estimates as independent variables in a multinomial model

I'm testing market efficiency on a fixed odds betting market, and a way of carrying out efficiency test as a statistical hypothesis is by using the multinomial model. However, I'm used to fit ...
2
votes
1answer
26 views

Estimating the time between samples

Let's say that an event happens every $\Delta x$ seconds and we sample the time $x$ with some error. The events are not sampled reliably so some are missing in the sample set. However, we do know the ...
2
votes
1answer
35 views

Why is the MLE for variance in single linear regression biased? [duplicate]

I understand that the Maximum Likelihood Estimator for variance, in general, is biased (the average calculated from the sample itself reduces the degree of freedom by 1 e.t.c): ...
1
vote
0answers
28 views

How to find an minimum variance unbiased estimator for an integer parameter?

Consider multiple observations x[n] for an integer parameter A under White Gaussian Noise w[n]: x[n]=A+w[n]; n=0,1,...,N−1 with w[n] ~ N(0,σ^2). Is it possible to have an minimum variance ...
0
votes
1answer
19 views

Error distributions and consistent and unbiased OLS

If OLS estimator is unbiased and consistent, what does it imply about the distribution of error terms? In linear regression model: $ y_i = \boldsymbol{x_i' \beta} + \epsilon_i $ if the OLS estimator ...
0
votes
0answers
10 views

OLS with asymmetrically distributed disturbances

My question is related to this post: [[here]1]1 parameters-for-a-regression-with-asymmetric-and-non-zero-dis However, I want to know if it's possible to get unbiased and consistent OLS estimator if ...
0
votes
1answer
36 views

Coverage probability for Wald confidence interval with small sample size

a) My understanding is that the sample variance (i.e. the squared deviation divided by n - 1 instead of n) constitutes a mean-unbiased estimator of population variance. However, I've run multiple ...
0
votes
1answer
49 views

Unbiasedness and consistency of OLS

Does unbiasedness of OLS in a linear regression model automatically imply consistency? Edit: I am asking specifically about the assumptions for unbiasedness and consistency of OLS. If the assumptions ...
8
votes
1answer
177 views

When can't Cramer-Rao lower bound be reached?

The Cramer-Rao lower bound (CRLB) gives the minimum variance of an unbiased estimator. One sentence in the wiki page says "However, in some cases, no unbiased technique exists which achieves the bound....
4
votes
2answers
151 views

Sample Variance and Dividing by $n-1$

In this video... https://www.youtube.com/watch?v=sHRBg6BhKjI ...and in many others, the explanation for why when calculating the sample variance we divide by $n-1$ instead of by $n$ is the following:...
5
votes
2answers
135 views

Unbiased estimator of the ratio of variances

Consider two samples $X_{1}, \cdots ,X_{m}$ and $Y_{1}, \cdots ,Y_{n}$ where $X_{i} \thicksim N(\mu_{1}, \sigma_{1}^2), i.i.d.$ and $Y_{j} \thicksim N(\mu_{2}, \sigma_{2}^2), i.i.d.$. Say that both $\...
0
votes
1answer
10 views

what is the issue with using sample count of unique values to estimate population count of unique values?

Let's say I am drawing a random sample of n values from a population with N values, and in the population, U of them are unique. If I observe u unique values in the sample, why can't I use that to ...
0
votes
1answer
49 views

Why is $\frac1n\sum_{i=0}^{n-1}\sum_{j=0}^i1_{\{\:X_i\:=\:Y_j\:\}}f(Y_j)$ an equivalent representation for the usual Metropolis-Hastings estimator?

At the beginning of section 2 of the paper A Vanilla Rao-Blackwellization of Metropolis-Hastings Algorithms, the usual Metorpolis-Hastings estimator of $\int f$ given by the ergodic average $\frac1n\...
0
votes
1answer
27 views

Finding the conditional distribution of single sample point given sample mean for $N(\mu, 1)$

Suppose that $X_1, \ldots, X_n$ are iid from $N(\mu, 1)$. Find the conditional distribution of $X_1$ given $\bar{X}_n = \frac{1}{n}\sum^n_{i=1} X_i$. So I know that $\bar{X}_n$ is a sufficient ...
2
votes
2answers
44 views

Unbiased estimator of the third central moment

An unbiased estimator of the variance is $$ \frac{\sum_{i=1}^N (X_i - Mean(X)) ^2}{N-1} $$ where $X_i$ is observation $i$ and $N$ is the number of observations. Am I right that an unbiased ...
2
votes
0answers
9 views

Alternative to plug-in estimation for log-tranformed linear model

I want to estimate a relationship of the form: $$y=ax^b\times\epsilon$$ If I log this model i get: $$\log(y)=\log(a)+b\log(x)+ \log(\epsilon)$$ If I then proceed and estimate this model using a ...
3
votes
2answers
86 views

demeaning or differentiation in fixed-effect equation

I have panel data (with short time-dimension $T>2$) and I consider a simple model of the form: $$y_{i,t} = x_{i,t} \beta + c_i + \epsilon_{i,t}$$ where $E(x_{i,s}\epsilon_{i,t})=0$ and $\epsilon_{i,...
0
votes
0answers
13 views

Unbiasedness and Variance of Predictions

Here is the problem I'm working on: I'm not quite sure if I'm showing either unbiasedness property right, and am stuck on finding the expressions for the variances. Here's what I've done so far. (a) ...
0
votes
0answers
28 views

unbiased estimator for a parameter

1000 nodes are randomly sampled from a very large computer network and their “degree” (number of connections to other nodes) is recorded. N= 1000 Mean= 68.266 Variance= 8807.2905 maximum=1294 min= ...
0
votes
1answer
25 views

Why is having low variance important in offline policy evaluation of reinforcement learning?

Intuitively, I understand that having an unbiased estimate of a policy is important because being biased just means that our estimate is distant from the truth value. However, I don't understand ...
0
votes
0answers
26 views

Which is the better estimator for standard deviation?

Let $X_i \sim^{\textrm{iid}} N(\mu, \sigma^2)$. If I have measured $n$ values of $\textrm{std}(X_i)$ as $\sigma_1,\cdots,\sigma_n$, then what is the better estimator for $\sigma$: $$\hat{\sigma}_1 = ...
0
votes
0answers
82 views

Unbiasedness of OLS estimator

Consider a very simple linear regression model with following assumptions: 1) No assumptions on how $x_i$ is generated (assume random design, not necessarily IID); 2) $\mathbb{E}[\epsilon_i|x_i]=0$, ...
2
votes
1answer
45 views

Does a doubly robust estimator magnify bias if *both* the outcome regression and inverse propensity score weighting are incorrect models?

The doubly robust estimator is a popular method for measuring the average treatment effect with observational data (assuming no unmeasured confounders): $$ \hat{\Delta}_{DR} = n^{-1}\sum_{i=1}^n \...
3
votes
1answer
79 views

Biased prediction (overestimation) for xgboost

I run xgboost and elastic-net on the same dataset for a classification problem, say we have ...
2
votes
1answer
74 views

When are biased estimators with lower MSE preferred? [duplicate]

From wikipedia https://en.wikipedia.org/wiki/Bias_of_an_estimator : because a biased estimator gives a lower value of some loss function (particularly mean squared error) compared with unbiased ...
0
votes
0answers
22 views

Analytical unbiased estimates for bootstraping

I've realized that calculating average of a central statistic over all possible bootstrap samples is equivivalent to using an unbiased version of it (sample central). Meaning that given a sample $\vec{...
2
votes
1answer
37 views

Unbiasedness of Bayesian Posterior Mean Under Bayesian and Frequentist Models [duplicate]

This is an extension to this previous question, and is related to exercise 4.7 from Gelman et al.'s BDA3. When is the Bayesian posterior mean $m(y) \equiv E[\theta \mid y]$ unbiased for $\theta$, ...
1
vote
0answers
115 views

Maximization bias in reinforcement learning

In Richard S. Sutton and Andrew G. Barto's book on reinforcement learning on page 156 it says: Maximization bias occurs when estimate the value function while taking max on it (that is what Q ...
11
votes
2answers
944 views

Unbiased estimator of exponential of measure of a set?

Suppose we have a (measurable and suitably well-behaved) set $S\subseteq B\subset\mathbb R^n$, where $B$ is compact. Moreover, suppose we can draw samples from the uniform distribution over $B$ wrt ...
2
votes
1answer
53 views

Using IPS(inverse probability weighting) with a deterministic policy as the logging policy

In a contextual bandit problem, why can't we use inverse probability weighting (inverse propensity score) with a deterministic policy as the logging policy? Could you give me a concrete example?
0
votes
1answer
53 views

Proof for how the drift estimator, for a random walk with drift, is unbiased?

Random walk with drift formula is: (Yt = α + Yt-1 + εt ) How do I go about checking that the drift estimator α-hat is unbiased.. which is proving that E(α-hat) = α? Is this something I would need ...
0
votes
1answer
22 views

Aggregating small area estimates to match a truth

Small area estimation are related group of techniques in the estimation of parameters associated with a sub-population. For example, suppose I have sub-populations $S_1, \cdots, S_n$ with total ...
3
votes
0answers
39 views

Who invented train/validation/Test method and when?

I can't seem to find here or in other places the earliest source for this method. it seems the holdout method was separately proposed by Highleyman in 1962, and cross validation was separately ...
4
votes
0answers
27 views

Bootstrapping for Control Variates

TLDR: I want to do Monte Carlo with control variates I work in the setting where you use Monte Carlo sampling to approximate the optimal coefficients for the control variates. I want to retain the ...
0
votes
0answers
20 views

Understanding signal-to-noise with isodensity ellipsoids

I am trying to understand how to plot an isodensity ellipse and recreate a graph similar to the following (by Belsley 1982): The graph seems to be made using a piecewise function of possible values ...
0
votes
1answer
34 views

Maximum Likelyhood Estimator (MLE) vs. Bias [duplicate]

If we use the MLE methhod to find the estimator of the variance we get: $\hat\sigma ^2 = \frac{\sum(x_i - \mu)^2}{n}$ Where we can plug the MLE estimator for $\mu$ and get: $\hat\sigma ^2 = \frac{\...
1
vote
6answers
110 views

Does random sampling implies non-biased results?

Let's say that we want to estimate how much deforestation is done in the world considering 2 moments in time. We have a satellite that can take pictures anywhere in the world, but not of the entire ...
0
votes
0answers
67 views

What are biased and inefficient estimators?

I’m studying statistics from Schaum’s Outline, which gives the following: ...
2
votes
2answers
76 views

Proof of (weak) consistency for an unbiased estimator

I want to prove a theorem stating: An unbiased estimator $\hat{\theta}$ of the unknown parameter $\theta$ is consistent if $V(\hat{\theta}_n$) $\to0$ for ${n\to\infty}$. I've tried using the ...
1
vote
0answers
35 views

cross-covariance estimation and variance reduction

Let $X,Y$ be two vector variables and $$ \mathrm{Cov}(X,Y) = \mathbb{E}[(X-\mathbb{E}X)(Y-\mathbb{E}Y)^T] $$ their cross-covariance (but I think we could just pretend that's the covariance between two ...
2
votes
2answers
114 views

Estimator with variance equal to Cramér-Rao lower bound in $N(x_i\theta,1)$-distribution

Let $Y_1,\ldots, Y_n$ be independent and $N(x_i\theta,1)$ distributed, with for each $Y_i$ a mean of $x_i\theta$ for known $x_1,\ldots,x_n$. In a previous section of this exercise I found that the ...
1
vote
2answers
49 views

Estimator of binomial probability from poisson number of trials

I'm a lowly physicist, so I hope you will forgive me if I botch some terminology and notation here. I perform an experiment where on each trial I start with some number of particles $N$ that is ...
0
votes
0answers
16 views

Conditional model with correlation between estimations

I am trying to estimate Click-Through-Rate (CTR) given the following two models: $$P(Click|Visible)$$ $$P(Visible)$$ The output is: $$P(Click) = P(Click|Visible)*P(Visible)$$ Unfortunately, $$P(...
1
vote
0answers
157 views

What is the difference between BLUE and MVUE?

What is the difference between a Best Linear Unbiased Estimator (BLUE) and a Minimum Variance Unbiased Estimator (MVUE)? I know that "best" mean efficient, but that is also what "minimum variance" ...
0
votes
0answers
40 views

Unbiasedness of Intrumental Variable Estimator

is instrumental variable estimator unbiased in the case of stochastic regressor X and how can i show this? (I know how to show consistency, i need to show unbiasedness)
0
votes
0answers
12 views

When is the bias of a statistic of the form a/n + b/n^2 + c/n^3 +

In many books the bias-correction of the Jackknife resampling method is being prooved under the assumption, that the bias has a special form, namely a/n + b/n^2 + c/n^3 * ... Sometimes it's written "...
1
vote
0answers
28 views

Do unbiased regression coefficents yield better prediction?

I ask myself if a have a omitted variables bias in my regression modell the coefficients of the model are biased so the mse growth because this coefficents are biased right? So does it mean if i ...
0
votes
0answers
26 views

Find $k \in R$ such that $P\left(\max\left\{\frac{{S_x}^2}{{S_y}^2}, \frac{{S_y}^2}{{S_x}^2}\right\} > k\right)= 0.05$

Let $\overline{X}$ and $\overline{Y}$ sample means and ${S_x}^2, {S_y}^2$ unbiased estimators for the variance of 2 independent random samples of size 7 with normal distribution with mean unknown and ...
0
votes
1answer
48 views

If sample standard deviation is biased, why do we use it in typical mean tests? [duplicate]

If sample standard deviation is biased, why do we use it in typical mean tests? Why do we not use an unbiased estimator by dividing the sample standard deviation by C4?