Questions tagged [estimators]

A rule for calculating an estimate of a given quantity based on observed data [Wikipedia].

Filter by
Sorted by
Tagged with
129
votes
3answers
140k views

What is the difference between a consistent estimator and an unbiased estimator?

I'm really surprised that nobody appears to have asked this already... When discussing estimators, two terms frequently used are "consistent" and "unbiased". My question is simple: what's the ...
21
votes
4answers
32k views

What is the relation between estimator and estimate?

What is the relation between estimator and estimate?
38
votes
2answers
10k views

When is a biased estimator preferable to unbiased one?

It's obvious many times why one prefers an unbiased estimator. But, are there any circumstances under which we might actually prefer a biased estimator over an unbiased one?
25
votes
2answers
6k views

Correlation between OLS estimators for intercept and slope

In a simple regression model, $$ y = \beta_0 + \beta_1 x + \varepsilon, $$ the OLS estimators $\hat{\beta}_0^{OLS}$ and $\hat{\beta}_1^{OLS}$ are correlated. The formula for the correlation ...
10
votes
4answers
2k views

How does one explain what an unbiased estimator is to a layperson?

Suppose $\hat{\theta}$ is an unbiased estimator for $\theta$. Then of course, $\mathbb{E}[\hat{\theta} \mid \theta] = \theta$. How does one explain this to a layperson? In the past, what I have said ...
13
votes
1answer
2k views

How does one show that there is no unbiased estimator of $\lambda^{-1}$ for a Poisson distribution with mean $\lambda$?

Suppose that $ X_{0},X_{1},\ldots,X_{n} $ are i.i.d. random variables that follow the Poisson distribution with mean $ \lambda $. How can I prove that there is no unbiased estimator of the quantity $ \...
11
votes
1answer
3k views

What's the difference between asymptotic unbiasedness and consistency?

Does each imply the other? If not, does one imply the other? Why/why not? This issue came up in response to a comment on an answer I posted here. Although google searching the relevant terms didn't ...
10
votes
3answers
9k views

Why is OLS estimator of AR(1) coefficient biased?

I am trying to understand why OLS gives a biased estimator of an AR(1) process. Consider $$ \begin{aligned} y_{t} &= \alpha + \beta y_{t-1} + \epsilon_{t}, \\ \epsilon_{t} &\stackrel{iid}{\...
30
votes
9answers
31k views

What is the difference between an estimator and a statistic?

I learned that a statistic is an attribute you can obtain from samples.Taking many samples of same size, calculating this attribute for all of them and plotting the pdf, we get the distribution of the ...
5
votes
1answer
303 views

Estimator that is optimal under all sensible loss (evaluation) functions

Consider a probability distribution $D$ with a parameter $\theta$ and an i.i.d. sample $S$ from that distribution. I am interested in an estimator $\hat\theta(S)$ of $\theta$ that satisfies the ...
8
votes
1answer
6k views

root-n consistent estimator, but root-n doesn't converge?

I've heard the term "root-n" consistent estimator' used many times. From the resources I've been instructed by, I thought that a "root-n" consistent estimator meant that: the estimator converges on ...
2
votes
1answer
346 views

Standard error of the estimate in logistic regression

We usually get an estimate of $\beta$ in the logistic regression by finding the $MLE$ of the observed random samples of $X_1,X_2....,X_N$. Then we use Wald's test i.e. ${[\hat \beta / S.E.(\hat \beta)]...
8
votes
0answers
1k views

Why don't asymptotically consistent estimators have zero variance at infinity?

I know that the statement in question is wrong because estimators cannot have asymptotic variances that are lower than the Cramer-Rao bound. However, if asymptotic consistence means that an estimator ...
7
votes
1answer
252 views

MLE of $f(x;\alpha,\theta)=\frac{e^{-x/\theta}}{\theta^{\alpha}\Gamma(\alpha)}x^{\alpha-1}$

Let $X_{1},X_{2},X_{3},...,X_{n}$ be a random sample from a distribution with pdf $$f(x;\alpha,\theta)=\frac{e^{-x/\theta}}{\theta^{\alpha}\Gamma(\alpha)}x^{\alpha-1}I_{(0,\infty)}(x ),\alpha,\theta&...
8
votes
1answer
5k views

Is the residual, e, an estimator of the error, $\epsilon$?

This question has come up in another thread that I started so I thought I would get more people's opinions on it. My question is Is the residual, e, an estimator of the error, $\epsilon$? The reason ...
2
votes
1answer
589 views

Why do we divide by the degree of freedom?

This might be trivial and vague question, but I still don't understand why when creating test statistics or estimators we always divide by the degree of freedom. Just to give examples of what I'm ...
22
votes
2answers
8k views

What is the oracle property of an estimator?

What is the oracle property of an estimator? What modelling goals is the oracle property relevant for (predictive, explanatory, ...)? Both theoretically rigorous and (especially) intuitive ...
22
votes
2answers
964 views

Shrunken $r$ vs unbiased $r$: estimators of $\rho$

There has been some confusion in my head about two types of estimators of the population value of Pearson correlation coefficient. A. Fisher (1915) showed that for bivariate normal population ...
10
votes
2answers
2k views

Why is an estimator considered a random variable?

My understanding of what an estimator and an estimate is: Estimator: A rule to calculate an estimate Estimate: The value calculated from a set of data based on the estimator Between these two terms, ...
21
votes
1answer
873 views

Anscombe-like datasets with the same box and whiskers plot (mean/std/median/MAD/min/max)

EDIT: As this question has been inflated, a summary: finding different meaningful and interpretable datasets with the same mixed statistics (mean, median, midrange and their associated dispersions, ...
5
votes
2answers
425 views

Drawing numbered balls from an urn

PROBLEM There is an urn with a set of balls where each ball is labeled with a different integer. The numbers on the balls are known and are not a range of integers. For example the set of balls could ...
20
votes
2answers
1k views

Is there a statistical application that requires strong consistency?

I was wondering if someone knows or if there exists an application in statistics in which strong consistency of an estimator is required instead of weak consistency. That is, strong consistency is ...
6
votes
2answers
3k views

Why do we use Vector Autoregressive Models?

Let's say we want to estimate the system $x_{1,t}=\phi_0+\phi_1 x_{1,t-1}+\phi_2 x_{2,t-1} +\epsilon_t$ $x_{2,t}=\gamma_0+\gamma_1 x_{1,t-1}+\gamma_2 x_{2,t-1} +\eta_t$ Do we gain anything be ...
4
votes
2answers
274 views

T-consistency vs. P-consistency

Francis Diebold has a blog post "Causality and T-Consistency vs. Correlation and P-Consistency" where he presents the notion of P-consistency, or presistency: Consider a standard linear regression ...
6
votes
1answer
167 views

Consistency of M-estimator based on plug-in estimator?

Suppose we estimate a quantity $\theta_0$ by the $\tilde{\theta} = \hat{\theta}(\eta)$ that solves the estimating equation $$S_n(\tilde{\theta}, \eta_0) = 0$$ where $\eta_0$ is a nuisance ...
3
votes
1answer
607 views

A constant as an admissible estimator

This is a homework question so I would appreciate hints. I believe I have the first part correct, but I fail to see how the second part is different. Assume square error loss, $L(\theta ,a)=(\theta -...
4
votes
2answers
1k views

Estimating unconditional variance in time series

Consider a time series process with a well-defined, finite unconditional variance. Given a realization of the process (a time series) and a model for it, there are at least two ways of estimating the ...
2
votes
1answer
232 views

Conditional Expectation / Estimator Confusion

Let $X_1, X_2, X_3 \sim N(0, d^2)$ and $T = X_1^2 + X_2^2 + X_3^2.$ I have an estimator for $d$, $$\hat{d} = \frac{\sqrt{T\ 2\pi}}{4},$$ and another estimator for $d$, $$\tilde{d} = \frac{1}{3} \...
20
votes
2answers
22k views

Maximum Likelihood Estimators - Multivariate Gaussian

Context The Multivariate Gaussian appears frequently in Machine Learning and the following results are used in many ML books and courses without the derivations. Given data in form of a matrix $\...
32
votes
3answers
4k views

Is p-value a point estimate?

Since one can calculate confidence intervals for p-values and since the opposite of interval estimation is point estimation: Is p-value a point estimate?
6
votes
1answer
202 views

James-Stein Estimator with unequal variances (Ch. 2)

After studying James-Stein estimators for a few weeks and looking at many different sources I am stuck at trying to understand how Efron and Morris calculated the Toxoplasmosis example in their 1975 ...
7
votes
3answers
205 views

How to find maximum likelihood estimates of an integer parameter?

H.W. Question: $x_1,x_2,\ldots,x_n$ are independent Gaussian variables with mean $\mu$ and variance $\sigma^2$. Define $y = \sum_{n=1}^{N} x_n$ where $N$ is unknown. We are interested in ...
5
votes
1answer
534 views

Example of a consistent estimator that doesn't grow less variable with increased sample size?

I've had it asserted to me that any consistent estimator must necessarily also grow less variable with increased sample size. I felt that this couldn't be correct, since there was nothing in the ...
4
votes
1answer
188 views

Does MCD estimator suffers from swamping effect?

If there are multiple outliers in the data set, the Mahalanobis distance suffers from masking and swamping effects. In order to rectify this problem, robust estimation of location and scale, such as ...
4
votes
1answer
3k views

How to calculate the scale parameter of a Cauchy random variable

Let $(X_n)$ be iid random variables and suppose they have mean 0 and follow Cauchy distribution. I know I can set the location parameter to 0. My question is how to find the corresponding scale ...
4
votes
2answers
3k views

Asymptotic distribution of MLE (log-normal)

Say we have a sample $X_{1},...,X_{n}$ from a log-normal distribution with parameters $\mu$ and $\sigma^{2}$. That is, $\ln(X)$~$N(\mu,\sigma^{2})$. Let $T_{n},Z_{n}$ denote the MLE's for $\mathbb{E}(...
5
votes
0answers
1k views

What are the main different/alternative correlation estimators?

I'm looking to learn about the main/popular alternatives when it comes to estimating correlations that I've missed in the following list. The best answer will provide a reference (can be Wikipedia), a ...
5
votes
1answer
4k views

Cramér-Rao Lower Bound for Exponential Families

I am having a problem with applying the Cramér-Rao inequality to identify the lower bound for the variance of an unbiased estimator and hoped that you guys could help me. The problem is the following: ...
4
votes
1answer
511 views

Better estimator of expected sum than mean

I am trying to find the optimal estimator for the maximal expected $\Sigma X_i$ where $X_i$ is sampled from an unknown distribution which is chosen to be maximal. To clarify and simplify, there are ...
4
votes
1answer
363 views

How should I solve the following simultaneous equations?

I have the set of simultaneous equations below from the paper Parameter estimation for 3-parameter generalized pareto distribution by the principle of maximum entropy (POME) by VP Singh and H Guo: $$ ...
3
votes
0answers
165 views

Paired comparison of instruments using different measurement samples

I have instruments A, B, C and D - I'm in search of the best one. The problem: For illustrative purposes, let's use an example of evaluating the best among the instruments measuring difference in ...
3
votes
1answer
2k views

Methods of moments for t distribution

The parameters of a t distribution can be estimated via 1) ML or 2) method of moments If we use the method of moments we have: $\mu=E(R)$ $\sigma^2=V(R)=\frac{\beta \nu}{\nu -2}$ $\kappa = \frac{...
3
votes
1answer
217 views

minimax estimator

In the lecture here https://www.stat.berkeley.edu/~yuekai/201b/lec6.pdf we have "minimaxity does not imply admissibility: a minimax estimator has the best worst-case performance, but its ...
2
votes
0answers
36 views

How to get an estimator from this heuristic argument?

In the answer of this question, a practical solution was suggested for: Let $X_1, X_2, ..., X_k$ be a sequence of i.i.d. random variables with $X_i \sim \mathcal{U}\{1, 2, ..., n\}$ (discrete ...
2
votes
2answers
174 views

$\sqrt{n}$-consistency of M-estimator based on plug-in estimator

Note: This is a follow-up on a previous question that was concerned about consistency, but this time seeking $\sqrt{n}$-consistency. Suppose we estimate a quantity $\theta_0$ by the $\tilde{\theta} = ...
1
vote
0answers
25 views

parameter estimator

I have a probability density function $$\frac{4x\arctan(\frac{x}{a})}{a^2(\pi-2)} $$ $0≤x≤a\,,\quad$ where $a>0$. I need to find an estimator using some order statistic. Does maximum ...
0
votes
1answer
4k views

constant $\times$ distribution

I know that if $U\sim\chi^2(k)$ then $aU\sim \Gamma(k/2,2a)$ for $a>0$. But i read about the estimator and its distribution $$\hat{\sigma}_k^2=\frac{1}{2k}\sum_{i=1}^k (X_{2i}-X_{2i-1})^2=\frac{2\...
0
votes
1answer
2k views

Asymptotic normal distribution via the central limit theorem

I have a sample $n = 100$ with two "successes" (Two kids having a disease among 100). So we obviously have a binomial distribution. First I had to compute the maximum likelihood (ML) estimator $\hat{...
7
votes
3answers
697 views

Consistent unbiased estimator for the location parameter of $\mathcal{Cauchy} (\theta, 1)$

Given Cauchy distribution with pdf $p(x) = \frac{1}{\pi ((x - \theta)^2 + 1)}$ how can I find a consistent unbiased estimator for $\theta$? My reasoning so far Tried MLE, but there seems to be no ...
5
votes
1answer
228 views

Variance computed using Taylor series does not agree with numerical experiment

I would like to estimate an angle $\theta\in\left(-\frac{\pi}{2},\frac{\pi}{2}\right)$ given the noisy observations of its sine and cosine (this is related to my earlier question). My estimator is ...