All Questions
Tagged with estimation point-estimation
59 questions
1
vote
2
answers
77
views
Cramér-Rao bound when the samples come from two distributions
Is there a version of the Cramér-Rao bound when samples are independent but not identically distributed? More specifically, I am considering a sample set that is divided in two subsets, each subset ...
- 1,191
3
votes
1
answer
111
views
Cramér-Rao / Wolfowitz bound with nuisance parameter
Let $F$ be a distribution with two parameters, $\theta$ and $\phi$, whose values are non-random but unknown. Consider a sampling procedure in which $N$ samples $x_1, \ldots x_N$ are obtained from i.i....
- 1,191
4
votes
1
answer
146
views
Efficiency of chi-squared denoising
Suppose my measurement $\theta+\epsilon$ is corrupted by IID additive noise $\epsilon$ distributed as chi-squared with (known) $d$ degrees of freedom, what is the efficiency of pooling multiple ...
- 6,317
8
votes
4
answers
1k
views
Why the variance of Maximum Likelihood Estimator(MLE) will be less than Cramer-Rao Lower Bound(CRLB)?
Consider this example. Suppose we have three events to happen with probability $p_1=p_2=\frac{1}{2}\sin ^2\theta ,p_3=\cos ^2\theta $ respectively. And we suppose the true value $\theta _0=\frac{\pi}{...
- 187
2
votes
0
answers
46
views
An estimation method/algorithm for estimating the value of a specific parameter in a convex function
I am looking for an estimation/iteration process to estimate the value of a specific unobserved parameter of a convex function that fits the observed data of the other variables closely. Specifically, ...
- 21
-1
votes
1
answer
2k
views
MLE - CDF vs PDF as the likelihood-function?
Would maximum-likelihood estimation: with the cumulative-distribution function as the likelihood-function and the probability-density function as the likelihood-function, yield the same/equal ...
- 240
2
votes
1
answer
49
views
Forming a consistent estimator for the area under the regression line
I am trying to solve the following problem:
Take the following simple linear regression model, where $x_i \in \mathbb R$:
$y_i=\beta_0 + x_i \beta_1 + \epsilon_i$
Given that:
$\mathbb E[\epsilon_i]=...
- 145
0
votes
1
answer
287
views
Generalized Bayesian estimator (rule) of θ
Question:
Let $X_1, · · · , X_n$ be a random sample from $Poisson(θ)$. The prior for θ is $G(α, β)$
Find the Bayesian estimator (rule) of θ under the SEL(squared error loss).
Find the generalized ...
- 327
0
votes
1
answer
226
views
What Cramer-Rao bound should I use?
I have been researching about the Cramer-Rao bound and I have found two inequalities:
$$\text{Var}\left(\hat{\theta}\right)\geq\frac{1}{\text{E}\left[\left[\frac{\partial}{\partial\theta}\ln f(X;\...
- 3
0
votes
1
answer
4k
views
What's the advantage of a point estimate over an interval estimate?
A point estimate is :
A single numerical value that is used to estimate the corresponding population parameter.
Whereas an interval estimate is :
An estimate that consists of two numerical values ...
- 37
0
votes
0
answers
99
views
Taking Expectation Over Inverse Sum of Indicator Functions?
I'm working with a zero inflated Poisson distribution that has the following pmf:
$$f(y|w,\lambda)=wI[y=0]+(1-w)\frac{e^{-\lambda}\lambda^{y}}{y!}$$
I would like to find the expectation of the ...
- 174
2
votes
1
answer
41
views
Variance Estimator Change if we know Population Mean? (Normal dist. example)
For a normal distribution $N(\mu, \sigma^2)$ a commonly used unbiased and consistent estimator of variance is
$$\hat \sigma^2=\frac{\sum_ix_i^2 + n(\bar x)^2}{n-1}=\frac{\sum_i(x_i-\bar x)^2}{n-1}$$
...
- 174
0
votes
2
answers
138
views
Parameter estimation for random variables where a control parameter is another r.v
Let $\{X_i\}$ a sequence of independent random variables.
Each $X_i$ has a p.d.f $p(m, \theta)$. Where $\theta$ is a real unknown parameter and $m$ the outcome of another random variable $M$ with p.d....
- 63
1
vote
1
answer
2k
views
How to find point estimator for $\lambda$ in Poisson distribution?
Suppose we have a random sample $(X_1,X_2,...,X_n)$ from a Poisson distribution $Poi(\lambda)$.
How to find a point estimator of $\lambda$, and compute the mean and variance of the estimator.
...
- 141
2
votes
1
answer
273
views
How to interpret a sampling distribution from a Frequentist and Bayesian perspective
I've read multiple of the threads about Bayesian vs Frequentist interpretations of probability, but I'm having trouble trying to reconcile them with the idea of the sampling distribution when ...
- 330
8
votes
1
answer
953
views
The "correct" way to approximate $\text{var}(f(X))$ via Taylor expansion
tl;dr: There are two commonly reported formulas for approximating $\text{var}(f(X))$, but one is notably better than the other. Since it isn't the "standard" Taylor expansion, where does it come from, ...
- 722
1
vote
0
answers
2k
views
MVUE for Poisson Distribution
Let $X_1,...X_n$ be $\text{Poi}(\lambda)$ distributed random variables. I want to construct a minimal variance unbiased estimator (MVUE) for $\lambda$.
By the Neyman Lemma, I know that $T:=\sum_{i=1}...
- 279
1
vote
1
answer
258
views
nonexistence of a sufficient statistic
Let $X_1,X_2,\dots,X_n$ be a random sample from a $\Gamma(\theta,\theta)$ distribution. Then
$$
\prod_{i=1}^n f(x_i;\theta) = \frac{1}{\Gamma(\theta)^n\theta^n}(\prod_{i=1}^n x_i)^{\theta-1}e^{-\frac{...
- 231
11
votes
1
answer
5k
views
When can't Cramer-Rao lower bound be reached?
The Cramer-Rao lower bound (CRLB) gives the minimum variance of an unbiased estimator. One sentence in the wiki page says "However, in some cases, no unbiased technique exists which achieves the bound....
- 231
3
votes
0
answers
495
views
Signal-to-noise-ratio, Fisher information and and "estimability"
Given a parametric statistical model, is it common to study the quantity
$$ Q_{\theta} = \theta^2 I_{\theta} \, ,$$
where $I_{\theta}$ is the Fisher information? (I focus on a single parameter for ...
0
votes
0
answers
40
views
Estimating Standard Deviation with only Linear Calculations
I have an input list of numbers which are assumed to be drawn from an underlying Gaussian distribution. I need to find the mean and standard deviation of the dataset. The code that I am working with (...
- 356
1
vote
1
answer
285
views
$T_0$ is MVUE for $\gamma(\theta)$ and $T_1$ is an u.e. for $\gamma(\theta)$ with efficiency 0.0169. Then corr($T_0,T_1$ )?
The estimator $T_0$ is MVUE for $\gamma(\theta)$ and $T_1$ is any other unbiased estimator for $\gamma(\theta)$ with efficiency 0.0169. Then what is the correlation coefficient between $T_0$ and $T_1$...
- 11
0
votes
1
answer
164
views
check understanding on unbiased and consistent estimator
I'm trying to understand the expected value of an estimate. Here's my understanding.
The expected value of the estimate $\bar x$ of the parameter $\mu$ is what the mean of xbar tends to as we ...
- 141
4
votes
3
answers
5k
views
Deriving likelihood function of binomial distribution, confusion over exponents
This question focuses on a specific aspect of this one:
How to derive the likelihood function for binomial distribution for parameter estimation?
In my own derivation, I start with:
$$f(x\mid p) = ...
- 484
2
votes
0
answers
129
views
Estimating the number of tokens in a pile
We have a pile of tokens numbered from $1$ to $n$, where $n$ is unknown. $k$ tokens are drawn from the pile at random without replacement(i.e. the numbers that we get from tokens are unique). Say the ...
- 21
0
votes
0
answers
167
views
Simultaneous estimation of a group of linear model (regression) parameters
Suppose $y=ax+z$ where $x, y, z$ are random variables with range in $\mathbf R$, $\mathbf E[x]=\mathbf E[z|x]=0$ and $a$ is a constant. Note the distribution of $z$ conditioned on $x$ depends on $x$. ...
- 1,035
9
votes
2
answers
11k
views
Proof that posterior median is the Bayes estimate of absolute loss?
It is always argued that the posterior median is the Bayes estimate associated to the absolute loss function. The proofs I have come across rely on differentiating the conditional Bayesian risk and ...
- 93
4
votes
1
answer
1k
views
MVUE is unique - wrong proof?
Here is the proof of "MVUE is unique" that my lecturer gave:
Now I understand the following:
The first expansion is done using the formula for the sum of correlated random variables (https://en....
- 2,286
4
votes
1
answer
104
views
It what situation is a distribution known to be symmetric, but about an unknown location?
A favorite example in theoretical statistics is this:
A sample of individuals are drawn independently from a distribution with density $f(x)$, where $f(x)$ is unknown, but is known to be symmetric ...
- 543
5
votes
1
answer
8k
views
Efficiency, Precision, Accuracy, and Consistency
Can anyone please explain me the statistical terms efficiency, precision, accuracy, and consistency in plain language with easy example (hopefully by daily life example)?
So far, my understanding of ...
- 359
1
vote
0
answers
156
views
comparison of two estimators
Assume we have a data set $\mathbf{x}_{n} = (x_{1}, \dots, x_{n})$. Let $\delta_{1}(\mathbf{x}_{n})$ and $\delta_{2}(\mathbf{x}_{n})$ be two consistent estimators of some parameter $\theta \in R^{k}$. ...
- 163
-1
votes
1
answer
267
views
Find the Method of moments estimate
So here i am getting E(X)=1/2,E(X^2)=0 and E(X^3)=Theta squared/4
How do i proceed now?How do i use the given x values ?
- 39
1
vote
1
answer
128
views
What is the maximum likelihood estimator of the given distribution?
Let $X_{1},...,X_{n}$ be independent random variables with $X_{k}$ having the normal distribution with mean $k\theta$ and variance $\sigma^2$. Find the maximum likelihood estimator of $\theta$.
My ...
- 1,048
0
votes
1
answer
405
views
Maximum likelihood estimator of $\theta$ of the following probability distribution
Let $x_{1}=-2,x_{2}=1,x_{3}=3,x_{4}=-4$ be observed values from the following density function:
$f(x|\theta)=\frac{e^{-x}}{e^{\theta}+e^{-\theta}}$ where the support is $-\theta \leq x\leq\theta$.
...
- 1,048
4
votes
1
answer
310
views
Does nonparametric bootstrapping work on correlated data?
I have $m$ weakly stationary observations $X_1,X_2,\cdots,X_m$ from a Markov chain. I want to estimate the variance of the mean. My idea was to use nonparametric bootstrapping to make $n$ bootstrap ...
- 831
2
votes
2
answers
1k
views
MLE estimator of $P(X \leq c)$ for $X~ normal(\theta,1)$
I need to find MLE estimator of $P(X \leq c)$ where X is $Normal(\theta,1)$, c is fixed. Note: $X_1,...,X_n$ are a random sample drawn from $Normal(\theta,1)$
Any hint on how to approach this ...
- 121
6
votes
1
answer
2k
views
What is the difference between complete statistics and complete family of distributions?
I fail to understand when we call a family of distribution is complete and when a statistic is complete. What is the difference between both?, Is there a relation between them? Please provide examples ...
- 560
1
vote
1
answer
38
views
Choosing one parameterization over another
Suppose I have a random response Y and a fixed predictor X. My model is
\begin{align*}
Y &= \frac{aX}{b + aX}
\end{align*}
where $X, Y, a, b > 0$. I have a sample $Y_1, ..., Y_n$ and n is ...
- 1,029
0
votes
1
answer
41
views
Obtaining group-level estimate of quantity, for e.g. correlation
say I have multiple estimates of correlation between source A and source B, for multiple subjects. the number of estimates vary between subjects. for example, subject 1 might have 10 estimates of ...
- 53
3
votes
1
answer
169
views
Bayesian estimator and prediction
For a Bayesian, if he/she can make predictions using the entire posterior, why bother to calculate a Bayes estimate like the posterior mean or MAP?
Thanks!
- 443
7
votes
2
answers
2k
views
Weighted arithmetic mean weight choice in a simplified Bayes estimator
A Bayesian estimator as defined in the Wikipedia article
Practical example of Bayes estimators balances the prior knowledge of the entire data set with the knowledge of the subset. This is usually ...
- 1,259
1
vote
1
answer
1k
views
Practical situation in which the posterior mean is prefered to the MAP
Sometimes experts for which we design models are interested in having a point estimate and in practical situations, they always say me "give us the most probable parameter value". And whether the ...
- 5,093
4
votes
1
answer
274
views
Estimating accurately the mean of an autocorrelated bounded integer time series
I have a bounded integer time series $X_{1:\infty}$ ($1\leq X_k\leq M$), and I want to estimate the mean
$$ s = \lim_{L\to\infty} \frac{1}{L}\sum_{k=1}^L X_k. $$
I'm assuming it exists and that $X_k$ ...
- 557
3
votes
1
answer
2k
views
How to get the maximum likelihood estimator of $U(\theta,\theta +1)$?
I know how to find the MLE for $U(0,\theta)$ but I am in trouble with this one.
let $X_1,\dots,X_n$ be a random sample from $U(\theta,\theta +1)$. Consider the following three estimators for $\theta,\...
- 143
3
votes
1
answer
813
views
Efficient estimators and CRLB
An estimator is efficient if it reaches the Cramér-Rao Lower Bound and since it is efficient, it is also the UMVU estimator of the parametric function $\tau(\theta)$. But Cramér-Rao inequality and the ...
- 3,209
34
votes
3
answers
7k
views
Is p-value a point estimate?
Since one can calculate confidence intervals for p-values and since the opposite of interval estimation is point estimation: Is p-value a point estimate?
- 1,350
40
votes
5
answers
191k
views
How to derive the likelihood function for binomial distribution for parameter estimation?
According to Miller and Freund's Probability and Statistics for Engineers, 8ed (pp.217-218), the likelihood function to be maximised for binomial distribution (Bernoulli trials) is given as
$L(p) = \...
- 1,092
2
votes
0
answers
171
views
Using Horvitz-Thompson for estimation from a simple random sample with unknown membership probability
I learnt about the Horvitz-Thompson estimator yesterday, and am trying to apply it to the degenerate case where $p$ is uniform for each, but I seem to have run into a bit of a confusing situation.
...
- 83
6
votes
1
answer
1k
views
Estimating sample mean from a biased sample (whose generative process is known)
I'm working on a problem where I'm trying to estimate some property of a dataset from a small non-uniform sample. (Let's just take the population mean because it's simple.)
Formally, assume I have ...
- 83
5
votes
1
answer
7k
views
Cramer-Rao Lower Bound
Let $X_1,..,X_n$ be an iid sample of $N(0,\sigma^2)$. Find an unbiased
estimator of $\sigma^2$ and its lower bound.
I found that $$\hat{\sigma}^2 = \sum_{i=1}^{n} X_i^2$$ is an unbiased estimator ...
user72621