Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [estimators]

"A rule, method, or criterion for arriving at an estimate of the value of a parameter."

1
vote
2answers
32 views

Maximum Likelihood Estimator (MLE) for $2 \theta^2 x^{-3}$

I'm having a bit of trouble solving this. $$ f(x_i; \theta) = 2 \theta^2 x_i^{-3}, 0 \le \theta \le x_i \lt \infty $$ I start by finding $f(\textbf{x}; \theta)$: $$ f(\textbf{x}; \theta) = \prod{f(...
2
votes
2answers
42 views

In linear regression, are the noise terms independent of the coefficient estimators?

In the Wikipedia article on the bias-variance tradeoff, the independence of the estimator $\hat f(x)$ and the noise term $\epsilon$ is used in a crucial way in the proof of the decomposition of the ...
1
vote
0answers
33 views

Is this the only way to determine if a parameter can be estimated efficiently?

I am tasked with determining if a particular parameter can be estimated efficiently. Given that an efficient estimator is an unbiased estimator which achieves the Cramer-Rao lower-bound, is the only ...
0
votes
0answers
20 views

Can't find much online about “Linear Regression Estimators” — Looking for help making sense of notes on the topic

I have recently been lectured on how to implement linear regression estimators for a project I have going - I was walked through it works but I couldn't make sense of what was going on. See below for ...
0
votes
0answers
43 views

How to prove variance of OLS estimator in matrix form?

I am reading Wooldridge's Introductory Econometrics (2000), don't judge me, old version = cheap second hand book, and in the page P94 Theorem 3.2 of Multiple Regression Analysis, it says that: $$ Var(...
2
votes
1answer
14 views

Covariance of An Empirical Distribution Function Evaluated at Different Points

The problem is extracted from All of Statistics (Exercise 7.5), Larry Wasserman. I don't have a solution manual to the book so I post here the problem together with my attempted answer: Let $x$ and $...
0
votes
1answer
26 views

Is a Kalman filter ever the optimal way to estimate a dynamic value given a full history of measurements?

I'm trying to get some intuition for Kalman filtering, and I conceived this toy example: Say that I have a sensor that tracks a moving 1-dimensional target. Say that the measurements from the sensor ...
1
vote
0answers
49 views

Finding Chernoff bounds maximum estimators

I am currently trying to resolve the following exercise about Chernoff bounds: Let $X_{1}, X_{2}, \dots, X_{n}$ be independent, identically distributed (i.i.d) random variables with distribution $N(0,...
1
vote
0answers
19 views

Distribution of coefficient on the error correction term in ECM and VECM

According to statistic academic literature, the cointegration test on coefficient $\alpha$ of the error term included in ECM or VECM does not follow a standard distribution. My question is: If so, ...
0
votes
0answers
30 views

consistency of an estimator not based on total sample size

How do I show the consistency of an estimator of a parameter, say $\mu$, that is not based on the sample size $n$ but a function of $n_{i}$'s where $\sum_{i=1}^{K}n_{i}=n$ ? Consider for example the ...
1
vote
1answer
36 views

Estimator of ratio of central moments

In the context of Control Variates one has to estimate, for example, the following ratios of central moments: $$\frac{\mu_{1,1}}{\mu_{0,2}} \quad \text{and} \quad \frac{\mu_{1,1}^2}{\mu_{0,2}}$$ ...
3
votes
0answers
109 views

Confusion in terminologies for simple linear regression model [closed]

Please go through my draft summary below and let me know if my conventions are correct, comprehensible, and non ambiguous. Simple Linear Regression Model Let given observed sample set be $\{(x_1,...
0
votes
0answers
34 views

Connections between pseudoinverse, linear regression, BLUE, ordinary least square

They all take similar forms, why is that, what are the connections here? pseudoinverse: $Ax=b, x=(A^TA)^{-1}A^Tb$ linear regression: $ \hat{y}=x^T(X^TV^{-1}X)^{-1}X^TV^{-1}y$, where X is the data, y ...
3
votes
0answers
56 views

Posterior mean estimator with MCMC (Metropolis Hastings Algorithm) - Concrete example

I have a little project for which I have to estimate parameters on a PSF (Point Spread Function = response of the system to a dirac, i.e a star in my case). I have the 6 parameters to estimate : $p=(\...
0
votes
1answer
61 views

Why do we divide by the degree of freedom?

This might be trivial and vague question, but I still don't understand why when creating test statistics or estimators we always divide by the degree of freedom. Just to give examples of what I'm ...
3
votes
1answer
60 views

Why is it important that estimators are unbiased and consistent?

I am clear on the definition of unbiasedness and consistency. But why are these the criteria we use to judge whether an estimator is a good one? There are other criteria, of course, like the variance ...
2
votes
1answer
63 views

Admissibility does not imply minimax

The answer to minimax estimator explains why minimax does not imply admissibility. The relevant statement is from https://www.stat.berkeley.edu/~yuekai/201b/lec6.pdf which says, minimaxity does not ...
0
votes
0answers
13 views

Existence of Minimum Risk equivariant estimator

$X\sim f(x-\theta)$ . Let $y_i=x_i-x_n$ for $i =1,\cdots,n-1$. Let loss $L(\theta,t)=\rho(t-\theta)$ . Let there exist an equivariant estimator $T_0$ with finite risk. If $\rho$ is convex and ...
7
votes
1answer
167 views

Efficient Estimator from Insufficient Statistic

Suppose that I have a statistic $T(X)$, and I know for sure that it is not sufficient to estimate a parameter $\theta$. Is it still possible to have an estimator $\hat\theta(T(X))$ that is efficient (...
1
vote
1answer
42 views

Heteroskedasticity-consistent standard errors

See https://en.wikipedia.org/wiki/Heteroscedasticity-consistent_standard_errors. Assume the model of interest is the linear regression model. If the errors are heteroskedastic, $\hat{\sigma}^2_i = \...
1
vote
1answer
49 views

Dominating Positive Part James-Stein

Is dominating Positive Part James Stein estimator when estimating the mean of a multivariate normal of dimension 3 with known variance(all equal) an open problem? If not, what is this estimator ...
0
votes
0answers
39 views

Interpretation of MINE Mutual Estimator function evaluated on individual samples

This paper proposes an estimator for MI over two channels with finite samples. The estimator (eq. 10 in the paper) uses an expression obtained from a parametric NN evaluated over a mixture of joint ...
6
votes
3answers
227 views

Consistent unbiased estimator for the location parameter of Cauchy (theta, 1)

Given Cauchy distribution with pdf $p(x) = \frac{1}{\pi ((x - \theta)^2 + 1)}$ how can I find a consistent unbiased estimator for $\theta$? My reasoning so far Tried MLE, but there seems to be no ...
0
votes
1answer
51 views

Dependence of estimator covariance on sample count

Say that $X$ is a set $\{X_1, X_2, ..., X_N\}$ of (non-independent) random variables, and that $\hat{\mu}$ is a set $\{\hat{\mu}_1, \hat{\mu}_2, ..., \hat{\mu}_N\}$ of estimators. Each $\hat{\mu}_i$ ...
0
votes
0answers
22 views

Do all robust estimators involve ranking?

Are there any robust estimators that do not include ranking as part of their method?
4
votes
2answers
83 views

Show that if $E\psi(x-\theta)= 0 $ then $P(X< \theta) \leq p \leq P(X \leq \theta)$

Define $$\psi(x)=\begin{cases} 1-p & x < 0 \\ 0 & x=0 \\ -p & x> 0 \end{cases}$$. I have to show that if $$E\psi(x-\theta)= 0 $$ then $$P(X< \theta) \leq p \leq P(X \leq \theta)$$...
1
vote
1answer
80 views

The definitions of estimator and estimate

This example demonstrates the difference between a theoretical observation and a realized observation. A theoretical observation is a random variable with a probability distribution, while its ...
1
vote
2answers
94 views

Bayes Estimator

Wikipedia has a section on the Bayes Estimator. https://en.wikipedia.org/wiki/Bayes_estimator Isn't Bayes Estimator simply the value of the parameter that minimizes the expected loss of a loss ...
0
votes
0answers
13 views

How to estimate contribution of noise sources

We're trying to estimate the contribution of a device on a performance indicator on the quality of transmission of some signal. The value for performance indicator is assumed to be normally ...
1
vote
0answers
23 views

Estimator for repeated sampling and fitting

Say I have a Normal distribution $\mathcal{N_1}(\mu_1,\sigma_1)$. Now I will sample $N$ samples $X_1$ from this distribution, and use estimators for $\hat\mu_2$ and $\hat\sigma_2$ to fit a new ...
0
votes
0answers
22 views

Robust estimator of a geometric average?

Is the geometric average of a sample the best robust estimator of the geometric average of a population, or is it something else like the median or trimmed mean or even the mode? Would it be correct ...
1
vote
1answer
52 views

Trouble understanding some r code

I have this code here (it's out of a book called "An Introduction to Bootstrap Methods with Applications to R") We are working with the estimator: $S_n^2=\frac{\sum_{i=1}^{n}(X_i-X_b)^2}{n}$ where $...
4
votes
3answers
86 views

What is the difference between a true estimation and an estimator?

In a machine learning class, the instructor was explaining Maximum Likelihood Estimation. He mentioned that $\hat{\theta}_{MLE}=argmax_\theta P(D;\theta)$ Where D is the data observed, $\theta$ is ...
1
vote
1answer
22 views

Validating new estimators

Can anyone please tell me whether there it be a problem if I validate an estimator by taking samples from one population. This is how most of the estimators for respondent driven sampling has been ...
1
vote
0answers
18 views

Unbiased estimate of sign of mean

Consider the set $\mathcal{P}$ of probability distributions that have a finite first moment and define the function $\operatorname{sgn} :\mathcal{P} \to \mathbb{R}$ as $$ \operatorname{sgn}(\mu) = \...
1
vote
0answers
39 views

What's the best estimator for expectation if we can draw samples iid *and* we know the likelihood of each sample we receive

Suppose that $X_1, X_2, \ldots X_n$ is a sequence of random variables on a set $S$, drawn independently according to the pdf $p : S \to [0,1]$. Part I: Given some $f : S \to \mathbb{R}$, I want to ...
2
votes
1answer
67 views

Maximum likelihood estimator for $\theta$

5. Find the MLE for $\theta$ based on a random sample of size $n$ from a distribution wit pdf $$f(x; \, \theta) = \begin{cases} 2\theta^{2} x^{-3} & \theta \leqslant x \\ 0 & x < \...
0
votes
0answers
21 views

Approximate distribution of the asymptotically normal estimator

By definition, point estimator $\hat\theta_n(\mathbf{X})$ is asymptotically normal if $$\sqrt{n}(\hat\theta_n - \theta) \, \overset{d}{\longrightarrow} \, \mathcal{N}(0, \sigma^2(\theta)), \qquad \...
0
votes
0answers
21 views

Estimation in case of data dependent noise

I am trying to estimate $a$ and $b$ in the below linear model $$y = ax + b + \epsilon$$ where $x \in R^n$ and $y \in R^n$ are given, and $\epsilon$ depends on the parameters and the $x$. Also, it is ...
0
votes
1answer
15 views

Estimators of population mean:first observation

I'm wondering if I have a random sample $y_0,y_1,...y_n$ drawn from $N(\mu, \sigma^2) $, and use $y_0$ as an estimator for the population mean, what would be the expectation and the variance of such ...
0
votes
0answers
19 views

Notation of Random Variables, samples and estimators

It seems common in the estimators subject to use the following notation: example, (wikipedia estimators) $X$, as a random variable. $x$, as a possible value or single draw from the variable X. $\...
1
vote
0answers
22 views

Estimator of (left-)censored normal distribution when mean>>std

Suppose there is a left-censored normal distribution, and we know there is a total of $m$ samples, for which we know $n$ of them. I am trying to estimate the mean and variance of the underlying normal ...
4
votes
1answer
48 views

What does this paper mean by mean square error?

I'm reading a paper about estimating fetal weight by ultrasound and other techniques. In Table 2 they give descriptive statistics, to wit: of all the examiners, the one with the highest mean square ...
5
votes
2answers
251 views

Notation in statistics (parameter/estimator/estimate)

In statistics, it is very important to differentiate between the following three concepts which are often confused and mixed by students. Usually, books denote by $\theta$ an unknown parameter. Then ...
0
votes
0answers
42 views

How to calculate the variance of the leave-one-out cross validation estimator and why is it high?

I read from Elements of Statistical Learning that the leave-one-out cross validation estimator has high variance, and I read the related stackexchange posts as to why this is the case 1. But I'm ...
0
votes
0answers
49 views

Difference-in-Difference Estimator: valid research design

I just saw a study that applied the following difference-in-difference estimator: $Y_{it} = \alpha + \alpha_1 T_{it} + \alpha_2 Post_{it} + \alpha_3 Post_{it} \times T_{it} + u_{it},$ where $i$ ...
5
votes
2answers
131 views

$\sqrt{n}$-equivalence of M-estimator based on plug-in estimator

Suppose our model has a nuisance parameter $\eta_0$ of which we possess a consistent estimator $\hat{\eta}_0$. We obtain an estimator $\hat{\theta}$ of a parameter of interests $\theta$ by finding ...
2
votes
2answers
98 views

$\sqrt{n}$-consistency of M-estimator based on plug-in estimator

Note: This is a follow-up on a previous question that was concerned about consistency, but this time seeking $\sqrt{n}$-consistency. Suppose we estimate a quantity $\theta_0$ by the $\tilde{\theta} = ...
9
votes
2answers
298 views

Improving the minimum estimator

Suppose that I have $n$ positive parameters to estimate $\mu_1,\mu_2,...,\mu_n$ and their corresponding $n$ unbiased estimates produced by the estimators $\hat{\mu_1},\hat{\mu_2},...,\hat{\mu_n}$, i.e....
6
votes
1answer
115 views

Consistency of M-estimator based on plug-in estimator?

Suppose we estimate a quantity $\theta_0$ by the $\tilde{\theta} = \hat{\theta}(\eta)$ that solves the estimating equation $$S_n(\tilde{\theta}, \eta_0) = 0$$ where $\eta_0$ is a nuisance ...