Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

"A rule, method, or criterion for arriving at an estimate of the value of a parameter."

3
votes
0answers
73 views

Confusion in terminologies for simple linear regression model

Please go through my draft summary below and let me know if my conventions are correct, comprehensible, and non ambiguous. Simple Linear Regression Model Let given observed sample set be $\{(x_1,...
0
votes
0answers
21 views

Connections between pseudoinverse, linear regression, BLUE, ordinary least square

They all take similar forms, why is that, what are the connections here? pseudoinverse: $Ax=b, x=(A^TA)^{-1}A^Tb$ linear regression: $ \hat{y}=x^T(X^TV^{-1}X)^{-1}X^TV^{-1}y$, where X is the data, y ...
3
votes
0answers
39 views

Posterior mean estimator with MCMC (Metropolis Hastings Algorithm) - Concrete example

I have a little project for which I have to estimate parameters on a PSF (Point Spread Function = response of the system to a dirac, i.e a star in my case). I have the 6 parameters to estimate : $p=(\...
0
votes
1answer
37 views

Why do we divide by the degree of freedom?

This might be trivial and vague question, but I still don't understand why when creating test statistics or estimators we always divide by the degree of freedom. Just to give examples of what I'm ...
-1
votes
0answers
11 views

Variance proof for the stochastic estimator for trace of a matrix

I'm reviewing the proof the estimator of the trace of a matrix and am having trouble reconciling a jump in the proof of the variance of the estimator. The paper with the proof is found here. The ...
0
votes
0answers
19 views

Variance estimator for sum of random variables [duplicate]

I want to compute the variance of $X$, which is the sum of 4 random variables $A, B, C, D$, i.e. $X=A+B+C+D$ For performance reasons (the overall context is that I use convolutional layers in PyTorch)...
2
votes
1answer
44 views

Why is it important that estimators are unbiased and consistent?

I am clear on the definition of unbiasedness and consistency. But why are these the criteria we use to judge whether an estimator is a good one? There are other criteria, of course, like the variance ...
2
votes
1answer
54 views

Admissibility does not imply minimax

The answer to minimax estimator explains why minimax does not imply admissibility. The relevant statement is from https://www.stat.berkeley.edu/~yuekai/201b/lec6.pdf which says, minimaxity does not ...
0
votes
0answers
11 views

Existence of Minimum Risk equivariant estimator

$X\sim f(x-\theta)$ . Let $y_i=x_i-x_n$ for $i =1,\cdots,n-1$. Let loss $L(\theta,t)=\rho(t-\theta)$ . Let there exist an equivariant estimator $T_0$ with finite risk. If $\rho$ is convex and ...
7
votes
1answer
162 views

Efficient Estimator from Insufficient Statistic

Suppose that I have a statistic $T(X)$, and I know for sure that it is not sufficient to estimate a parameter $\theta$. Is it still possible to have an estimator $\hat\theta(T(X))$ that is efficient (...
1
vote
1answer
29 views

Heteroskedasticity-consistent standard errors

See https://en.wikipedia.org/wiki/Heteroscedasticity-consistent_standard_errors. Assume the model of interest is the linear regression model. If the errors are heteroskedastic, $\hat{\sigma}^2_i = \...
1
vote
1answer
48 views

Dominating Positive Part James-Stein

Is dominating Positive Part James Stein estimator when estimating the mean of a multivariate normal of dimension 3 with known variance(all equal) an open problem? If not, what is this estimator ...
0
votes
0answers
15 views

Understanding test statisic $x$ for a sample from a continuous pdf [duplicate]

Can someone explain to me what using $x$ as a test statistic indicates, based on a sample? For example, say $x$ is my test statistic (not necessarily a good one, just trying to understand the concept)...
0
votes
0answers
37 views

Interpretation of MINE Mutual Estimator function evaluated on individual samples

This paper proposes an estimator for MI over two channels with finite samples. The estimator (eq. 10 in the paper) uses an expression obtained from a parametric NN evaluated over a mixture of joint ...
5
votes
3answers
188 views

Consistent unbiased estimator for the location parameter of Cauchy (theta, 1)

Given Cauchy distribution with pdf $p(x) = \frac{1}{\pi ((x - \theta)^2 + 1)}$ how can I find a consistent unbiased estimator for $\theta$? My reasoning so far Tried MLE, but there seems to be no ...
0
votes
1answer
50 views

Dependence of estimator covariance on sample count

Say that $X$ is a set $\{X_1, X_2, ..., X_N\}$ of (non-independent) random variables, and that $\hat{\mu}$ is a set $\{\hat{\mu}_1, \hat{\mu}_2, ..., \hat{\mu}_N\}$ of estimators. Each $\hat{\mu}_i$ ...
0
votes
0answers
25 views

Bayes risk with explicit wegihted average for $\mu$

I have the following problem to solve: $Y_i\overset{i.i.d.}{\sim} N(\mu,1) \text{ for } i = 1,\dots,n$. Let $\hat{\mu}(Y_{1:n})$ denote an estimator of $\mu$. The loss is quadratic and the ...
0
votes
0answers
22 views

Do all robust estimators involve ranking?

Are there any robust estimators that do not include ranking as part of their method?
4
votes
2answers
81 views

Show that if $E\psi(x-\theta)= 0 $ then $P(X< \theta) \leq p \leq P(X \leq \theta)$

Define $$\psi(x)=\begin{cases} 1-p & x < 0 \\ 0 & x=0 \\ -p & x> 0 \end{cases}$$. I have to show that if $$E\psi(x-\theta)= 0 $$ then $$P(X< \theta) \leq p \leq P(X \leq \theta)$$...
1
vote
1answer
70 views

The definitions of estimator and estimate

This example demonstrates the difference between a theoretical observation and a realized observation. A theoretical observation is a random variable with a probability distribution, while its ...
1
vote
2answers
82 views

Bayes Estimator

Wikipedia has a section on the Bayes Estimator. https://en.wikipedia.org/wiki/Bayes_estimator Isn't Bayes Estimator simply the value of the parameter that minimizes the expected loss of a loss ...
0
votes
0answers
13 views

How to estimate contribution of noise sources

We're trying to estimate the contribution of a device on a performance indicator on the quality of transmission of some signal. The value for performance indicator is assumed to be normally ...
1
vote
0answers
20 views

Estimator for repeated sampling and fitting

Say I have a Normal distribution $\mathcal{N_1}(\mu_1,\sigma_1)$. Now I will sample $N$ samples $X_1$ from this distribution, and use estimators for $\hat\mu_2$ and $\hat\sigma_2$ to fit a new ...
0
votes
0answers
15 views

Robust estimator of a geometric average?

Is the geometric average of a sample the best robust estimator of the geometric average of a population, or is it something else like the median or trimmed mean or even the mode? Would it be correct ...
1
vote
1answer
51 views

Trouble understanding some r code

I have this code here (it's out of a book called "An Introduction to Bootstrap Methods with Applications to R") We are working with the estimator: $S_n^2=\frac{\sum_{i=1}^{n}(X_i-X_b)^2}{n}$ where $...
4
votes
3answers
83 views

What is the difference between a true estimation and an estimator?

In a machine learning class, the instructor was explaining Maximum Likelihood Estimation. He mentioned that $\hat{\theta}_{MLE}=argmax_\theta P(D;\theta)$ Where D is the data observed, $\theta$ is ...
1
vote
1answer
22 views

Validating new estimators

Can anyone please tell me whether there it be a problem if I validate an estimator by taking samples from one population. This is how most of the estimators for respondent driven sampling has been ...
1
vote
0answers
16 views

Unbiased estimate of sign of mean

Consider the set $\mathcal{P}$ of probability distributions that have a finite first moment and define the function $\operatorname{sgn} :\mathcal{P} \to \mathbb{R}$ as $$ \operatorname{sgn}(\mu) = \...
0
votes
0answers
34 views

What's the best estimator for expectation if we can draw samples iid *and* we know the likelihood of each sample we receive

Suppose that $X_1, X_2, \ldots X_n$ is a sequence of random variables on a set $S$, drawn independently according to the pdf $p : S \to [0,1]$. Part I: Given some $f : S \to \mathbb{R}$, I want to ...
2
votes
1answer
66 views

Maximum likelihood estimator for $\theta$

5. Find the MLE for $\theta$ based on a random sample of size $n$ from a distribution wit pdf $$f(x; \, \theta) = \begin{cases} 2\theta^{2} x^{-3} & \theta \leqslant x \\ 0 & x < \...
0
votes
0answers
18 views

Approximate distribution of the asymptotically normal estimator

By definition, point estimator $\hat\theta_n(\mathbf{X})$ is asymptotically normal if $$\sqrt{n}(\hat\theta_n - \theta) \, \overset{d}{\longrightarrow} \, \mathcal{N}(0, \sigma^2(\theta)), \qquad \...
0
votes
0answers
21 views

Estimation in case of data dependent noise

I am trying to estimate $a$ and $b$ in the below linear model $$y = ax + b + \epsilon$$ where $x \in R^n$ and $y \in R^n$ are given, and $\epsilon$ depends on the parameters and the $x$. Also, it is ...
0
votes
1answer
15 views

Estimators of population mean:first observation

I'm wondering if I have a random sample $y_0,y_1,...y_n$ drawn from $N(\mu, \sigma^2) $, and use $y_0$ as an estimator for the population mean, what would be the expectation and the variance of such ...
0
votes
0answers
18 views

Notation of Random Variables, samples and estimators

It seems common in the estimators subject to use the following notation: example, (wikipedia estimators) $X$, as a random variable. $x$, as a possible value or single draw from the variable X. $\...
1
vote
0answers
22 views

Estimator of (left-)censored normal distribution when mean>>std

Suppose there is a left-censored normal distribution, and we know there is a total of $m$ samples, for which we know $n$ of them. I am trying to estimate the mean and variance of the underlying normal ...
4
votes
1answer
47 views

What does this paper mean by mean square error?

I'm reading a paper about estimating fetal weight by ultrasound and other techniques. In Table 2 they give descriptive statistics, to wit: of all the examiners, the one with the highest mean square ...
5
votes
2answers
237 views

Notation in statistics (parameter/estimator/estimate)

In statistics, it is very important to differentiate between the following three concepts which are often confused and mixed by students. Usually, books denote by $\theta$ an unknown parameter. Then ...
0
votes
0answers
23 views

How to calculate the variance of the leave-one-out cross validation estimator and why is it high?

I read from Elements of Statistical Learning that the leave-one-out cross validation estimator has high variance, and I read the related stackexchange posts as to why this is the case 1. But I'm ...
0
votes
0answers
47 views

Difference-in-Difference Estimator: valid research design

I just saw a study that applied the following difference-in-difference estimator: $Y_{it} = \alpha + \alpha_1 T_{it} + \alpha_2 Post_{it} + \alpha_3 Post_{it} \times T_{it} + u_{it},$ where $i$ ...
5
votes
2answers
129 views

$\sqrt{n}$-equivalence of M-estimator based on plug-in estimator

Suppose our model has a nuisance parameter $\eta_0$ of which we possess a consistent estimator $\hat{\eta}_0$. We obtain an estimator $\hat{\theta}$ of a parameter of interests $\theta$ by finding ...
2
votes
2answers
90 views

$\sqrt{n}$-consistency of M-estimator based on plug-in estimator

Note: This is a follow-up on a previous question that was concerned about consistency, but this time seeking $\sqrt{n}$-consistency. Suppose we estimate a quantity $\theta_0$ by the $\tilde{\theta} = ...
9
votes
2answers
296 views

Improving the minimum estimator

Suppose that I have $n$ positive parameters to estimate $\mu_1,\mu_2,...,\mu_n$ and their corresponding $n$ unbiased estimates produced by the estimators $\hat{\mu_1},\hat{\mu_2},...,\hat{\mu_n}$, i.e....
6
votes
1answer
107 views

Consistency of M-estimator based on plug-in estimator?

Suppose we estimate a quantity $\theta_0$ by the $\tilde{\theta} = \hat{\theta}(\eta)$ that solves the estimating equation $$S_n(\tilde{\theta}, \eta_0) = 0$$ where $\eta_0$ is a nuisance ...
5
votes
1answer
93 views

James-Stein Estimator with unequal variances (Ch. 2)

After studying James-Stein estimators for a few weeks and looking at many different sources I am stuck at trying to understand how Efron and Morris calculated the Toxoplasmosis example in their 1975 ...
3
votes
1answer
35 views

Consistency and rates of convergence

Suppose that I have two statistics that are known to be consistent , e.g : $ S_{n} ^2 $ (biased sample variance about sample mean) and $ S_{n-1}^2$ (bessel-corrected sample variance, that is unbiased)....
0
votes
0answers
39 views

Maximum Likelihood Estimation (MLE) for Markov Chain Rates $R_{ij}$ a.k.a. $Q_{ij}$

The following past question deals with using MLE to estimate the transition probabilities matrix $P_{ij}$ directly. I, however, am looking to estimate the rates matrix $R_{ij}$ a.k.a. $Q_{ij}$, which ...
2
votes
0answers
24 views

Show that there is no efficient estimator for the variance of a normal distribution using properties of the exponential family

I want to prove the statement in the title using the following statement from Wikipedia: it was proved that efficient estimation is possible only in an exponential family, and only for the natural ...
3
votes
0answers
42 views

Can an asymptotically efficient estimator be biased?

In "Theory of point estimation" by Lehmann and Casella (1998) there is the following definition: It is also said that So terms of the asymptotically normal sequence of estimators can be ...
0
votes
0answers
18 views

Why does Agresti define the natural exponential family this way?

I am reading through Agresti's Categorical Data Analysis. And in section 4.1.1 he defines the natural exponential family as: $f(y_i;\theta_i) = a(\theta_i)b(y_i)*exp(y_iQ(\theta_i))$ Why is this ...
3
votes
0answers
38 views

Find unbiased estimators for $\lambda$ and $\lambda^2$.

For the spatial homogeneous Poisson process, find unbiased estimators for $\lambda$ and $\lambda^2$. Attempt: Since the homogeneous Poisson process is over an area, how i would i go about ...