Questions tagged [estimators]

A rule for calculating an estimate of a given quantity based on observed data [Wikipedia].

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Estimating moments of stationary time series — degrees of freedom correction?

I have read that, for a weakly stationary time series we use the empirical moments $$ \hat{\mu} = \frac{1}{T}\sum_{t=1}^T y_t = \bar{y}, \\ \hat{\gamma_0} = \frac{1}{T} \sum_{t=1}^T (y_t-\bar{y})^2, \\...
-1
votes
0answers
9 views

decomposing and estimating multiplicative time series

I'm looking through some time series books and looking at different time of models with seasonality we have $X_t = m_t + S_t + Y_t$ or $X_t = m_t*S_t + Y_t$ where $m_t$ is a systematic trend ...
4
votes
2answers
41 views

Any theoretical basis for estimating parameter using $P(\theta | D)$ instead of MLE?

To my understanding, when trying to estimate the value of a parameter $\theta$ of a model (e.g. $mu$ of a Normal distribution) given some data $D$ , we can find the MLE which is $\hat{\theta} = ...
0
votes
0answers
7 views

Statistical estimators for limited dataset

You can ignore this context but I think it adds a little interest to the question.. In finance pricing information is often proprietary and firms do not want other firms to know their price, but of ...
1
vote
0answers
9 views

What is an intuitive of definition of “point identification” (point identified parameter) in econometrics?

I've recently come across the notion of point identification in several econometric papers. See, e.g., https://scholar.harvard.edu/files/tamer/files/pie.pdf, who mentions point identification ...
0
votes
0answers
14 views

Finding the mean and variance of an estimator

I am trying to find the mean and the variance of a plug-in sample mean estimator of the coefficient of difficulty. In this case, the plug-in sample mean estimator is denoted as $\hat{d}(a,b):=n^{-1}...
1
vote
1answer
13 views

Some thoughts on finite sample properties of an estimator

I derived the mean squared errors (MSE) of a consistent estimator for two models: restricted and unrestricted. In addition, I showed that both has the same rate of convergence (that is, the asymptotic ...
0
votes
0answers
6 views

How to estimate the confidence interval for a “predicted difference” from a quadratic model?

Assume you have past consumption levels $c_1, \dots c_n$ at times $t_1, \dots t_n$ and cumulated consumption levels $y_1=c_1, y_2 = c_1 + c_2, \dots y_n=\sum_{k=1}^{n} c_k$. (I use the quadratic term ...
0
votes
0answers
36 views

Independence of MLEs of 2 parameter exponential, and showing functions of them are chi-square

Consider a random sample of size $n$ from a two-parameter exponential distribution, $X_i \sim $EXP($\theta,\eta$), and let $\eta^*$ and $\theta^*$ be the MLEs. a) Show that $\eta^*$ and $\theta^*$...
2
votes
1answer
56 views

Comparing Variances of two Hit-or-Miss methods for estimating volume

My question is on Exercise 1.4 of Neal Madras' "Lectures on Monte Carlo Methods" (problem pictured below). My current work is as follows: Method 1: Let $X_1,X_2,\ldots,X_N$ be i.i.d. uniform on the ...
0
votes
0answers
32 views

Proof Sample Variance is Minimum Variance Unbiased Estimator for Unknown Mean

I am trying to prove that the unbiased sample variance is a minimum variance estimator. In this problem I have a Normal distribution with unknown mean (and the variance is the parameter to estimate so ...
2
votes
0answers
30 views

Is $\hat \beta$ of general least squares an orthogonal projection?

$\hat \beta_{GLS} = (X'V^{-1}X)^{-1}X'V^{-1}Y \\ Y=X\hat \beta_{GLS} + \epsilon=X(X'V^{-1}X)^{-1}X'V^{-1}Y + \epsilon = \\X(X'V^{-1}X)^{-1}X'V^{-1}Y + (I-X(X'V^{-1}X)^{-1}X'V^{-1})Y$ It is clear that ...
1
vote
0answers
16 views

Non asymptotic error bound for$f(x)=\mathbb{E}[Y|X=x]$

I am considering the following model: $(X_i,Y_i)_{i=1}^n$ are iid random pairs with $X_i\in[0,1]$ and $Y_i\in\mathbf{R}$. Let $f(x)=\mathbb{E}[Y|X=x]$. Consider an estimate $\hat{f}_n$ of $f$. Under ...
1
vote
0answers
14 views

Weakest possible assumptions to guarantee consistency in linear regression estimators?

For a linear regression model where $Y = \beta_0 + \beta_1 X + \epsilon$ and $E[\epsilon|X=x]=0$, $Var[\epsilon|X=x]=\sigma^2$ for any $x$. For a sample $(X_1,Y_1),\dots,(X_n,Y_n)$, I'm wondering if ...
1
vote
1answer
32 views

How to determine margin of error for a population size estimation using a size-biased sampling process?

I am trying to estimate the size of a discrete population $\Omega$. To do so, I draw independent samples $s_1\ldots s_n\in\Omega$ according to some distribution $p(s):\Omega\to(0,1)$. For each ...
0
votes
0answers
22 views

Non asymptotic error bound for non parametric estamation $f(x)=\mathbb{E}[Y|X=x]$

I am considering the following model: $(X_i,Y_i)_{i=1}^n$ are iid random pairs with $X_i\in[0,1]$ and $Y_i\in\mathbf{R}$. Let $f(x)=\mathbb{E}[Y|X=x]$. Consider an estimate $\hat{f}_n$ of $f$. For ...
0
votes
0answers
31 views

Error $|\hat{f}_n(x)-f(x)|$ with regressogram estimator

I am learning about non parametric estimation, and more specifically about regressogram: Let $(X_i,Y_i)_{i = 1}^n$ be a sequence of random variables in $[0,1]$ variables and $E[Y_i|X_i] = f(X_i)$. ...
1
vote
1answer
43 views

Do Bayesian estimators under square error loss have an invariance property?

I feel like this is something we went over in class but it's not coming to me for some reason. I need to find the Bayesian estimator for $\tau(\theta)=e^{-\theta}$ under square error loss. I already ...
0
votes
0answers
26 views

Definition of Bootstrap Estimate (of a statistics) of an Estimator

Given n samples $X_i$ from a population, let $x_i$ be the actual realizations and $g(x_1, x_2, \dots, x_n)$ be a plug-in estimator of the sample mean. Now I am asked to provide a bootstrap estimate of ...
1
vote
1answer
29 views

Are all estimators biased? Is the unbiasedness only a theoretical or approximation case?

The definition of unbiased estimator says that it's expected value has no difference comparing to a true value. So can we say that all estimators are biased (even slightly)? I thought that only in ...
0
votes
0answers
44 views

Variance estimates for a huge number of estimates

I'm estimating a finite number ($\approx1e5$) of integrals $\lambda g$ using the Metropolis-Hastings algorithm with target distribution $\mu=\frac{p\lambda}c$ (where $c:=\lambda p\in(0,\infty)$) and ...
2
votes
1answer
33 views

How to find the bias of $\hat{\sigma}^2=\frac{\sum^n_{i=1}(X_i-\bar{X})^2}{n}$ for the population variance $\sigma^2$

My question is The bias of an estimator $\hat{\theta}$ for parameter $\theta$ is defined as $E(\hat{\theta})-\theta$ please find the bias of $\hat{\sigma}^2=\frac{\sum^n_{i=1}(X_i-\bar{X})^2}{n}$ ...
1
vote
1answer
39 views

Ratio Estimator as WLS

Apologies in advance for whatever rules this post breaks. I'm looking at a problem where we're currently using a ratio estimator for a certain survey. $$r = \dfrac{\sum_i{y_i}}{\sum_i{x_i}}$$ This ...
0
votes
1answer
55 views

Weibull's MLE consistency and asymptotic normality

Let X = $(X_1, \dots, X_n)$ be a sample from Weibull distribution $W(\alpha, \beta)$ with fixed and known $\alpha$. Find MLE of parametric function $g(\beta) = \beta^{\alpha}$. Check if bias is equal ...
0
votes
0answers
30 views

Moments method and plug-in method

We have the following discrete distribution on $\{1,2,3\}$: $$\mathbb{P}(X=1)= \theta^2, \mathbb{P}(X=2) = 2\theta(1-\theta), \mathbb{P}(X=3)=(1-\theta)^2$$ with $\theta \in (0,1)$ and want to use ...
0
votes
0answers
15 views

Use of variance of estimators in cross-validation

Let's suppose we are using K-fold cross validation on a set of data of dimension $N_{data}$. We do not want to fix any parameter but just to get a confidence of the predictive power using the ...
0
votes
1answer
33 views

$(X^\prime X)^{-1}$ as a measure of estimator precision

According to this discussion, the highest variance $X^\prime X$ matrix should correspond with the lowest variance $\beta$ which makes sense to me. But when I ran the following lines I got some curious ...
1
vote
0answers
61 views

Moving estimators for nonstationary time series, like loglikelihood: l_T=sum_{t<T} a^{t-T} ln(rho(x_t))?

While in standard ("static") e.g. ML estimation we assume that all values are from a distribution of the same parameters, in practice we often have nonstationary time series: in which these parameters ...
0
votes
1answer
32 views

Does bias mean additional constant in any estimator? Can I say proportional estimator unbiased estimator?

I saw one question in which the sample mean was estimated as follows (I don't know why they divided by $n-1$ instead of $n$ here for estimation), $$ \widehat{\mu} = \frac{\sum_{i=1}^n x_i}{n-1} $$ ...
1
vote
0answers
26 views

What is an example of a weakly consistent but not strongly consistent estimator?

I just can't think of any example. I am using definitions: weakly consistent: $\forall \varepsilon > 0 \lim_{n\rightarrow \infty} P(|\hat{\theta}_n - \theta| \geq \varepsilon) = 0$; strongly ...
2
votes
1answer
29 views

How to interpret a sampling distribution from a Frequentist and Bayesian perspective

I've read multiple of the threads about Bayesian vs Frequentist interpretations of probability, but I'm having trouble trying to reconcile them with the idea of the sampling distribution when ...
1
vote
1answer
19 views

Monte Carlo Gradient Estimator [closed]

How do we derive this Monte Carlo Estimator? \begin{equation} \nabla_{\phi}\mathbb{E}_{q_{\phi}(z)}[f(z)] = \mathbb{E}_{q_{\phi}(z)}[f(z) \nabla_{q_{\phi}(z)}\ln q_{\phi}(z)] \end{equation}
0
votes
0answers
13 views

Justification for estimating covariance matrix from paper “Averaging Correlated Data”

The paper I am referring to can be found here https://iopscience.iop.org/article/10.1088/0031-8949/51/6/002/pdf. I am in a situation where I have some indexed data $\{y_i | i \in I\}$ with ...
5
votes
1answer
134 views

Lasso, Ridge and Best Subset estimator for orthogonal cases

I am reading the book "Elements of Statistical Learning". In the book the author compares the OSL estimator with Lasso, Ridge and Best Subset for the special case of Orthogonal X. I am attaching the ...
0
votes
0answers
35 views

Breakdown of the median

I am looking for an accessible review of the breakdown properties of the median (estimator of location) and possible practical solutions for dealing with it. (By accessible I mean short and readable ...
0
votes
1answer
21 views

How to find MLE for multinomial distribution and Expectation of X1 and X2

There are 3 types of flowers that can grow from planting a seed. $$P(\text{Daisy}) = \theta_1$$ $$P(\text{Rose}) = (1-\theta_1)\theta_2$$ $$P(\text{Sunflower}) = (1-\theta_1)(1-\theta_2)$$ The total ...
1
vote
0answers
15 views

GEE Std. Error Estimators: Pros and Cons of using jack-knife or bootstrap estimator of std. errors rather than sandwich when clusters$>30$?

What are the Advantages and disadvantages of using jack-knife, bootstrap estimator rather than sandwich (Huber-White) estimator in the context of generalized estimating equations? I heard sandwich ...
4
votes
1answer
65 views

Which is a better estimator, averaged functions vs. A function of an average?

Problem: Assume that we want to estimate $f(\theta)$ with a pre-specified strictly increasing function $f$ and a parameter $\theta$. Let $\hat{\theta}_1$ and $\hat{\theta}_2$ be unbiased estimators ...
0
votes
0answers
16 views

Simplifying last step of IV estimator derivation with matrices

I've been reading about IV in this textbook. On page 39 (page 5 of the PDF), I'm confused by the last step. How does [(z'z)^(-1)z'y]/[(z'z)^(-1)z'x] simplify to (z'x)^(-1)z'y? I can see that the (...
0
votes
0answers
52 views

Using a single sample sequence for estimates of several integrals whose integrands have disjoint support

Let $(E,\mathcal E,\lambda)$ be a measure space $f:E\to[0,\infty)$ be $\mathcal E$-measurable with $\lambda f<\infty$ $q:E\to[0,\infty)$ be $\mathcal E$-measurable with $\lambda q=1$ and $$\{q=0\}\...
2
votes
1answer
48 views

How does Monte Carlo method for parameter estimation work in practice?

I'm new to parameter estimation world and I'm studying this model with two parameters $\mu$ and $\sigma$: $$\tag1 dX_t = \mu X_t dt + \sigma X_t dB_t^H $$ where $B_t^H$ is a fractional Brownian ...
0
votes
1answer
46 views

Variance of a sample and normal distribution

I'm a bit confused about the concept of estimating population variance through a sample. An exercise about theoretical vs empirical distribution of data asks to graphically represent the data of a row ...
1
vote
1answer
138 views

Jensen inequality and bias of sample standard deviation

I am currently studying Introduction to Probability, second edition, by Blitzstein and Hwang. In studying the Jensen inequality, the following example is presented: Example 10.1.6 (Bias of sample ...
1
vote
2answers
93 views

Which estimator is preferred for a random sample from $P_\theta(X=x)=\theta^x(1-\theta)^{1-x}, x=0,1; 0 \le \theta \le \frac{1}{2}$?

Let $X_1,\cdots,X_n$ be an i.i.d sample from $P_\theta(X=x)=\theta^x(1-\theta)^{1-x}, x=0,1; 0 \le \theta \le \frac{1}{2}$. Its the method of moments estimator of the MLE better? Why? My work: I ...
1
vote
1answer
46 views

Consistency of M-estimator?

The reported results can be found in van der Vaart's "Asymptotic Statistics". I am having some difficulties to understand the logic behind the following proof provided by the author: Theorem $5.7.$...
0
votes
0answers
40 views

Warning messages in occupancy modelling (unmarked)

I have an issue with occupancy modelling and I have some warning messages that I don’t understand. I hope you can explain to me what they mean and how I can resolve them. I am building single-species, ...
0
votes
1answer
22 views

If we have Y=β0 + β1D + U, where D is a dummy variable that can be either 0 or 1, how do we prove the estimators of β0 and β1?

Specifically, how do we show β0 is ȳ0 and the estimator of β1= ȳ1- ȳ0 if we know how OLS estimators are normally supposed to be? ȳ1 and ȳ0 are the mean of y1 and y0. I started off with Σ(Di-D̄)yi/...
2
votes
0answers
50 views

Does there exist a Bayesian analysis of bias-variance decomposition of an estimator?

I was wondering if anyone could spare a moment to help with the answers to the following questions. Suppose we have an estimator $\hat{\theta}:\mathbb{R}^{d}\rightarrow\mathbb{R}$ such that the ...
1
vote
1answer
48 views

Variance and expectation of $\frac{1}{n}\sum^n_{i=1}X^2_i$

Let $X = (X_1, . . . , X_n)$ consist of independent and identically Normal $N(0, θ)$ random variables, with mean $0$ and variance $θ \gt 0$. The Moment Estimator for $\theta$ is given by $\hat \theta ...
0
votes
1answer
76 views

What does checking the “behavior of the likelihood on the boundary of the parameter space” mean

What does the latter mean? If someone could explain. I already showed in this question that $\hat \theta(X)$ is the MLE.

1
2 3 4 5
11