"A rule, method, or criterion for arriving at an estimate of the value of a parameter."

learn more… | top users | synonyms

1
vote
0answers
19 views

Variance of an unbiased estimator is 0 when the sample size goes to infinity

So I would like a proof for the following but I can't seem to do it myself. I have a random variable $X$ and I draw $n$ samples($\{X_1, \ldots, X_n\}$) from it and I have $$ Z_n = \frac{\sum_{i = ...
5
votes
1answer
65 views

Variance computed using Taylor series does not agree with numerical experiment

I would like to estimate an angle $\theta\in\left(-\frac{\pi}{2},\frac{\pi}{2}\right)$ given the noisy observations of its sine and cosine (this is related to my earlier question). My estimator is ...
1
vote
0answers
15 views

Variance of Estimator (uniform distribution)

In my script for statistical signals, I have some troubles to get the same result for the variance of an estimator $T$. Here is the example: Given the observations $X_1, \dots , X_N$ of a uniquely ...
4
votes
2answers
33 views

Minimising MSE of $\sigma^2$ estimator of specific form

I have found a past exam question for a statistics course and can't seem to find the required result. Part A is fine but my working for part B must be incorrect [see below]. Can anyone figure out ...
1
vote
0answers
24 views

Non-linear least squares and the distribution of an estimator

I have been trying to find the asymptotic normality of the non-linear least squares estimator. If I start with $0=X_t(\beta)'(y_t-x_t(\beta))$. I know that I have to perform Taylor expansion around ...
0
votes
0answers
5 views

A specific way to minimise the variance when importance sampling

Consider the following problem. We are interested in approximating from samples the expectation of $h$: $$ \int_t p(t) h(t) d(t) $$ We seek to obtain a lower-variance estimate by using importance ...
4
votes
0answers
80 views

What's the difference between asymptotic unbiasedness and consistency?

Does each imply the other? If not, does one imply the other? Why/why not? This issue came up in response to a comment on an answer I posted here. Although google searching the relevant terms didn't ...
4
votes
2answers
70 views

Example of a consistent estimator that doesn't grow less variable with increased sample size?

I've had it asserted to me that any consistent estimator must necessarily also grow less variable with increased sample size. I felt that this couldn't be correct, since there was nothing in the ...
3
votes
1answer
47 views

Estimating R90 (radius of a circle expected to include 90% of impacts)

I want to determine how big a target I can hit with a bow at a certain distance with 90% probability. I place some paper targets at that distance and shoot 20 arrows at them. I have a ruler and a ...
6
votes
1answer
50 views

Estimating a variable from its cosine corrupted by additive Gaussian noise

I observe $y_i=\cos(\theta)+z_i$, $i=1,\ldots,n$, where each $z_i\sim\mathcal{N}(0,\sigma^2)$ is an i.i.d. zero-mean Gaussian random variable. I am interested in estimating $\theta\in[0,\pi]$ with ...
1
vote
2answers
68 views

Rule of Succession for Unfair Coin

Given the first n flip results from an unfair coin, we wish to estimate the probability that the next flip is a heads. I can take 2 approaches to this: Frequentist:...
0
votes
1answer
13 views

Comparing large and small populations

Here is a hypothetical that mimics my actual data (sensitive so cannot post actual details). I have 149 students in a class. In preparation for the final exam (which is pass/fail, NOT ranked), I held ...
-1
votes
1answer
173 views

Why is the intercept of linear regression biased?

Out of curiosity, I conducted the following simulation (code below). Why is it that when the variance of the error term is large coefficient associated with the intercept is biased? Can you recommend ...
0
votes
1answer
17 views

Confidence interval difference for the difference of two estimators

Let us assume that we havo two parameters $\alpha$ and $\beta$ and the two maximum likelihood estimators $\hat{\alpha}$ and $\hat{\beta}$. We have also the confidence interval of these estimators, ...
1
vote
0answers
13 views

How well does sample average range estimates sigma?

Suppose there are m subgroups of n items each. The following practice is described by Montgomery in the construction of a $\bar{X}-R$ chart: Let $\bar{R}$ be the average range of the m subgroups. ...
3
votes
0answers
35 views

Is M-estimation valid only for regression models?

Is M-estimation valid only for regression models or does it's working hold good for robust estimation of parameters in other statistical models? I understand that M-estimators are asymptotically ...
0
votes
3answers
36 views

Square of the Sample Mean as estimator of the variance

Suppose we have the following random variables $X_1$, $X_2$,....$X_n$,.., that are $iid$ but we dont know what distribution they follow. I know that the sample mean $\bar{X}$ is an unbiased ...
0
votes
0answers
17 views

Reporting Data with Estimates and SE

What do you think? When reporting data in an article, a usual way is to show the mean values +/- the standard errors in a table. For Example, when you measure Abundance of pigeons in parks of five ...
1
vote
1answer
30 views

Variance of estimator(exponential distribution)

I have exponential distributed data $Exp(\lambda)$ with sample n = 50. Also, The sample mean = 2.17. I need to find the estimator of parameter $\lambda$ by the method of moments and to build 95% ...
0
votes
0answers
17 views

Are point estimators and set estimators the same thing?

My professor defined a point estimator as A mathematical rule that maps a random sample $\lbrace x_i \rbrace_{i=1}^n$ into a 'best guess' at the parameter $\theta$ I am confused about what ...
1
vote
0answers
33 views

What are the bias and variance of a model returning the observed mean for a training set?

It seems to me that bias = variance = 0 but MSE > 0, possibly very high, so clearly my intuition, and math, are wrong. For a training set $T$ and a regression problem let $M(T) = \text{Ave}(y(T))$. ...
25
votes
2answers
992 views

When is a biased estimator preferable to unbiased one?

It's obvious many times why one prefers an unbiased estimator. But, are there any circumstances under which we might actually prefer a biased estimator over an unbiased one?
1
vote
1answer
54 views

Identification from minimum value of truncated distribution

Suppose that a given population is endowed with a pair of characteristics $T$ and $K$. Let's think of these characteristics as random variables $$(T,K) \sim \operatorname{BiNormal}((\mu_T, \mu_K), (\...
0
votes
0answers
14 views

Cancelling roots in ARMA(1,1) with external regressors

I am trying to find out what cancelling roots would imply for the estimators of my external regressors in my ARMA(1,1) model. Unfortunately however I'm stuck in my final step since I'm insecure about ...
4
votes
1answer
263 views

root-n consistent estimator, but root-n doesn't converge?

I've heard the term "root-n" consistent estimator' used many times. From the resources I've been instructed by, I thought that a "root-n" consistent estimator meant that: the estimator converges on ...
0
votes
0answers
12 views

(Self-study) Optimal Estimators that minimizes MSE

Show c||x-(x_bar)(1)||^2 minimizes MSE among all such estimators for c = 1/(n+1) I understood everything about Lehman-Scheffe, Rao-Blackwell, UMVU and such, but I have no idea how to get started on ...
0
votes
0answers
12 views

Asymptotics of the estimator y'y/y'x in a linear model

I am trying to learn to understand how to derive asymptotic distributions. In an exercise, I am trying to analyze the asymptotic behaviour of the estimator $\hat{\beta} = \frac{y'y}{y'x}$, where $y = ...
8
votes
2answers
150 views

Is bias a property of the estimator, or of particular estimates?

As an example, I often encounter students who know that Observed $R^2$ is a biased estimator of Population $R^2$. Then, when writing up their reports, they say things like: "I calculated Observed $R^...
0
votes
0answers
18 views

What is a good estimator for the reciprocal of covariance?

Let $X,Y$ be random variables with unknown but nonnegative covariance. What is a good estimator for $1/\operatorname{Cov}[X,Y]$? Specifically, how does one deal with negative sample covariance when it ...
1
vote
0answers
8 views

The estimator of variance of linear regression targets

In Section 3.2 of the book The elements of Statistical Learning (2ed), I read the text: Typically one estimates the variance $\sigma^2$ by $$ \hat{\sigma}^2 = \frac{1}{N-p-1}\sum_{i=1}^{N}(y_i-\hat{y}...
1
vote
0answers
25 views

Linear regression: Estimator for OLS and Minimum Absolute Deviations

I've received conflicting information in 2 different statistics classes, and I want to better understand the problem before I ask either of them for clarification Prof 1: We use OLS for linear ...
1
vote
0answers
20 views

Procedure for calculating a sampling distribution

I'm still trying to understand the basics of understanding the intuition of sampling distributions and calculating the sampling distributions of common estimators. For example, I understand the ...
3
votes
0answers
68 views

Paired comparison of instruments using different measurement samples

I have instruments A, B, C and D - I'm in search of the best one. The problem: For illustrative purposes, let's use an example of evaluating the best among the instruments measuring difference in ...
0
votes
0answers
9 views

help me understand the proof in the paper “restricted ridge estimation”

I'm reading the paper "restricted ridge estimation" by Grob(2003). I can not understand the proof of theorem 1 in this paper. I don't know how this estimator $\hat{\beta}_{r}(k) = \hat{\beta}(k,\...
1
vote
1answer
93 views

Comparing spread (dispersion) between samples

EDIT: This question has been heavily paraphrased and re-asked in a broader, but better way here: Paired comparison of instruments using different measurement samples This is going to be a long one, ...
0
votes
1answer
25 views

Most likely event in a multinomial distribution setting

I'm looking at the following scenario: $k$ categories, distributed by a multinomial ($p_1,\dots,p_k$) such that $p_1 \ge \dots \ge p_k$. Draw $n$ samples. I'm interested in estimators/lower bounds ...
1
vote
1answer
26 views

Compare two multivariate distributions

I have a multivariate distribution (for which I know the parameters) that I simulate data from. I then fit several distributions to this simulated data using several different approaches (similar to ...
1
vote
1answer
35 views

Linear Regression - Conditions for unbiased estimate

When is the linear regression estimate of $\beta_1$ in the model $$ Y= X_1\beta_1 + \delta$$ unbiased, given that the $(x,y)$ pairs are generated with the following model? $$ Y= X_1\beta_1 + X_2\...
1
vote
1answer
35 views

If $X \sim \mathcal{P}(u)$, show that $S=(-1)^X$ is the UMVUE of $e^{2u}$

If $X \sim \mathcal{P}(u)$, show that $S=(-1)^X$ is the UMVUE of $e^{2u}$. I can't figure this out, finding UMVUE always confuses me. Any help is greatly appreciated
2
votes
3answers
63 views

How to Prove Unbiased Estimator

I'm unsure of how to convince myself that $$\hat{\beta} = \frac{\sum X_i Y_i}{\sum X_i^2}$$ is an unbiased estimator when the regression model $$Y_i = \beta X_i + \epsilon_i$$ follows basic OLS ...
4
votes
1answer
75 views

“Consistent estimator” or “consistent estimate”?

Question: Are both expressions "consistent estimator" and "consistent estimate" meaningful? The quote below is intended to be illustrative; however, I am interested in the question above in a ...
1
vote
0answers
27 views

robust estimator when $\mu/\sigma$ is constant

I have a large set of measurements consisting each of about 100 points, (normally distributed), with up to 20% outliers. The outliers are all shifted towards positive values. From the physics, I know ...
0
votes
0answers
45 views

Minimum mean squared error linear combination of random variables

Consider the following objective function: $$ \mathbb{E}((Y-X\beta)^2)\rightarrow \min_\beta $$ where $Y$ and $X$ are (generally not independent) random variables and $\beta$ is a constant. That is, ...
1
vote
2answers
71 views

Estimating unconditional variance in time series

Consider a time series process with a well-defined, finite unconditional variance. Given a realization of the process (a time series) and a model for it, there are at least two ways of estimating the ...
1
vote
0answers
38 views

When to assume that $\Pr(X=x, Y=y) = |\mathcal{Y}|^{-1} \Pr(X=x|Y=y)$?

Background Let $\mathcal{X} = \{x_1, x_2, \ldots, x_n\}$ be a set of samples, each that corresponds to a label in the set of labels $\mathcal{Y}$. Ideally, our objective is to find the joint ...
4
votes
1answer
55 views

Cramér-Rao Lower Bound for Exponential Families

I am having a problem with applying the Cramér-Rao inequality to identify the lower bound for the variance of an unbiased estimator and hoped that you guys could help me. The problem is the following: ...
1
vote
1answer
57 views

How to get the maximum likelihood estimator of $U(\theta,\theta +1)$?

I know how to find the MLE for $U(0,\theta)$ but I am in trouble with this one. let $X_1,\dots,X_n$ be a random sample from $U(\theta,\theta +1)$. Consider the following three estimators for $\...
1
vote
0answers
35 views

Show that weighted least squares estimator for a specific model is not consistent

Here is the background for this problem: $\qquad\qquad\qquad$ $Y_{1},...,Y_{n}$ iid $N(\mu,c^2\mu^2)$, $\,\,$ $c^2$ known. $\,$ The problem is as following: Consider the above model. Define $\hat{\...
0
votes
0answers
43 views

Negative of log-likelihood on test data decreases but the parameters Mean Square Error(MSE) increases. How to justify the situation?

We develop an EM algorithm to model a problem. We generate some synthetic data of the model with parameters $\Theta$. We call the data $\text{D}$ which is decomposed it into two separate sets, $\text{...
-1
votes
1answer
29 views

Normal approximation of parameter $p$ of $Bin(n,p)$? [closed]

I've seen normal approximation applied for approximating a binomial distribution $B(n,p)$. However, if one estimates the parameter $p$, then can the parameter $p$ be "normally approximated" just as ...