All Questions
Tagged with variance estimation
162 questions
0
votes
0
answers
11
views
Variance estimation from dependent data
I would like to estimate the variance of a zero-mean normal distribution, $x_n \sim \mathcal{N}(0, \sigma^2)$, from data of the form $y_n = u_n x_n$ where the input $u_n \in [u_{\min}, u_{\max}]$ can ...
- 1
1
vote
1
answer
43
views
Unbiased Variance MLE Distribution
If you take $10000$ samples from a normal distribution, the unbiased variance MLE (with Bessel's correction) is
$$\hat{\sigma}^2 = \frac{1}{9999}\sum_i (x_i - \hat{\mu})$$
Apparently the distribution ...
- 503
1
vote
1
answer
45
views
Cross-fitting seems to always reduce asymptotic variance for estimators converging slower than $\sqrt{n}$ - how can this be true?
Setup: Imagine the situation where you for a fixed value of your covariates have a regression estimator $\tilde{f}$ based on $n$ i.i.d. observations which is asymptotically normal with convergence ...
6
votes
1
answer
245
views
Variance of MLE's in mixture distribution
I am studying mixture models, and I am interested in calculating the variance of the estimators using maximum likelihood. How is the variance calculated in this case? I already implemented the EM ...
- 281
1
vote
1
answer
29
views
Calculating mean variance between double determined measurement of random variable
I have two sets of data, measuring a varible that changes at random (concentration of a gas). The measurement are double determined, providing two data points for each measurement.
I would like to ...
William
0
votes
1
answer
35
views
Error in derivation of variance of $\beta_1$ in SLR [duplicate]
I'm trying to derive the variance of the slope parameter for a simple linear regression in the following way, however I'm running into an issue I don't know how to resolve. Define $y_i=\beta_0+\beta_1\...
- 181
1
vote
0
answers
32
views
Conditional variance of random walk with given start points and ending points
If I have a random walk with 0 drift and I observe that $X_1 = x_1$ and $X_k = x_k$, bu I don't have all the points from $X_i$ for $i \in \{2, ..., k-1 \}$, how do I estimate them and given an CI? I ...
- 235
0
votes
0
answers
25
views
Estimating variance from several samples
If several samples are taken from a distribution, say Gaussian, each sample having size n1,n2,n3,... and the SD of the underlying distribution is estimated from each of the samples, how can those ...
- 152
0
votes
0
answers
40
views
Can I compare two estimates by their sample variance?
For example, to estimate the population mean $\mu$, I am given two sample mean $\bar{x}_1$ and $\bar{x}_2$ from two (independent) data sets of $N_1$ and $N_2$ observations respectively.
Without access ...
- 51
1
vote
0
answers
56
views
Standard deviation of standard deviation under non-normality
In this post, an unbiased estimator for the standard deviation of the standard deviation under normality is provided.
I would be interested in such an estimator without the normality assumption, i.e., ...
- 435
1
vote
0
answers
103
views
Adjusting standard errors in two-step maximum likelihood estimation
Suppose we want to solve
$$max_{\theta} \sum_{i}^N log f(y_i|x_i; \theta, \gamma).$$
Here, $\theta$ and $\gamma$ are two parameter vectors. The problem above derives an estimate of $\theta$, taking ...
- 21
1
vote
1
answer
40
views
Is there a theory for estimating "node differences" using "edge samples" over graph
Assume a system consisting of several components. Each component is characterized by some real number.
A sample of a pair of components in the system, is a random variable normally distributed around ...
- 13
1
vote
1
answer
1k
views
Bootstrap for variance estimation
I think I might have misunderstood the purpose of the bootstrap, because I think the following argument proves that bootstrapping is useless.
Let $N \in \mathbb{N}^*$, $(X_n)_{n < N}$ a $N$-tuple ...
- 262
0
votes
2
answers
105
views
Finding the variance of this estimator
I'm not sure how to express the variance of this estimator. Here's the setup.
We have $X\sim N(0,\sigma^2)$ and want to estimate $\mathbb{E}[\phi(X)]$ where $\phi : \mathbb{R}\to\mathbb{R}$ is some ...
- 103
0
votes
0
answers
30
views
How to find the variance of an independent variable across the big number of linear regression equations?
Question
I have a big number of linear regression equations with known dependent variables and coefficients, in a form of:
T = Aa + Bb + Cc + Dd
where ...
- 121
5
votes
3
answers
1k
views
Variance estimation for small sample size
The following variance estimator of a set of data points $x = (x_1, ..., x_N)$
$$
\text{Var}\,(x) = \frac{1}{N-1} \sum_{i=1}^N (x_i - \bar{x})^2
$$
has itself a large variance when $N$ is small (in my ...
- 51
0
votes
0
answers
20
views
What is the expression for covariance in the context of Monte-Carlo estimator? [duplicate]
I am trying to calculate the variance:
$$
\langle(\bar{O}-<O>)^2\rangle
$$
of the Monte-Carlo estimator
$$
\bar{O}=\frac{1}{M}\sum_{m=1}^M{O_m}
$$
For uncorrelated samples.
In order to do so, I ...
- 1
4
votes
1
answer
55
views
Behavior of AR process
It is known that variance of AR(1) $y_t=\phi y_{t-1}+\varepsilon_{t}$ is
$$\text{Var}(y)=\frac{\sigma_\varepsilon^2}{1-\phi^2}$$
Let $\varepsilon_{t}$ has continuous uniform distribution on $[-1, 1]$ ...
- 253
1
vote
0
answers
49
views
Estimating heteroscedastic variance using a log transform
In a paper I read, they are using a NN to estimate the heteroscedastic variance in a regression scenario (actually a bit more complicated than this, but irrelevant to the question). After they fit 1 ...
- 3,680
1
vote
0
answers
51
views
"Variance" estimate for quasi-Monte Carlo
Problem Setting
I am playing with a toy example and I would like to better understand the variance results that I get when using low-discrepancy sequences versus random values. I have independent and ...
- 111
1
vote
0
answers
27
views
Estimation of mean, variance, proportion for cakes baking in 30 ± 2 minutes, or estimation of parameters of a law for a distribution: is it the same? [closed]
I'm a beginner in statistics and I'm trying to figure things to ensure that I can consider parameters the same manner in two cases :
I believe that if we have enlighten a distribution with the ...
- 243
0
votes
0
answers
33
views
Condition for the asymptotic non-zero point estimation of the variance
we know that a condition for a non-zero point estimate of the variance for a finite sample is that there exist at least two integers $i,j$ such that $X_i\neq X_j$. In other words $\frac{1}{n}\sum\...
1
vote
0
answers
50
views
Tradeoff when estimating the variance of the autocorrelation estimator
Autocorrelation estimator
Given a wide-sense stationary process $\{X_t\}_{t\in\mathbb{N}}$ one can estimate its autocorrelation defined as $R[k]=\mathbb{E}[X_tX_{t+k}]$ from $N$ observations $\{X_1,...
3
votes
1
answer
101
views
Ideal Settings for Longitudinal Models?
The way I see it, logically speaking - Longitudinal Data (e.g. medical patients being measured repeatedly over a period of time) can have one of two forms:
Case 1: All patients are measured exactly &...
0
votes
1
answer
2k
views
What is the variance of $s^2$?
I am trying to calculate the variance of $s^2=\frac{1}{n-1}\sum (x_i-\bar x)^2$. So what I want to find is $ Var(s^2)$.
I have seen different posts, but many of them seem to make the assumption that ...
- 3
1
vote
1
answer
562
views
Empirical variance of simulation estimate
Consider the following quantity of interest:
$$I[a,b]=\int_{a}^{b}g(\theta)h(\theta), \ldots (1)$$
that is, the expected value of some function $h(\theta)$, of $\theta$ distributed $g(\theta)$.
...
- 309
4
votes
1
answer
174
views
How can population variance be estimated from a bivariate sample?
Let's assume a bivariate population with a correlation $\rho$ and a common $\sigma$ so that $\Sigma = \sigma^2 \begin{pmatrix}1 & \rho \\ \rho & 1\end{pmatrix}$.
I would like to know the ...
- 351
3
votes
3
answers
375
views
We're estimating mean, variance, proportion or compare samples with them. I understand for mean and variance. But why is it required for proportions?
I have learned how to estimate a mean, variance or proportion from a sample.
and also, how to compare those for samples.
I'm understanding well why we might need to estimate or compare means or ...
- 243
1
vote
0
answers
23
views
does assumptions effect the bias or variance?
in machine learning text it is often said that assumptions affect bias like the following text from Kevin Murphy:
"Given the large variety of models in the literature, it is natural to wonder ...
- 21
1
vote
0
answers
358
views
Variance of a vector function
One way of defining the variance of a vector is as follows
\begin{align*}
\text{Var}(g) = \mathbb{E}[ \, \lVert g \rVert_2^2\, ] - \lVert\,\mathbb{E}[ g ] \, \rVert_2^2. \tag{1}\label{1}
\end{align*}
...
- 380
1
vote
1
answer
612
views
How does Michaud Resampling improve Mean-Variance Optimization?
Michaud Resampling claims to reduce estimation error through the following process:
Step 1. Sample a mean vector and covariance matrix of returns from distribution of both
centered at the original (...
- 13
11
votes
2
answers
2k
views
How to estimate the variance of correlated observations?
Assume we have n observations $x_i$ (i from 1 to n), each from the a normal distribution with mean 0 and some variance component: $X_i \sim N(0, \sigma^2)$. The random variables $X_i$s have some (let'...
- 21.9k
2
votes
1
answer
55
views
Is it harder to estimate the variance of a Gaussian compared to its mean?
In various ML talks I keep hearing that variance estimation is harder than mean estimation but I never really get why the above statement is correct. Is there a theoretical argument or a published ...
- 1,502
7
votes
2
answers
303
views
variance estimation using order statistics
I have four largest samples drawn from a distribution of N i.i.d Gaussian R.V. with standard deviation (Sigma) where sigma is unknown. N is known to be between 50-200. Mean is given to be 0.
How do ...
- 71
0
votes
0
answers
48
views
unbiased estimation of the variance of $p$ (proportion) of a random sample without replacement
Given a random sample without replacement of size $n$ from population of size $N$ and $p$ is the estimator of the proportion $P$. How could one show that:
\begin{equation*}
\frac{N-n}{N(N-1)}pq
\end{...
- 145
1
vote
1
answer
45
views
Where plus 1 came from in variance estimation [duplicate]
While
$$
\mathrm{E}(\tilde{\mathrm{y}})=\alpha+\beta \tilde{\mathrm{x}}
$$
Subject is Regression Analysis and this formula is from the "Features of Estimation ".
and y is a neutral variable.
...
1
vote
0
answers
13
views
Choose strata size so that dispersion of an estimator is the least
The reseacher has 20000 dollars in their disposition to run a survey. It's known that from all households, 90% have stationary phones. Interview by phone costs 10 dollars per household. Each ...
- 157
1
vote
1
answer
1k
views
How to calculate a var of the sum of two coefficients in linear regression [duplicate]
Essentially after performing regression on three variables,
$$
y = a_0 + a_1 \cdot x_1 + a_2 \cdot x_2 + a_3 \cdot x_3
$$
I want to find variance for $a_1+a_2$ to get CI. Logically, I think I can do
$$...
- 21
1
vote
1
answer
73
views
Which variance should I use
Let $X_1,\ldots X_n \sim Bern(p)$, be random variables with Bernoulli distribution and $x_1,\ldots x_n$ the observed data. This distribution has $\sigma^2$ the variance. Suppose I choose for the ...
- 547
4
votes
0
answers
1k
views
What is the probability distribution and variance of the OLS estimate $s^2$ of the error variance $\sigma^2$ in linear regression?
Consider the standard linear regression model
$$
y = X \beta + \varepsilon,
$$
where the error $\varepsilon$ has fixed variance $\sigma^2$. We can make an unbiased estimate of the error variance in a ...
- 805
1
vote
0
answers
36
views
maximum likelihood estimation of the variance [closed]
In what situations the maximum likelihood estimation of the variance of distribution can severely ruin the estimation?
- 149
2
votes
1
answer
135
views
Why do the mean and proportion measurements take the spotlight in estimation?
Based on information I have read and from this website, sampling distributions do exist for statistic variants of measurements other than the mean. Sample ranges, maxima, minima, variance and ...
- 231
2
votes
1
answer
41
views
Variance Estimator Change if we know Population Mean? (Normal dist. example)
For a normal distribution $N(\mu, \sigma^2)$ a commonly used unbiased and consistent estimator of variance is
$$\hat \sigma^2=\frac{\sum_ix_i^2 + n(\bar x)^2}{n-1}=\frac{\sum_i(x_i-\bar x)^2}{n-1}$$
...
- 174
1
vote
2
answers
284
views
Doubt in derivation of expectation of sample variance
I am studying statistics on my own. Please help me in understanding following
Here in evaluation of expectation $E[\frac{(n-1)S^2}{\sigma^2}]$, why $\sigma^2$(population variance) is treated as ...
0
votes
0
answers
25
views
Combination of different estimates of the same quantities?
Suppose we have $n$ number of estimates for a parameter, each derived independently. How will one combine these estimates to get a single estimate which has lower variance than each individual ...
- 1
5
votes
1
answer
366
views
Point estimator for product of independent RVs
Let $X$ and $Y$ be two independent random variables. Given an (iid) random sample of size $n$ of $X$ and a random sample of size $n$ of $Y$, what is a good way to estimate the mean of their product, $...
- 359
5
votes
2
answers
94
views
What estimation method establishes sample mean and sample variance as estimators of mean and variance?
Sample mean and sample variance can be derived as MLE estimators for the mean and variance of a normal distribution.
For a distribution in general, what kind of estimation method leads to sample ...
- 19.8k
4
votes
1
answer
577
views
Variance of MLE of a function of bernoulli parameter
Given $m$ i.i.d. Bernoulli( $\theta$ ) r.v.s $X_{1}, X_{2}, \ldots, X_{m},$ I'm interested in finding the variance (Not asymptotic) of estimator of $(1-\theta)^{1 / k},$ when $k$ is a positive ...
- 224
2
votes
1
answer
86
views
Best Choice of Estimator. How to compute Variance of Estimator.Basis for it?
I am working on an exercise asking which of the two following estimators; $X1, X2 $for the population mean of a normal population with parameters $ \mu, \sigma$ is best and why
$X1:=\frac {X_1+X_2+......
- 579
8
votes
1
answer
954
views
The "correct" way to approximate $\text{var}(f(X))$ via Taylor expansion
tl;dr: There are two commonly reported formulas for approximating $\text{var}(f(X))$, but one is notably better than the other. Since it isn't the "standard" Taylor expansion, where does it come from, ...
- 722