All Questions
1,569 questions with no upvoted or accepted answers
12
votes
0
answers
14k
views
How to normalize data prior to computation of covariance matrix
In all my self-study, I have come across many different ways in which people seem to normalize their data, prior to the computation of the covariance matrix. I am confused as to what ways are 'correct'...
- 1,685
9
votes
0
answers
2k
views
Taylor Series and Multivariate Delta Method
I asked this question on https://math.stackexchange.com/ but did not get any answer. Sorry for cross posting.
I'm trying to understand delta method for matrices and vectors to find the variance-...
- 2,759
8
votes
0
answers
450
views
What does the second moment tell us that variance does not?
What does the second moment tell us that variance does not?
I can wrap my brain around what the first moment tells us, and I can wrap my brain around what the variance tells us, but interpreting the ...
- 67.2k
8
votes
0
answers
1k
views
Applying a variance-stabilizing transform to a fitted function (rather than data)
Outline
I'm working with data corrupted by a mixed Poisson-Gaussian noise model (for example with images gathered in astronomy or electron microscopy), and have been using the generalized Anscombe ...
- 163
7
votes
0
answers
102
views
Keeping track of the variance of a Metropolis-Hastings estimator
Let $(E,\mathcal E,\lambda)$ and $(E',\mathcal E',\lambda')$ be measure spaces, $p,q$ be probability densities on $(E,\mathcal E,\lambda)$, and $\varphi:E'\to E$ be bijective and $(\mathcal E',\...
- 213
7
votes
0
answers
241
views
why use diagonal $\Sigma$ when working with Bayes decision theory?
My prof. said in the class that for Bayes decision rule, the likelihood is Gaussian and in practice, we will almost always work with a diagonal $\Sigma$. Why is that? I know that a diagonal $\Sigma$ ...
- 235
7
votes
0
answers
1k
views
Bias Variance tradeoff from a Bayesian perspective
I know the general question about bias variance has been asked before. I understand the frequentist approach and the concept of model selection and the impact of bias and variance on "accuracy" of a ...
- 131
7
votes
0
answers
192
views
Estimating population size, minimum variance estimators
I am trying to understand what can be proved about minimum variance estimators. I am a little confused by Cramér–Rao and how to apply it even to really simple examples or if it's even the right tool ...
- 71
7
votes
1
answer
125
views
Measurement error in maximum counts
I'm familiar with the concept of a mean value of data and the variation around the mean. Is it possible to quantify variation around maximum values?
For example, take the below data collected across ...
- 14.6k
6
votes
0
answers
1k
views
Calculating a sample size based on the target width of a confidence interval with stratification
I am reviewing a sampling design devised by a colleague and completely fail to understand it, although I am not a novice in statistics (but not a huge expert either). The said colleague is no longer ...
- 572
6
votes
0
answers
681
views
Gradient-informed global optimization
I am looking for a review or comparison of global optimization techniques where the gradient of the function is available and utilized to speed up search, like the following:
A hybrid descent method ...
- 1,033
6
votes
1
answer
353
views
Comparing variances of forecast errors
I am forecasting a weekly commodity price series. I use a rolling window for estimating my model, and from each window I make point forecasts for one and two steps ahead.
I want to investigate ...
- 69.5k
6
votes
0
answers
1k
views
Asymmetric confidence intervals on bootstrap estimates
I've performed bootstrapping on my leastsq parameters and now I have a load of data from which I can get the mean and standard deviation for each parameter. Lovely.
But when I look at a histogram of ...
- 479
5
votes
0
answers
235
views
Running maximum of $\sum_{1\leq k\leq n} X_i$ for Cauchy random variables $X_i$
Suppose $X_i$ are $\mathrm{Cauchy}(0,~\gamma)$ IID RV's and let $S_n=X_1+\cdots+X_n$ be their sum. Does an expression exist for the CDF of the running maximum up to an index $1 \leq k \leq n$?
Edit:
...
- 61
5
votes
0
answers
194
views
Why is homoscedasticity (homogeneity of variance) important in neural network layers?
I'm studying the famous Xavier initialization paper (Understanding the Difficulty of Training Deep Feedforward Neural Networks (Glorot and Bengio, 2010)) and had a question.
When they explain the ...
- 4,177
5
votes
1
answer
1k
views
Relation between bias and R-square
I am trying to understand relation between bias and R-squared value in linear regression.
High bias means that the model is underfit. By this I am assuming that the R-square d will be less.
So my ...
- 61
5
votes
0
answers
182
views
Ratio = 1 or Difference = 0
In two-sample testing, a standard way of examining for differences in variance is testing a null hypothesis that the ratio of two variances is 1. Let's set up the problem.
Why not test that the ...
- 67.2k
5
votes
0
answers
387
views
Is my explanation of profile likelihood plots correct?
Using the metafor package in R to conduct a mixed-effects meta-analysis and meta-regression, I checked the profile likelihood ...
- 491
5
votes
0
answers
343
views
Variance of quotient of Poisson random variable and sum of the Poisson sample
Let
$$Y_1\sim \operatorname{Poisson}(\lambda_1)\\Y_2\sim \operatorname{Poisson}(\lambda_2),$$ where $Y_1$ and $Y_2$ are independent, and $\lambda_1, \lambda_2>0$.
What is the variance of $$\frac{...
- 105
5
votes
0
answers
396
views
Fourier transform of white noise - Phase and magnitude?
Assume that $X(t)$ describes white noise in time, with $\langle X(t) \rangle =0 $ and $\langle X(t) X(t') \rangle = \sigma^2 \delta(x-x')$. I want to know the distribution of it's Fourier transform. ...
- 151
5
votes
0
answers
618
views
Noninformative prior for variance, understanding and coding
I have three questions regarding the understanding behind and implementation of a noninformative prior for variance. I'm attempting to build a Metropolis sampler and I'm trying to sample from a ...
- 619
5
votes
0
answers
657
views
Does pooled variance correct for/protect from unequal variance when calculating effect size?
This may be a lame question, but I got stuck and can't get my head around it. I am running a gene expression analysis, comparing $\sim 10,000$ genes between two groups, $n=6$ samples per group. My ...
- 211
5
votes
1
answer
855
views
Find variables most responsible for variance between groups
I have a set of data with continuous features $x_1, x_2,...,x_n$, as well as a continuous $y$ which is some complicated, unknown function of the $x_i$. Each data point, furthermore, has a discrete ...
- 116
4
votes
0
answers
479
views
What's the intuition behind the fact that sample mean and sample variance are independent when sampling from a normal population?
Let $X_1, \dotsc,X_n$ be i.i.d. from $N(\mu,\sigma^2)$, then we know that sample mean $\bar X\equiv \frac{1}{n}\sum_{i=1}^nX_i$ and $S^2=\frac{1}{n-1}(X_i-\bar X)^2$ are independent. Obviously, they ...
- 3,050
4
votes
2
answers
597
views
The variance of the weighted median and optimal weights
The median $\tilde{\mu}$ of a sample in many ways is analogous to the sample mean $\mu$.
Both are an estimate for the population median or mean respectively, and both approach a Gaussian distribution ...
- 151
4
votes
0
answers
55
views
Show that unimodal distribution variance is smaller than uniform distribution
Let $f(x)$ be a unimodal distribution on bounded interval $[-1,1]$. Can we show that the variance is upper bounded by variance of Unif$[-1,1]$?
- 41
4
votes
0
answers
83
views
The use of a pseudo variance $\int_{-\infty}^\infty \text{sign}(x) (x-\mu)^2 f(x) dx$ in place of $\int_{-\infty}^\infty (x-\mu)^2 f(x) dx$
Difference between odd central moments and even central moments
When we compute $n$-th central moments
$$\mu_n = \int_{-\infty}^\infty (x-\mu)^nf(x) dx$$
then there is a difference in interpretation ...
- 86.5k
4
votes
0
answers
75
views
Estimation of the density at the bound of the support of a real random variable
Let $X$ be a random variable with real values and with density $f$.
Assume the support $f$ is bounded with supremum $m$ and has a positive value at that supremum:
$$\forall x > m, f(x) = 0 \text{ ...
- 2,629
4
votes
0
answers
119
views
truncation of bivariate normal under quadratic condition
Consider a complex normal variable $Z \sim \mathcal{CN}(\mu,2\sigma^2)$ with real component $X \sim \mathcal{N}(\mu,\sigma^2)$ and imaginary component $Y \sim \mathcal{N}(0,\sigma^2)$. We can write ...
- 121
4
votes
1
answer
234
views
Are power law relations between means and standard deviations inherent in normally distributed data?
In a recent paper I submitted for publication I document a power law relation between the means and standard deviations of several time series. That is, when plotting the log of the means of each of ...
- 515
4
votes
0
answers
334
views
Variance of a gamma distribution being proportional to its mean - a vacuously true statement?
I used to believe that the negative binomial distribution (for count data) and Gamma distribution (for continuous data) shared the property that the variance can take arbitrary values regardless of ...
- 816
4
votes
0
answers
1k
views
What is the probability distribution and variance of the OLS estimate $s^2$ of the error variance $\sigma^2$ in linear regression?
Consider the standard linear regression model
$$
y = X \beta + \varepsilon,
$$
where the error $\varepsilon$ has fixed variance $\sigma^2$. We can make an unbiased estimate of the error variance in a ...
- 805
4
votes
0
answers
770
views
Error bars in repeated k-fold cross-validation
Suppose I want to compute the expectation of the loss $L$ based on Repeated K-fold Cross-Validation (KFCV). Just to be precise by repeated KFCV I mean the following: I repeat the $K$ cross-validation ...
- 141
4
votes
0
answers
116
views
Variance of Multivariate KDE
I am struggling for 2 hours now and decided to give up. I want to compute the variance of the KDE $$\hat f_H(x) = n^{-1}\sum_{i=1}^n\det(H^{-1})K(H^{-1}(x - X_i)).$$
My steps:
I got to the point ...
- 399
4
votes
0
answers
125
views
How are the variance of an estimate to $\int_Bf\:{\rm }d\mu$ and the deviation of $f$ from the mean $\frac1{\mu(B)}\int_Bf\:{\rm d}\mu$ related?
Let $(E,\mathcal E,\mu)$ be a probability space, $(X_n)_{n\in\mathbb N_0}$ be an $(E,\mathcal E)$-valued ergodic time-homogeneous Markov chain with stationary distribution $\mu$ and $$A_nf:=\frac1n\...
- 213
4
votes
1
answer
826
views
Why is the Neyman estimator for the variance of diff in means conservatively biased?
Numerous lecture slides and papers claim that the Neyman estimator for the variance of difference in means is conservatively biased (i.e., the estimate is larger than the true variance). I must be ...
- 61
4
votes
0
answers
76
views
How Does Variance Propagate From Likelihood Function To MCMC Posterior?
Suppose we are trying to obtain the posterior distribution of three parameters that influence a discretely observed population. The likelihood function is unfortunately intractable, as it is a mix of ...
- 51
4
votes
0
answers
155
views
Is it possible to show that this estimator has minimum variance?
Doing some exercises I stumbled upon this tricky one:
Suppose we have an independent random sample $(X_1, ... , X_n)$ with $X_i \sim Poisson(\lambda)$. Define $\theta = e^{-\lambda}$.
Let $$ \...
4
votes
0
answers
83
views
How to pick the daily volatility component in Multiplicative Components GARCH modelling?
Recently I've been drawn to the rather interesting Multiplicative Components GARCH model for intraday volatility modelling, a draft paper written on it can be found here: Chanda, Engle, Sokalska, 2005 ...
- 282
4
votes
0
answers
149
views
Log-normal variance estimation
I have generated some log-normal sample data with python (I tried also with Wolfram Mathematica). Let's say with parameters $\mu = -14.6$ and $\sigma = 3.6$, but even with other parameters I observe ...
- 51
4
votes
0
answers
153
views
Entropy evolution while learning?
It is fairly well known that $$H(X|Y)\le H(X),$$ the posterior entropy is smaller than the prior entropy. This is similar to
$$\mathbb{E}_Y[\mathbb{V}ar_X[X|Y]]\le \mathbb{V}ar_X[X]$$ which follows ...
- 611
4
votes
0
answers
223
views
How to prove that the prior for which Bayes rule is also the minimax rule, is the least favorable prior?
I have read in the book Mathematical Statistics: A Decision Theoretic Approach by Thomas Ferguson that The prior for which the Bayes rule is also minimax rule, then that prior is Least favorable prior....
- 51
4
votes
0
answers
374
views
Confidence Interval vs Credible Interval for the Variance
I understand the conceptual difference between confidence and credible intervals. But I have difficulties applying these concepts to my application.
I would like to know the concrete difference ...
- 435
4
votes
0
answers
66
views
Brownian bridge to unknown via extremum
Suppose, I know what's the minimum $\min$ of a random walk $w_t$ in period $[0,\Delta t]$. I also know $w_0$ and $\sigma$. How to construct the Brownian bridge for the latter period?
I guess it's not ...
- 62.3k
4
votes
0
answers
680
views
Is there a relationship between R squared and linear regression coefficients variances?
I performed a multivariate meta-analysis for a set of regression lines, that each of them represent the relationship between soybean-yield a and a fungal disease in a field experiment.
The meta-...
- 389
4
votes
0
answers
218
views
Comparison of Difference of Expectations of Conditional Variances
I want to show (if possible) that
$$\mathrm{E}[\mathrm{Var(Y|X_1, X_2)}] - \mathrm{E}[\mathrm{Var(Y|X_1)}] \geq \mathrm{E}[\mathrm{Var(Y|X_1, X_2, X_3)}] - \mathrm{E}[\mathrm{Var(Y|X_1, X_3)}] \tag 1$...
- 113
4
votes
1
answer
439
views
Population, sample and model
Background:
In Statisical Methods by Pfaffenberger and Patterson, they say that
"A parameter is a numerical measure of a population characteristic." (p. 306)
"A statistic is a numerical measure ...
- 359
4
votes
0
answers
1k
views
Maximum Prediction in Gaussian Process
A Gaussian process (GP) is defined as a collection of random variables with a joint Gaussian distribution (Rasmussen 2006). It is well known that given observations $\left \{ \mathbf{x},\mathbf{y}\...
- 2,214
4
votes
0
answers
703
views
Expectation and variance of sample mean with random sample size
I have a question regarding sampling where the sample size itself is a random variable.
Say I have two sub-populations $A$ and $B$ from which I can sample a real valued random variable with ...
- 41
4
votes
0
answers
2k
views
Posterior distribution under Cauchy prior?
I have a (I hope) simple question! If I had a linear regression,
$Y_t = \alpha + \beta X_t + \epsilon_t$
with $\epsilon_t \sim N(0,\sigma^2)$
and I assume a Cauchy prior for $\sigma$, is it ...
- 153