Questions tagged [normal-distribution]
The normal, or Gaussian, distribution has a density function that is a symmetrical bell-shaped curve. It is one of the most important distributions in statistics. Use the [normality] tag for asking about testing for normality.
1,562
questions with no upvoted or accepted answers
23
votes
0
answers
18k
views
When should I use the Normal distribution or the Uniform distribution when using Xavier initialization?
Xavier initialization seems to be used quite widely now to initialize connection weights in neural networks, especially deep ones (see What are good initial weights in a neural network?).
The ...
- 2,190
11
votes
1
answer
3k
views
Expected value of softmax transformation of Gaussian random vector
Let $\mathbf w_1,\mathbf w_2,\ldots,\mathbf w_n \in \mathbb R^p$ and $\mathbf v \in \mathbb R^n$ be fixed vectors, and $\mathbf x \sim \mathcal N_p(\boldsymbol{\mu}, \mathbf{\Sigma})$ be an $p$-...
- 548
10
votes
0
answers
3k
views
Fourier transform of a Gaussian process
I would like to discuss and ask a question regarding the Fourier transform of a Gaussian process, if it makes sense.
For that purpose, let me describe the following situation.
Let $z(s)$ be a ...
- 101
8
votes
0
answers
308
views
Simulate Gaussian variables conditional on their sum of squares
Consider a $d$-dimensional Gaussian random vector $\mathbf{Z}=[Z_i]_i$
with mean $\boldsymbol{\mu}$ and covariance matrix
$\boldsymbol{\Sigma}$. What would be the more efficient method(s) to
simulate $...
- 5,398
8
votes
0
answers
274
views
Confidence interval for a linear combination of $\mu$ and $\sigma$
Given $n$ independent observations $X_1,X_2,\ldots X_n\sim\mathcal{N}(\mu,\sigma^2)$, where both the variance and the mean are unknown. How can I write down a confidence interval for $\mu+c\sigma$, ...
- 81
8
votes
0
answers
1k
views
QR decomposition of normally distributed matrices
Assume $M$ is an $N \times k$ Gaussian matrix, i.e., its entries are i.i.d. standard normal random variables, with $N>>k$. Take $D=\text{diag}(\lambda_1, \dotsc ,\lambda_N)$ for some fixed real ...
- 181
8
votes
0
answers
118
views
Can these asymptotic conditional expectations be bounded from above?
Problem Setup
Let $\{X^d_1, X^d_2, \cdots, X^d_n\}$ be a $d-$dimensional zero-mean, i.i.d. random variables. Let $S_n^d$ be
$$
S^d_n = \frac{\sum_{i=1}^n X_i^d}{\sqrt{n}}
$$
Let $Y^d$ be a zero-...
- 591
8
votes
0
answers
3k
views
Assumptions of correlation test vs regression slope test (significance testing)
If my understanding is correct, then
the test on a regression slope in a simple bivariate regression - i.e. the test of $\mathcal{H}_0$: $b = 0$ in $Y' = a + bX$
and
the test of a correlation, i.e. $...
- 391
7
votes
1
answer
960
views
Conditional expectation in the multivariate normal distribution
Suppose $(X_1, X_2, X_3)^T$ is multivariate normal.
What is the conditional expectation $E(X_1 \mid X_2, I(X_3 > 0))$?
Here, $I(X_3>0)$ is the random variable that takes the value one when $...
- 4,541
7
votes
0
answers
1k
views
Square roots of sums absolute values of i.i.d. random variables with zero mean
In an earlier question, I asked about the limiting distribution of the square root of the absolute value of the sum of $n$ i.i.d. random variables each with finite non-zero mean $\mu$ and variance $\...
- 39.6k
7
votes
0
answers
236
views
why use diagonal $\Sigma$ when working with Bayes decision theory?
My prof. said in the class that for Bayes decision rule, the likelihood is Gaussian and in practice, we will almost always work with a diagonal $\Sigma$. Why is that? I know that a diagonal $\Sigma$ ...
- 235
7
votes
0
answers
244
views
How to derive the characteristic function of a polar coordinates representation of a bivariate normal
Suppose to have a bivariate normal variable $\mathbf{x}=(x_1,x_2)$ with mean $\mu$ and covariance matrix $\Sigma$. I move from $\mathbf{x}$ to $(\theta,r)$ where $x_1 = r \cos \theta$ and $x_2 = r \...
- 1,260
7
votes
0
answers
2k
views
How to compare models with different distributional assumptions for response variable in GLM?
Let's say I have measurements $Y$ which are all positive, and the distribution seems to be somewhat skewed. I'm modelling $Y$ in GLM framework. Now I could set my GLM using different distributional ...
- 136
7
votes
0
answers
965
views
Variance of marginal posterior distribution
Suppose $Y_1,\dots,Y_n\mid\mu,\sigma^2 \sim \text{ iid } N(\mu,\sigma^2)$ and suppose the priors $\mu \mid \sigma^2 \sim N(\mu_0, \sigma^2 / \kappa_0)$ and $1/\sigma^2 \sim \text{gamma}(\nu_0/2, \nu_0 ...
- 216
6
votes
0
answers
214
views
Why use the student's t-test rather than z-score?
Suppose we are given IID r.v's $X_1, \ldots, X_n$ that are not necessarily normally distributed. Mean $\mu$ and standard deviation $\sigma$ are unknown and we want to construct a confidence interval ...
- 39
6
votes
0
answers
2k
views
Distribution of multivariate "$Z$-score"?
Suppose $\mathbf{X}_1, \dots, \mathbf{X}_n \sim N_p(\mathbf{\mu}, \Sigma)$ where $\mu \in \mathbb{R}^p$ and $\Sigma$ is a $p \times p$ covariance matrix.
Suppose $\hat{\Sigma}$ is the sample ...
- 15.6k
6
votes
0
answers
97
views
Maximizing the information gain on a Gaussian RV with a noisy comparison question
The question
Let
$X \sim \mathcal{N}(0,1)$ be a random variable denoting the location of a target on the real line.
$Y_a$ be a binary random variable encoding the (noisy) answer to the question: "is ...
- 354
6
votes
0
answers
1k
views
Entropy of a mixture of Gaussians
I need to estimate as fast and accurately as possible the differential entropy of a mixture of $K$ multivariate Gaussians:
$$
\mathcal{H}[q] = -\sum_{k=1}^K w_k \int q_k(\textbf{x}) \log \left[\sum_{...
- 5,186
5
votes
0
answers
88
views
What's the distribution of $|y-z|^2/|y-\bar{y}|^2$ for vectors with i.i.d. standard normal coordinates?
Let $y_1, y_2, \ldots, y_n$ and $z_1, z_2, \ldots, z_n$ be samples of size $n$ of a normal distribution $\mathcal{N}(0,1)$. My goal is to find the distribution of
$$\frac{\sum_{i=1}^n (y_i - z_i)^2}{\...
- 151
5
votes
0
answers
73
views
The expected value of $\frac{1}{\sqrt{1-r}}$ where $r$ is Pearson correlation
I am looking to unbias the sample statistic $\frac{1}{\sqrt{1-r}}$ where $r$ is a Pearson correlation. The population is assumued binormal with equal variance $\sigma$ and with true correlation $\rho$....
- 249
5
votes
0
answers
2k
views
Wasserstein distance between Gaussian and the empirical distribution
Wasserstein distance between two gaussians has a well known closed form solution. Does the same hold for the distance between a Gaussian with a fixed variance(say 1) and the empirical data ...
- 51
5
votes
0
answers
424
views
Bayesian linear regression - posterior distribution
This is about bayesian linear regression. In this link http://fourier.eng.hmc.edu/e161/lectures/gaussianprocess/node2.html there's a derivation for
=
The part that I don't understand is how it is ...
- 81
5
votes
0
answers
4k
views
Detecting outliers in percentages
My dataset looks like below -
Total Success Percentage
100 65 65%
50 25 50%
30 20 66.6%
50 40 80%
Plot -
Each row is ...
- 51
5
votes
0
answers
78
views
Processes behind statistical distribution laws: a compendium?
The simple processes that "explain" the binomial, Gaussian or Poisson distribution are relatively well-known. Johnson or shot noises may be known in restricted area of science. Sometimes, a ...
5
votes
1
answer
127
views
Decomposing a locally stationary covariance matrix
Say I have a non-stationary Gaussian Process with a square exponential covariance whose shape varies throughout space. The covariance entries are:
$$
K_{ij} = N(|x_i-x_j|,\sigma_i^2+\sigma_j^2)
$$
...
- 1,002
5
votes
0
answers
381
views
Central Limit Theorem when the dimension size increases with the sample size
Let $X_1, X_2,\ldots, X_n \in \mathcal{R}^d$ and be zero-mean, unit variance random variables. Here the dimension ($d$) is a function of the sample size($n$) i.e, $d=f(n)$. For example $d = \sqrt{n}$. ...
- 591
5
votes
1
answer
3k
views
Probability that one random variable is larger than another with known correlation
Let's say I have a normally distributed random variable $X_1$ with known standard deviation $\sigma_1$ and $E[X_1]$ is $0$. Let's say I have another variable with known standard deviation $\sigma_2$ ...
- 553
5
votes
1
answer
4k
views
Orthogonal transformations of random vectors and statistical independence
In this old CV post, there is the statement
"(...) I have also shown the transformations to preserve the independence, as the transformation matrix is orthogonal."
It refers to the $k$-...
- 59.1k
5
votes
0
answers
226
views
Distribution of $X'X$ if $X\in\mathbb{R}^{T \times N}$ and $X_i'\sim N(\mu,\sigma^2I_N)$
Let $x_i\in\mathbb{R}^N$ be multivariate normal distributed with mean vector $\mu\in\mathbb{R}^N$ and no correlation: $x_i\sim N(\mu,\sigma^2 I_N)$.
Given $T$ iid samples, define the matrix
$$X:=\left(...
- 1,330
5
votes
0
answers
2k
views
Multivariate Normal Orthant Probability
For bivariate zero-mean normal distribution $P(x_1,x_2)$, the quadrant probability is defined as $P(x_1>0,x_2>0)$ or $P(x_1<0,x_2<0).$
$P(x_1>0,x_2>0) = \frac{1}{4}+\frac{sin^{-1}(\...
- 311
5
votes
0
answers
681
views
Confusion related to Parzen window
I was going through this tutorial related to Parzen window at http://www.cs.utah.edu/~suyash/Dissertation_html/node11.html. However, I have some confusion related to Parzen window with gaussian kernel
...
- 6,777
5
votes
1
answer
2k
views
Confidence interval about a multivariate normal probability
From a given sample $x_i \underset{\mathrm{iid}}{\sim} {\cal N}(\mu, \sigma^2)$ it is possible to get a confidence interval about the probability $\Pr(x_i \geq a)$ for any number $a$, by "inverting" ...
- 19.2k
4
votes
0
answers
72
views
Intuition/meaning of information geometry distances and geodesics?
In information geometry, we consider a manifold of probability distributions, together with the Fisher Information metric (given by the Fisher Information matrix).
I have some intuition (see ...
- 1,268
4
votes
0
answers
115
views
Estimating covariance based on Delta approach
I have a bivariate normal distribution $\left(X_1, X_2\right)$ with mean vector $\left(\mu_1, \mu_2\right)$ and some VCV matrix ...
- 373
4
votes
0
answers
337
views
What's the intuition behind the fact that sample mean and sample variance are independent when sampling from a normal population?
Let $X_1, \dotsc,X_n$ be i.i.d. from $N(\mu,\sigma^2)$, then we know that sample mean $\bar X\equiv \frac{1}{n}\sum_{i=1}^nX_i$ and $S^2=\frac{1}{n-1}(X_i-\bar X)^2$ are independent. Obviously, they ...
- 2,778
4
votes
0
answers
148
views
Unbiasing estimator of $\|\Sigma\|_F^2$
I have access to samples of some distribution with second-moment matrix $\Sigma=E[xx^T]$ and need an estimate of $\|\Sigma\|_F^2$ (which can be used to set optimal size for LMS)
We can use Frobenius ...
- 6,209
4
votes
0
answers
489
views
Can GMM approximate any given probability density function?
I am currently studying on Bayesian models, and still new to probability theory.
I learned that Gaussian Mixture Model is used to represent the distribution of given population as a weighted sum of ...
- 93
4
votes
0
answers
264
views
Understanding prediction interval for normal distribution
I try to learn prediction intervals for normal distribution , so i found this wikipedia source. Everything was okey until i came to the case where the population mean is known but population variance ...
4
votes
0
answers
133
views
Are there distributions for skewness and kurtosis? Similarly to mean (normal) and variance (chi-squared)
My question is really straightforward.
The distribution of the sample means approaches a normal distribution (CLT).
The distribution of the sample variance approaches a chi-square distribution (...
- 51
4
votes
0
answers
89
views
Expectation of x/√(x²+2px+1) under Normal distribution
I'm need to find (or at least approximate) as a function of $p$, the expectation under $x \sim Normal(0,1)$ of:
$$f(x) = \frac{x}{\sqrt{x^2+2px+1}},\hspace{1em}\textrm{where}-1<p<1$$
Wolfram ...
- 41
4
votes
0
answers
202
views
Fisher Information for frequency estimation under non-circular complex Gaussian noise
Consider a non-circular complex Gaussian noise $v$, which is given as $v = v_r + iv_i$. Here, real and imaginary components are independent and are Normal random variables such as $v_r \sim \mathcal{N}...
- 121
4
votes
1
answer
209
views
Are power law relations between means and standard deviations inherent in normally distributed data?
In a recent paper I submitted for publication I document a power law relation between the means and standard deviations of several time series. That is, when plotting the log of the means of each of ...
- 515
4
votes
0
answers
137
views
What's the probability the cumulative average of multiple gaussian variables to exceed a certain value?
Consider a number $n$ of independent, normally distributed variables $Y_1$, $Y_2$, ..., $Y_n$.
Consider also the $n$ variables defined by the average of the last $h$ terms, $X_h = \frac{\sum^{n}_{k = ...
- 91
4
votes
0
answers
228
views
Binary $Y$ but normal residuals?
In regression, it is abundantly clear that $Y$ can be non-normal while the residuals $\epsilon = Y - \beta_0 - \beta_1 X$ are normal. But can $Y$ be binary when the $\epsilon$ are normally distributed?...
- 5,891
4
votes
0
answers
3k
views
KL divergence between two multivariate Gaussians with close means and variances
KL divergence between two Gaussian distributions denoted by $\mathcal{N}(\mathbf \mu_1, \mathbf \Sigma_1)$ and $\mathcal{N}(\mathbf \mu_2, \mathbf \Sigma_2)$ is available in a closed form as:
$$\...
- 41
4
votes
0
answers
2k
views
KL divergence between samples from a unknown distribution and a Normal distribution with zero mean and unit variance
If you draw samples of unknown distribution, how can you measure the KL-divergence between the unknown distribution and a gaussian distribution with zero mean and unit variance N(0,1)?
Can we use ...
- 41
4
votes
0
answers
2k
views
Quantile transform vs Power transformation to get normal distribution
I was introduced to the concept of quantile-based gaussian transform. To my understanding, it changes the value of the original data by each percentile to the matching percentile of gaussian ...
- 1,041
4
votes
0
answers
903
views
Why was the letter z chosen in z-scores?
Why was the letter z chosen in the name of a z-score = (data value minus mean) / standard deviation and not any other letter?
This might be related to the same letter z being frequently used for the ...
- 41
4
votes
0
answers
56
views
Arbitrary manipulation of percentile given the mean and SD
Say I have math scores from a class of 50 students.
What I know is the mean midterm score is $50$, and the standard deviation is $30$.
So it is reasonable to assume that there is $50 * .16 = 8$ ...
- 1,523
4
votes
1
answer
143
views
Testing equivalence of two censored distributions
If we observe two censored distributions, where all observations above a cutoff are set at the value of the cutoff, how can we test whether the observable distributions suggest that the two censored ...
- 1,302