Linked Questions
25 questions linked to/from Why shouldn't the denominator of the covariance estimator be n-2 rather than n-1?
2
votes
0
answers
2k
views
Degrees of freedom in covariance calculation [duplicate]
When calculating the sample covariance, why do we divide by $n-1$ instead of $n-2$? Don't we lose two degrees of freedom since we need to calculate two sample means? For example, when estimating the ...
- 249
4
votes
0
answers
74
views
Sample Covariance [duplicate]
The sample covariance is defined as $\hat{\sigma}_{xy}:=\frac1{n-1} \sum_{i=1}^n (x_i -\bar{x})(y_i-\bar{y})$. What is the intuition for using the correction term $n-1$ instead of $n-2$. Because we ...
- 363
1
vote
0
answers
64
views
Understanding "we lose degrees of freedom in product deviations"? [duplicate]
I'm having a tricky time understanding the statement:
we lose $n−1$ [degrees of freedom] by computing the product
deviations [in $cov(x,y)$]
I'm not necessarily looking for an overview of ...
- 21
2
votes
0
answers
29
views
Why does computing product derivations eliminate degrees of freedom? [duplicate]
I'm trying to get an intuition as to why degrees of freedom is n - 1 when calculating sample covariance. This is what I found:
1) Start with 2n degrees of freedom from the bivariate data
2) Lose 2 ...
- 121
30
votes
10
answers
4k
views
Sample two numbers from 1 to 10; maximize the expected product
Assume you sample two numbers, randomly drawn from 1 to 10; you could choose two strategies: 1) pick with replacement and 2) pick without replacement. Which strategy would you prefer to maximize the ...
- 363
9
votes
2
answers
12k
views
Covariance of two sample means
I am trying to derive the covariance of two sample means and get confused at one point. Given is a sample of size $n$ with paired dependent observations $x_i$ and $y_i$ as realizations of RVs $X$ and $...
- 6,724
7
votes
2
answers
7k
views
Variance and covariance in the context of deterministic variables
Questions:
Can we talk about:
variance of a deterministic variable?;
covariance between a deterministic variable and a stochastic variable?;
covariance between two deterministic variables?
Are these ...
- 69.5k
9
votes
3
answers
1k
views
Why is Covariance Useful?
There are a number of topics related to covariance on this site. What I am having trouble grasping: why is covariance a useful thing to calculate?
As far as I see it, covariance is not a helpful ...
- 185
7
votes
2
answers
9k
views
Variance of the sum of random vectors
Let $X_1, X_2, \dots , X_n \sim G$ where $G$ is some distribution and the samples are not independent. If $X_i \in \mathbb{R}$, then I know that
$$\text{Var}\left(\sum_{i=1}^{n} X_i \right) = n\text{...
- 16k
8
votes
1
answer
7k
views
Efficient way to compute distances between centroids from distance matrix
Let us have square symmetric matrix of squared euclidean distances $\bf D$ between $n$ points and vector lengthed $n$ indicating cluster or group membership ($k$ clusters) of the points; a cluster may ...
- 58.8k
2
votes
2
answers
2k
views
When are OLS linear regression parameters inaccurate?
Q1: Show quantitatively that OLS regression can be applied inconsistently for linear parameters estimation.
OLS in y returns a minimum error regression line for estimating y-values given a fixed x-...
- 13.3k
2
votes
2
answers
2k
views
Standard Deviation is to variance as ____ is to covariance?
I'm looking at the bivariate relationships among a set of time-series data that have the same units. I have computed the covariance matrix and am working on interpreting it. My hunch is that it would ...
- 83
1
vote
1
answer
2k
views
formula for sample covariance: Bessel's correction
Two options for the sample covariance between X and Y:
1)(with Bessel's) COV(X,Y) = $1/(n-1)$ * $\Sigma$ $(Xi - mean(X))$*$(Yi - mean(Y))$
2)(without) COV(X,Y) = $1/n$ * $\Sigma$ $(Xi - mean(X))$*$(...
- 33
4
votes
1
answer
1k
views
Uncorrected sample standard deviation in correlation coefficient
There seems to be a common use of the uncorrected sample standard deviation in calculating the correlation coefficient:
https://www.experts-exchange.com/articles/2728/Covariance-and-Correlation-in-MS-...
- 411
3
votes
1
answer
2k
views
Dividing by degrees of freedom [duplicate]
When estimating parameters such as (I don't care about this specific instance particularly) Variance of a random variable X, one usually adopts Bessel's correction, i.e. using the formula $\hat{Var}{(...
- 141
1
vote
0
answers
939
views
Why absolute value of eigenvalues are used in PCA or LDA?
In PCA and LDA techniques, eigenvectors with the $k$ largest eigenvalues give principal components. However, when selecting these eigenvalues, are they to be sorted by the absolute value (regardless ...
- 11
2
votes
1
answer
195
views
How is $\text{Cov}(\bar{Y}, Y_i - \bar{Y}) = \dfrac{1}{n^2} \text{Cov} \left( \sum_{j = 1}^n Y_j, nY_i - \sum_{j = 1}^n Y_j \right)$?
I have this example of sufficiency:
Let $Y_1, \dots, Y_n$ be i.i.d. $N(\mu, \sigma^2)$. Note that $\sum_{i = 1}^n (y_i - \mu)^2 = \sum_{i = 1}^n (y_i - \bar{y})^2 + n(\bar{y} - \mu)^2$. Hence
$$\...
- 2,204
2
votes
3
answers
132
views
Rationale for elliptical region of correlations in gene expression data
I am analysing an RNA seq data set and I am trying to look at correlation between expression values of significant genes in 4 different biological duplicates and their clinical parameters.
Here, I ...
2
votes
0
answers
351
views
Law of total covariance with two conditioning variables
How to decompose the covariance with two conditioning random variables?
For example, there is a law of total variance with two conditioning variables in the Wikipedia
$$
\text{Var}(Y)=\text{E}[\text{...
- 353
2
votes
0
answers
324
views
how to find asymptotic joint distribution of two linear combination of order statistics?
Suppose I have n order statistics from some unknown continuous distribution funciton F(x), $X_{1}\leqslant X_{2}\leqslant...\leqslant X_{n}$. And I have two linear combination of these order ...
- 290
3
votes
0
answers
286
views
Unbiased estimator variance of sample variance
I was reading the section on k-statistics on wolfram alpha. It was known to me that for the sample variance
$k_2 = \frac{1}{n-1}\sum_{i=1}^n (x_i - \overline{x})^2$
it holds that its variance ...
- 71
2
votes
0
answers
242
views
Is the covariance between the product of two variables and one of the variables zero?
For two centered (zero expectation) random variables $X$ and $Z$ I am interested in the covariance of the product $XZ$ and either $X$ or $Z$.
$$Cov(X,XZ) = E( X(XZ - E(XZ))) = E(X^2Z)$$
I think the ...
- 6,724
1
vote
0
answers
215
views
Distribution of the average of multivariate normals?
I have seen that the sum of $n$ iid multivariate normal vectors (mean $\mu$ and variance $\Sigma$), $X_1+\dots+X_n$, is distributed as a normal with mean $n\mu$ and variance $n\Sigma$. Is the ...
- 11
1
vote
0
answers
52
views
The covariance of a data matrix
Please let me know if the below statement is valid or not ;
Suppose that $X$ is an $n\times p$ data matrix with $p$ features and $n$ data samples.
Suppose further that each feature(column) is zero ...
- 11
0
votes
0
answers
34
views
Calculate multivariate betas using correlations and standard deviations [duplicate]
In a simple regression context:
$$
y = \alpha + \beta x + e
$$
We can estimate beta from:
$$
\hat{\beta} = \frac{cov(x,y)}{var(x)} = \rho_{xy} \frac{\sigma_y}{\sigma_x}
$$
This last decomposition is ...
- 106