Questions tagged [covariance]
Covariance is a quantity used to measure the strength and direction of the linear relationship between two variables. The covariance is unscaled, & thus often difficult to interpret; when scaled by the variables' SDs, it becomes Pearson's correlation coefficient.
1,727
questions
2
votes
0
answers
20
views
Weighted least squares estimator variance using noisy weights
I have a linear system with uncorrelated, heteroscedastic noise, $Y \propto \mathcal{N}(Xβ,Σ)$ where $Σ$ is a diagonal matrix with elements $σ_{ii}^2$. The MLE is given by weighted least squares (WLS) ...
- 21
5
votes
1
answer
114
views
Is there anything interesting to be taken from the fact that $E[(X-E[X])(Y-E[X])] = E[(X-E[X])(Y-E[Y])]$?
While playing around with the formula for covariance, I discovered something I wasn't expecting. Replacing the $E[Y]$ in the definition of covariance with an $E[X]$ appears to simplify back down to ...
- 176
1
vote
0
answers
23
views
Weighted average of covariance matrixes
My issue is as follows:
In my model there are 4 different states, which each have a calculated probability of happening. I also have calculated covariance matrixes for my variables in each of these ...
- 11
1
vote
2
answers
145
views
What affects correlation in this situation?
Suppose a dataset I with 20 samples pairs of x and y collected from 20 different farms in State A.
Another dataset II with 20 samples pairs of x and y collected from 20 different farms in State A, B, ...
- 27
6
votes
2
answers
156
views
Is there a simpler proof than mine for this obvious proposition about correlations?
$\newcommand{\e}{\operatorname E}$"Obviously" if $g$ is a weakly increasing function and $X$ and $g(X)$ are both random variables with finite variance, then the covariance (and hence the ...
- 8,521
2
votes
0
answers
37
views
Expected Squared Error Derivation of Least Square Model (in The Elements of Statistical Learning Book)
I'm studying chapter 2.5 of The Elements of Statistical Learning by T.Hastie.
Here, they assume the ground truth relation between Y and X as
$$
Y=X^T\beta +\varepsilon,
$$
where $\varepsilon\sim \...
- 21
0
votes
0
answers
8
views
Is the spectrum of a signal circularly symmetric if the signal itself is circularly symmetric?
Let’s consider a signal that is circularly symmetric complex Gaussian process (proper):
$$ s \sim \mathcal{CGP}(0, C, 0) $$, and,
the covariance has the following form:
$$ \mathbf{C} = \mathbf{C}_{rr} ...
- 113
0
votes
0
answers
17
views
Confusion with Complex Gaussian process with Auto-covariance
I have a complex sequence $z(t)$ in time which I know to be a Gaussian process. I read that the complex Gaussian process is not only characterized by the covariance, but also the pseudo-covariance ...
- 113
3
votes
1
answer
220
views
Distribution of a sum of linear combinations of random variables, each drawn from a set of random variables
Question.
Let $X_1, X_2, ..., X_n$ be a set of normal random variables, each with variance ${\sigma }^2$ and mean 0. For each $i,j$ in pair in $X$, $Cov(X_i,X_j)=V$.
Further, let $Y_1, Y_2, ..., Y_m$ ...
1
vote
0
answers
23
views
Covariance of square root of random variables [closed]
Suppose I have the following expressions, where $X_1$,$X_2$,$Y$,$Z_1$,$Z_2$ are all random variables.
$$X_1^2 = B_1 Y + B_{Z_1} Z_1$$
$$X_2^2 = B_2 Y + B_{Z_2} Z_2$$
I'm interested in $Cov(X_1,X_2)$.
...
- 11
1
vote
0
answers
57
views
Ensemble mean of a fraction
I want to compute the ensemble mean of the term: $\frac{Y^2}{X}$
Both $X$ and $Y$ are random variables that are not independent. I want to compute $E[\frac{Y^2}{X}]$. I proceed as follows, (Using the ...
- 111
2
votes
1
answer
31
views
Covariance between two binomial random variables or expectation of product of binomial random variables
I have an empirical distribution $S_n(x)$ (= proportion of samples less than equal to x) from a random sample $X_1, X_2, ..., X_n$ for a random variable $X \sim F_X$. Consider the random variable $T_n(...
- 61
5
votes
2
answers
162
views
All uncorrelated marginals are independent: Only for joint Gaussian?
Let $X$ be a random vector in $\mathbb{R}^p$, where $p\geq 2$, with the following property: Any two uncorrelated marginals are independent. Formally:
(1) For any $\alpha,\beta\in \mathbb{R}^p$, if $...
- 213
0
votes
0
answers
30
views
Correlation of GLM coefficent when Y is correlated
First to motivate the problem, let me start with a solved example. Say I have some fixed design matrix $X$ and random vectors $Y_1, Y_2, \dots, Y_N$ which have a normal distribution. I can get the ...
- 97
1
vote
0
answers
29
views
Covariance between sample mean and sample standard deviation
I am trying to figure out the covariance between sample mean and sample standard deviation. The only related post I can find here is this: asymptotic covariance between mean and standard deviation and ...
7
votes
1
answer
359
views
Covariance between sample mean and sample variance
I am trying to figure out the covariance between sample mean and sample variance from a population. We DO NOT know whether the population is normal (if it's normal, then the covariance is zero between ...
0
votes
0
answers
11
views
What are the consequences of removing covariance of random effects in multi-level models for model reliability?
I am trying to asses the impact of cross-level effect on a dependent first-level variable. The problem I am facing is that the model returns singular fit. Some sources (https://www.learn-mlms.com/07-...
- 1
0
votes
0
answers
14
views
Estimate null hypothesis for correlation of linear combinations of variables?
Setting up the problem
Suppose I have a variable $x$ of length $n$ and I have another $p$ variables $y_1, y_2, \dots, y_p$, where $y_i$ is also of length $n$.
Based on the y's, I can make a linear ...
- 43
0
votes
0
answers
16
views
Covariance of functional data object
This is a follow-up question on my earlier question Functional Regression in R: Functional Response Regressed on Scalar Covariate.
I have a functional response $Y(t)$ (i.e., a stochastic process) ...
- 83
1
vote
1
answer
43
views
Deriving covariance of joint distributions of MVN [Linear Gaussian systems?]
Let $z$ ∈ R^L be an unknown vector of values, and $y$ ∈ R^D be some noisy measurement of z. We assume these variables are related by the following joint distribution
$p(z) \sim N(z|\mu_{z}, \Sigma_{z})...
- 111
0
votes
0
answers
53
views
Law of Total Covariance with Potential Outcomes
I'm attempting to derive the covariance between the sample mean for the treatment and control arm of a completely randomized experiment where the units in the experiment, $n$, are a random sample from ...
- 1,249
1
vote
1
answer
72
views
Upper bound for covariance of Hortvitz-Thompson Estimators
I need to bound on a covariance quantity that has come up in a sampling problem. $\widehat{Y}$ and $\widehat{T}$ are Horvitz-Thompson estimators of population totals, $Y=\sum_{i=1}^N y_i$ and $T=\sum_{...
- 21
3
votes
2
answers
319
views
Correlation of X/Z with Y vs X with YZ
Let’s say that $X$, $Y$ and $Z$ have some fat tailed distribution, $X$ and $Y$ are stock returns, i.e., the sample set can have both positive and negative values. While $Z$ is some random variable ...
- 183
0
votes
0
answers
15
views
How do I set up my covariates?
I am trying to design a fairly basic lme in Matlab for an experiment. I have been trying to wrap my head around covariates design and my advisor has been less than helpful.
I have various metric (...
- 1
1
vote
1
answer
65
views
van der Vaart Asymptotic Statistics, page 38, why does $e_\theta'=\operatorname{Cov}_{\theta}t(X)$?
On Page 38 of van der Vaart's Asymptotic Statistics (near the bottom of the page), it says
By differentiating $E_\theta t(X)$ under the expectation sign (which is justified by the lemma), we see that ...
- 2,516
6
votes
1
answer
183
views
Asymptotic efficiency of IQR
I was wondering about the asymptotic efficiency of the Interquartile Range (IQR) in the Gaussian case. I have calculated it empirically using a Monte Carlo estimator, and it appears to be equal to ...
- 61
0
votes
0
answers
13
views
covariance of coefficient estimates in a linear regression model
I am confused about how the above paper gets the correlation of the coefficient estimates $Cor^2(\hat{\gamma_j}, \hat{\gamma_k})$. The covariance matrix of the coefficient would be $(Z^TZ)^{-1}\sigma^...
- 23
0
votes
0
answers
16
views
Name for matrices with $M_{i,j} = M_{n-i+1, n-j+1}$ symmetry (eg. 2D autocorrelation)?
Covariance matrices are symmetric positive definite matrices. The 'Symmetric' part of this means that the strict upper elements are redundant with the strict lower elements, i.e $M_{i,j} = M_{j,i}$. ...
- 448
1
vote
1
answer
30
views
Variance of $X + \alpha^\top Y$ where $X$ is a scalar random variable and $Y$ is a random vector [duplicate]
Consider a scalar random variable $X\in\mathbb{R}$, a vector random variable $Y\in\mathbb{R}^n$ and a constant (non-random) vector $\alpha\in\mathbb{R}^n$. I want to compute
$$
\mathbb{V}[X + \alpha^\...
- 655
2
votes
2
answers
61
views
When calculating sample variance around a sample mean can I take degrees of freedom to be a fraction
Normally if you have $N$ samples of a random variable, from which you estimate both the sample mean and sample variance, you need to account for the missing degree of freedom in the sample variance:
$$...
- 313
0
votes
0
answers
38
views
Keep or remove insignificant covariance values when modifying CFA model?
I am working to improve goodness of fit in a CFA model and am wondering about if I should retain added covariance values that are insignificant. I am adding covariances that are indicated by ...
- 11
0
votes
0
answers
105
views
Seeking Appropriate Mixed Effects Model for Covariance Structure in Grouped, Repeated Measures Data
I'm currently working on a project where I need to identify an appropriate model that accurately fits the data generation process of my study. The goal is to have a model in which the covariance ...
- 561
0
votes
1
answer
72
views
Finding trend similarity between two different time series
I am very new to this data analytics field, so bit confused about what to do.
I have 2 time series datasets of two different products that have the same attributes. I want to find trend similarity ...
- 101
0
votes
0
answers
22
views
How is allowing residual covariances related to changing total score of a scale?
For what reason should I think about changing the total score of a scale if I allowed residual covariances for similar sounding items?
If I still work with all of the items should I think about ...
- 1
0
votes
0
answers
11
views
Rate of convergence of covariance of functional
Consider $X_n$ and $Y_n$ to random variable that are bounded in probability.
I know that
$$cov(X_n, Y_n) = O(n^{-1})$$
and that
$$(X_n, Y_n) \rightarrow_d (E_1, E_2)$$
where $E_1$ and $E_2$ are two ...
- 309
0
votes
0
answers
20
views
Is there an alternate estimator for a sample covariance matrix when n < p such that the estimator is not singular
Let's say I have $n$ samples which are vectors of length $p$. I know that the $p \times p$ sample covariance matrix is singular if $n \leq p$. Is there another estimator for the covariance that ...
- 97
1
vote
0
answers
14
views
Need to come up with an equation or method to calculate a ratio between two arrays of numbers
I have two arrays of data:
# of Files
Time to Process in seconds
1
8
2
20
3
31
4
76
What I'm wanting to do is come up with an estimate of how long it will take to process n number of files. I ...
1
vote
0
answers
16
views
Name of a ratio using a variance-covariance matrix of regression coefficients for main effects and interaction effects
I consider a linear model with an interaction term.
Y = b0 + b1X + b2Z + b3XZ
X, Z are the independent variables. b1 to b3 are the regression coefficients. The variances of b1 and b3 are s11 and s33, ...
- 33
1
vote
0
answers
36
views
Intersection of matern covariance functions with different parameters
I am trying to prove some ideas about the identifiability of covariance function parameters for small samples. From various numerical experiments, it seems that the graphs of two matern covariance ...
- 43
1
vote
0
answers
20
views
Geostatistics: Covariance vs Semivariance
I am confused by the following page in Geostatistics for
Environmental Scientists, Webster & Oliver:
My understanding
Given locations specified by a vector $\mathbf{x}$, we assume an underlying ...
- 111
1
vote
1
answer
136
views
Does zero variance imply zero covariance?
Consider a random variable $V$ with variance $\sigma_V^2$.
Since the covariance between a random variance and a constant is zero, I think, if $\sigma_V^2=0$, the covariance between $V$ and another ...
- 191
1
vote
1
answer
34
views
Prove covariance between sufficient statistic and logarithm of base measure in exponential family is equal to zero
Exponential family form is
$$f_X(x) = h(x)\exp(\eta(\theta)\cdot T(x) - A(\theta))$$
I know
$$\operatorname{Cov}(T(x), \log(h(x)) = 0.$$
But how can I prove it?
- 11
0
votes
0
answers
11
views
Exogeneity Check of Control Variables? Covariance
I have performed an exogeneity check on my control variables to assess their validity before incorporating them into my regression analysis. Upon conducting the analysis, I observed a significant ...
- 1
0
votes
1
answer
56
views
Covariance of two Random Variables
Suppose $r \geq 1$ distinct books are distributed at random among
$n \geq 3$ children.
(a) For each $j \in {0, 1, 2, . . . , r}$, compute the probability that
the first child gets exactly $j$ books.
(...
- 45
0
votes
0
answers
26
views
Variance and Covariance Expectations in 2-Level Model with Random slope & intercept
1. Could you provide an example of how to hypothesize the variance and covariance expectations for a multi-level model with random slopes and random intercepts?
I do not yet have access to my data, so ...
3
votes
1
answer
105
views
Finding covariance between internally studentized residuals
Consider a linear regression model $\boldsymbol y=X\boldsymbol\beta+\boldsymbol\varepsilon$, where $\boldsymbol y$ is an $n\times 1$ response vector, $X$ is an $n\times p$ matrix of covariates (fixed),...
- 10.6k
0
votes
0
answers
10
views
Scalar LMMSE based on N Observations
Supposing I have a Gaussian Random Variable X of zero mean and unit variance and N samples of the variable. On each sample these is also Noise W that is a Gaussian Random Variable with zero mean and ...
0
votes
0
answers
32
views
Rotation Matrix from Covariance of 3D point-cloud
I am trying to retreive rotation matrix from a rotated 3D point cloud covariance matrix, using SVD decomposition (as done in SimNet and MVTrans). Here how I computed the covariance matrix from 3D ...
- 1
0
votes
0
answers
28
views
Bessel correction for the variance of dependent sample
Assuming a sample $X_1, X_2, ..., X_n$, the sample variance is calculated as
$s^2 = \frac{1}{n-1} \sum (X_i-\bar{X})^2$
The fact that there is $n-1$ in the denominator instead of $n$ is called the ...
- 217
0
votes
0
answers
26
views
Equivalence between two expressions for autocorrelation
Have that
$$
\text{Corr}(X_t,X_{t+h}) = \frac{\text{Cov}(X_t,X_{t+h})}{\sqrt{\text{Var}(X_t)\text{Var}(X_{t+h})}}
$$
and
$$
\rho(h) = \frac{\gamma_X(h)}{\gamma_X(0)}.
$$
Those are both ways to express ...
- 207