Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Covariance is a quantity used to measure the strength and direction of the linear relationship between two variables. The covariance is unscaled, & thus often difficult to interpret; when scaled by the variables' SDs, it becomes Pearson's correlation coefficient.

3
votes
0answers
46 views

Maximum likelihood: Why is the number of non-zero eigenvalues equal to $x^T \hat{\Sigma}^{-1} x$

I've been reading this code (based on this R package) and I found that the number of non-zero eigenvalues of the estimated covariance is roughly equal to $x_i^T \hat{\Sigma}^{-1} x_i$. I want to know ...
3
votes
0answers
48 views

Confusion in terminologies for simple linear regression model [on hold]

When I try to understand big picture between regression and correlation, the terminologies get in the way, because of different conventions used in statistic books and online materials and also due to ...
1
vote
1answer
32 views

Variance of $Y|x$ from regression line

Using simple linear regression model, and sample correlation coefficient $r$, for a sample set $X,Y$, the true regression line could be given as below. $$ \hat{Y}|x = \overline{y} + r\dfrac{s_Y}{...
2
votes
0answers
24 views

Proving covariance equals zero given a specific conditional expectation

I'm trying to prove the following: Given $𝐸[𝑋|𝑌 = 𝛽] = 𝐸[𝑋]$ for any value of $\beta$, prove that $\operatorname{Cov}(𝑋,𝑌) = 0$; So I was thinking to start with the definition of $\...
0
votes
0answers
8 views

Is this definition of partial covariance correct?

I have been looking for a proper measure-theoretical definition of partial correlation. Let $X_1, X_2, Y_1, ..., Y_n$ be Random Variables. Then the partial covariance between $X_1$ and $X_2$ given $Y$ ...
0
votes
0answers
15 views

user friendly interpretation of linear model covariance output

Given the following scenario: I have 3 parameters used in a linear model to estimate the 4th. all three are positively and significantly correlated to the predicted parameter slope (~0.2-0.4) (p<0....
0
votes
0answers
65 views

Is my Correlation reasoning correct?

I am trying to understand how to arrive at $r = \dfrac{Cov(X,Y)}{\sigma_X\sigma_Y}$ with a logical narrative. This in fact is kind of continuation from my this unanswered question. I see that by ...
1
vote
0answers
47 views

How to derive correlation using regression without empirical proof?

I just finished learning MLE, Regression, Covariance and now in to Correlation.I want to transform logically from Regression to Correlation using Covariance. Regression: A simple regression model ...
0
votes
0answers
8 views

Correlation 2D vector fields

Having multiple (hundreds) of 2D flow maps, ie vector fields how would one find statistical correlation between these? Plotting yields, for visualization purposes only: I am thinking about comparing ...
2
votes
1answer
30 views

Interpreting a graphed covariance function

I'm looking through a slide deck (slide 9) about Gaussian Processes, and I came to a slide that describes one example of a covariance function: Matérn $\frac{3}{2}$ Covariance. $$C(x_1,x_2) = (1+\...
0
votes
1answer
27 views

How to find the marginal densities of the given functions

The fraction $X$ of male runners and the fraction $Y$ of female runners who compete in marathon races are described by the joint density function$$f(x,y) = \begin{cases} 8xy & 0 \le x \le y \le1 ...
2
votes
1answer
41 views

Linear regression with a given covariance structure in R

I want to fit a linear model in R with a given covariance structure: $$y=X\beta+\epsilon$$ where the covariance matrix of $\epsilon$ is block diagonal by a grouping factor. Suppose there are $B$ ...
0
votes
1answer
37 views

Confusion with covariance

For distributions of random variables X and Y, their covariance can be defined as the difference between the multiplication of X and Y, normalized by their joint probability and the multiplication of ...
0
votes
0answers
24 views

GP posterior covariance between f(x) and f(x')

I am working through the examples in the Rasmussen's Gaussian Processes, specifically the GP regression figure attached right -- I can't seem to get the the posterior covariance between f(x) and f(x') ...
0
votes
0answers
13 views

residualized covariance matrix from pca/eigenvalue decomposition

I understand that given N dimensional data you can use PCA to construct an N dimensional orthonormal basis that explains 100% of the variance of the original data. However, you can also construct ...
0
votes
1answer
17 views

Mean of two time series

I'm trying to estimate the covariance of two time series using the formula where X and Y are two time series. I don't ...
0
votes
0answers
10 views

Understanding MCD

I have recently stumpled upon the robust MCD (Minimal Covariance Determinant) Estimator. If I have $n$ datapoints of dimension $p$. Let`s say we want to obtain a robust estimate for the Covariance ...
0
votes
1answer
26 views

Visualize Covariance when only probability mass and marginal functions are given

I am trying to intuitively understand Covariance like here. So if a random sample set given, I could draw rectangles with them, one of the cornes being fixated on mean $(\overline{x},\overline{y})$. ...
0
votes
1answer
28 views

Covariance between Linear Combinations of random vectors

Given a random vector $x\sim N(0, \Sigma)$ of dimension $p$ and matrices $A$ and $B$ (both $m\times p$, what is $Cov(Ax, Bx)$? It seems to me that the covariance should be $A\Sigma B^T$ but I am ...
0
votes
0answers
48 views

Covariance variation in different directions

I was watching a lecture on Gaussian Process and when the covariance matrix was introduced, the tutor explained that the matrix is $(n \times n)$ because every point is covered twice - we include the ...
0
votes
0answers
13 views

Proof that Newey West standard error estimator is positive semi defnite

I was going through the Newey West (1987) paper: "A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix" and was trying to understand the proof that ...
0
votes
0answers
19 views

How to calculate change in covariance following addition of random noise

I have a dataset consisting of two variables, X and Y. The covariance of the dataset = 12.5. Following addition of specified random noise to each observation of X and Y, the covariance decreases to 12....
0
votes
1answer
26 views

Why are variances of observed variables estimated in Factor Analysis / SEM?

Let's say we have one latent factor L and three observed variables X1, X2 and X3. From the variance/covariance matrix of X1, X2, X3, we create this latent factor that explains the most out of the ...
0
votes
0answers
31 views

If $|r| < 1$, how extreme is $y$ for a change in $x$?

This is regarding one of problem sets in Udacity Intro to Statistics course. Problem: X and Y are given variables, and the correlation coefficient $|r| < 1$, the question is how extreme our ...
0
votes
0answers
17 views

Covariance of the inner product of a random vector and a constant vector?

Let $c$ be a constant $n$-dimensional vector. Let $Y$ be a $n$-dimensional random vector. What is: cov($c^\top Y$)? I do not understand this since surely $c^\top Y$ will return a $1 \times 1$ ...
2
votes
0answers
24 views

Why is the sum of two variances is larger than two times the covariances?

I saw the following inequality: $$ \sigma_{X_1}^2 + \sigma_{X_2}^2 \geq 2\sigma_{X_1X_2} $$ and the brief explanation said that it is based on Cauchy-Schwarz inequality. But I couldn't make the ...
0
votes
0answers
33 views

Covariance of a nxm matrix with n<m is not positive semidefinite?

Is it true that, for a matrix $A \in \mathbb{R}^{n \times m}$ with $n<m$ (so with more features than samples), its covariance matrix is (or might be?) not positive semidefinite? If that's the case, ...
0
votes
1answer
50 views

Dependence of estimator covariance on sample count

Say that $X$ is a set $\{X_1, X_2, ..., X_N\}$ of (non-independent) random variables, and that $\hat{\mu}$ is a set $\{\hat{\mu}_1, \hat{\mu}_2, ..., \hat{\mu}_N\}$ of estimators. Each $\hat{\mu}_i$ ...
1
vote
1answer
38 views

analyzing residuals vs. fitting a full model

In my field, some scientists look for relationships between a dependent variable, y and a covariate, x1, while controlling for a ...
0
votes
2answers
43 views

ANOVA multicollinearity adjustment

I am using the statsmodel.ols module to compute an omnibus (ANOVA) F-test for three within-subjects factors; 2*3*2 levels design. The Cond. No. of the omnibus test (...
0
votes
1answer
26 views

How parameters formulated for Simple Regression Model

I am reading Simple Regression Model from this book, Section 6.5 (page 267 in downloaded pdf, 276 if viewed online). The author starts with below equation for a simple linear regression model, $$...
0
votes
0answers
36 views

Robust (Newey West) beta, covariance and variance

Let's start with a simple linear model $$Y_t = β_0 + β_1*X_t + ϵ_t$$ If $ϵ_t$ is serially correlated and heteroskedastic, I can calculate N-W standard error of $β_1$. I have also read in the ...
2
votes
2answers
47 views

Does $Cov(X,Y)=0$ imply that the sample covariance between realizations of $X$ and $Y$ is always zero?

For instance, in linear regression, we have that $$ Cov(e,\hat Y) = 0 $$ That is, the residuals and fitted values are uncorrelated. Is this always true in any sample realization of residuals and ...
1
vote
1answer
23 views

Population autocovariance goes to zero, assuming covariance stationary

In time series context, let $\gamma_j=E[(y_t-\mu)(y_{t-j}-\mu)]$ denote population autocovariance, where $\mu$ is population mean of $y_t$, assuming covariance-stationary. Then, $\gamma_j$ goes to $0$ ...
1
vote
0answers
22 views

Finding three coefficients (weight/ratio) by minimising variance

I am finding the value of a,b and c for minimising the variance of the following equation (four variables are correlated): $$ Var(\Delta V)=V ar[\Delta S-a\Delta F_1 - b\Delta F_2 -c\Delta F_3] $$ ...
1
vote
1answer
29 views

given $Z_i ~ N(0,1)$ and $Z^2 = Z_1^2 + Z_2^2$ what is $Cov(Z^2,Z_1)$?

Currently I am at the stage where: $Cov(Z^2,Z_1) = E(Z^2*Z_1) - E(Z^2)E(Z_1)$ $=E(Z^2*Z_1)$, because expectations of normal, or combination of normal variables is zero. After this I have no idea ...
0
votes
2answers
50 views

Calculating covariance and ICC in mixed models?

I'm a bit confused on how to start calculating by hand the covariance and intraclass correlations for mixed effects models. For example, in the particular example below: $$ y_{ijk} = \beta'\...
0
votes
0answers
16 views

Confusion about asymptotic properties of Yule-Walker estimator

The Yule-Walker estimator of an AR(1) process has some well-known asymptotic properties ... except there are TWO results governing these properties. I am not sure which one to use? The first result ...
2
votes
2answers
58 views

Outlier and correlation

Hi, I have a question. The scatter plot doesn't show any type of correlation and there is an outlier. If the outlier was to be removed, would the correlation: Increase dramatically Increase ...
0
votes
0answers
12 views

Showing the 2nd Order Properties of 2 ARMA Processes are Identical

Given 2 processes $$ Z_t = \epsilon_t + \theta\epsilon_{t-1} $$ $$ Z_t' = \epsilon_t' + \theta^{-1}\epsilon_{t-1}' $$ where $$ \epsilon_t \overset{iid}{\sim}\mathcal{N}(0, \sigma^2) $$ $$ \epsilon_t' ...
6
votes
2answers
113 views

Can an optimal weighted average ever have negative weights?

I've got a few measurements $\vec{x}$ for some real-world value $\hat{x}$. These measurements have some uncertainty, and are correlated. Given these estimates, and their covariances, I want to take ...
0
votes
0answers
7 views

Determining Polynomial Trend, Stationarity, and Covariance of a Process

I'm given $$ Y_t = p(t) + \epsilon_t $$ where $\epsilon_t$ is a stationary series with covariance $\gamma_t$. Also given $$ p(t) = \sum_{r=0}^kK_rt^r $$ where the $K$s are constants, for the ...
2
votes
1answer
17 views

Prove positivity of chi-squared statistic with general covariance matrix

For a "simple" chi-squared test statistic $\chi^2 = \sum_i (x_i - \mu_i)^2 / \sigma^2$, it's clear that the domain is positive since both the numerator and denominator of every term in the sum over ...
0
votes
0answers
40 views

Taking M samples with replacement from N options, what is the covariance?

Suppose that you take $M$ samples with replacement from numbers $\{1,2,..., N\}$. Denote the number of time each number is sampled by $\{K_1, ..., K_N\}$. Is it possible to say something about the ...
2
votes
1answer
53 views

Showing the covariance and autocorrelation functions of a stationary time series are symmetric around 0

I need to show that the covariance and autocorrelation functions of a stationary time series are symmetric around zero. From my understanding, this entails $$ \gamma(h) = \gamma(-h) $$ $$ \rho(h) = \...
0
votes
2answers
32 views

Covariance of random variables whose sum is less than a constant

Suppose that we have integer random variables $X>0$ and $Y>0$ and constant number $a$. We have: $X+Y < a$. Can we say that the covariance of these random variables is less than or equal to ...
1
vote
0answers
35 views

Is the covariance matrix a diagonal matrix with variances on the diagonals?

I am a geophysicist learning about geophysical inverse problems. In many papers, the authors discuss the "covariance matrix" as it applies to the inverse problem. In most geophysical applications, ...
0
votes
0answers
16 views

A non-ANCOVA situation for correcting slope heterogeneity with a covariate

I have a statistics problem than I have spent hours trying to tackle with no success. I know what I want to test, I just can't figure out what the method is. I am looking at the relationship of ...
1
vote
1answer
22 views

If one dimension of the data is scaled by a factor, how would it affect the probability of the Gaussian distribution?

I have fitted a maximum likelihood Gaussian distribution $N(\mu, \Sigma)$ on a multidimensional data set $X$. I wonder how would $p(X)$ change if one dimension of $X$ is scaled by a factor? It's ...
1
vote
1answer
75 views

Why is distance covariance defined squared, while covariance is not?

I am dealing in a data science project with correlation analyses using pearson and distance correlation. While trying to understand the differences between them, I learned about the differences by ...