A measure of the degree of linear association among a pair of variables.

learn more… | top users | synonyms

0
votes
1answer
13 views

Confidence interval for correlation

I know the confidence interval of an estimated correlation can be calculated after performing Fisher transformation. But I never figured out why. I also couldn't find the explanation online. Can ...
0
votes
0answers
26 views

Decomposing a random variable with random mean into a sum

I have two random variables: $X\sim \mathcal{N}(0,\sigma^2+1)$. $Z$, a gaussian with mean $X$, distributed so that $E_{X,Z}[(X-Z)^2]=s^2.$ We know that: $$s^2\geq\sigma^2+1 \Leftrightarrow Z ...
1
vote
1answer
47 views

Correlation matrix is not positive definite… But why?

As part of an analysis I am conducting (Structural Equation Modeling) the estimated correlation matrix among some variables ended up looking like this: ...
1
vote
1answer
27 views

Time Series Comparison - Correlation and Regression Model

I am trying to see if and how the news for affects the financial markets. I have a time-series for both of them. Should I standardise the series? I have a monthly return on prices from the Dow and a ...
1
vote
2answers
33 views

Evaluate correlation between two datasets

I created an algorithm that evaluates the user experience concerning web applications depending on response times and errors. That is, one of the following five states is calcualted: Very Happy, ...
0
votes
0answers
15 views

Measuring correlation or dependence between two data sets (x1,y1) vs (x2,y2)

Is there any statistical test or measure to evaluation the degree of correlation or dependence between two sets of data-points each point has (x,y)
4
votes
1answer
85 views

Why does Pearson's r have a non-normal sampling distribution at high values of ρ?

I read at http://davidmlane.com/hyperstat/A98696.html that [W]hen the absolute value of the correlation in the population is low (say less than about 0.4) then the sampling distribution of ...
0
votes
0answers
27 views

ols and multiple regression model wth two variables

My model is as follows: $y=b_1*X+b_2*\max(X,0)+u$ Will I have any problems with the variables $X$ and $\max(X,0)$, concerning any correlation issues? Can I just apply the classic OLS methodology?
0
votes
1answer
34 views

How can I correlate groups of variables?

I have conducted a survey to find out the correlation between organizational justice and turnover intention within an organization. So, in the survey, I have asked some Likert-scale questions around ...
0
votes
0answers
38 views

Remove column which has weak correlation between other

I have data set which contains 12 feature columns. The last one has Pearson correlations between 0.001 and 0.09. For Spearman and Kendall correlations are worth. Can improve the performance of ...
1
vote
2answers
48 views

Statistical Significant vs Correlation

In my regression one of the control variables has high statistical significance. But when I check the correlation coefficient between this variable and the dependent variable the correlation is almost ...
0
votes
1answer
15 views

Using the average as a cut off to group data and compare groups?

I measured cell size of human muscle and I wanted to examine the effect of age on the parameter. I graphed my data as a scatter plot and calculated Pearson's correlation coefficient but then also ...
0
votes
1answer
35 views

Apparent paradox about using a pilot study to determine sample size

I'm using some pilot data to determine the necessary sample size for an upcoming study. The primary goal of this study is to measure the correlation between two variables, A and B, and to determine ...
1
vote
0answers
41 views

Multiple Correlation Coefficient for Time Series?

I am using Pearsons correlation coefficient to calculate the correlation between two time series. Now I would like to calculate the correlation between a set of time series A, B, C and the time series ...
1
vote
0answers
25 views

Correlations in panel data

In a panel dataset of multiple years and multiple countries, what is best practice for running correlations among variables? I know that pooling by both countries and years is not recommended. Is ...
0
votes
0answers
35 views

Correlation with categorical variables - Interpretation of aov()

I want to know if one (or more) out of three categorical variables (season of measurement, geology, grazing) influence the numerical variable (spread of a plant). Sure, I read the answers here ...
0
votes
0answers
8 views

Comparing the value of Likert scale lengths

I am going over the results and interpretation of results created by a former employee in an medical education setting where I work. I am concerned that the comparisons that have been made between a ...
0
votes
1answer
25 views

How to account for correlation in a pre-existing linear model?

By using a pre-existing model we get weights $w_1, w_2 ... w_n$ assigned to $n$ variables. But the model does not take correlation between those $n$ variables into account. How can the correlation ...
0
votes
0answers
11 views

Correlation of scores, when most scores are the highest

Let's say we have 10,000 observations of two kinds of scores for each item, i.e. each observation has two scores reported. The scores are from 0 to 100. Within each kind of score 9,900 observations ...
1
vote
0answers
16 views

How does a discordance test work?

Could someone explain how the discordance test works (and what it actually is)? The explanations I found online are unclear: [...] the concordance test can be viewed as the evaluation of the ...
2
votes
1answer
54 views

Can $x'x$ be written as correlation matrix?

$x'x=$ $$ \begin{bmatrix} \sum_{i=1}^{n}(X_{1i}-\bar X_1)^2&\sum_{i=1}^{n}(X_{1i}-\bar X_1)(X_{2i}-\bar X_1)\cdots & \sum_{i=1}^{n}(X_{1i}-\bar X_1)(X_{ki}-\bar X_k) \\ ...
3
votes
0answers
75 views

sparse covariance/correlation thresholding

In our project, we would like to do some optimization on sparse matrices. The idea is to scrape massive amounts of data, form a covariance/correlation matrix, and form a sparsity pattern basically by ...
4
votes
3answers
107 views

Random variables have non-zero covariance but expected sample covariance is zero? (intuition)

This post asks "why a familiar and widely used estimator of sample covariance has expected value zero, in a situation where the variables involved are characterized by non-zero and equal pair-wise ...
0
votes
0answers
8 views

Appropriate correlation measure for ordinal vs. categorical data

I have the following data: ...
1
vote
0answers
14 views

Meta analysis; significance test of correlations

I have performed a meta analysis using the 'Metafor' package in R. I have some corrected correlations. Now I am wondering how I can calculate whether they are significant and whether there is a ...
4
votes
2answers
51 views

Truncating data reduces correlation?

Here is an argument I came across: by limiting your sample by some range of one of the variables the (Pearson) correlation coefficient between the two variables is likely to be reduced. I can't see ...
1
vote
0answers
19 views

Q-Methodology: which correlation coefficient to use: Pearson vs Spearman vs Kendall

Please note: This question pertains to Q Methodology, a research method used to study people's subjectivity. Q embodies ontological and epistemological assumptions that sometimes differ markedly ...
3
votes
0answers
53 views

When is r$^2$ not equal to $R^2$?

This blog post has a nice description of when the square of the Pearson correlation coefficient, r, is equal to the coefficient of determination, $R^2$. Specifically, states that they will be the same ...
2
votes
1answer
33 views

Correlation of aggregated data - is it valid

This is a common analysis situation which I've come across, but it makes me feel a little uneasy. Its finding the correlation of 2 variables after they have been aggregated. I haven't got any concrete ...
0
votes
0answers
9 views

Inference drawn from multiple correlation coefficient and partial correlation coefficient

I would like an explanation to why we need both types of correlation coefficients. Google helps me with the appropriate formula to calculate the coefficients so I do not think I need any help on that. ...
0
votes
0answers
15 views

Moving-window averages and spurious correlations

I am taking moving window averages of two variables and then correlating the new data. I was noticing extreme pseudo-cyclical changes in the correlation values with changing window size. I injected ...
1
vote
1answer
33 views

Calculation for the Test of the Difference Between Two Independent Correlation Coefficients

I have been told to take a closer look at: http://www.quantpsy.org/corrtest/corrtest.htm but how exactly does this work? E.g. Given two groups of data A and ...
0
votes
2answers
35 views

Ill-posed covariance matrix

I have a gene expression matrix $A$ with dimension $7000\times 30$ where $A_{ij}$ represents the expression of gene i at time j. I need to make a gene co-expression network but the problem is the ...
0
votes
0answers
21 views

Determining if data follows a Sigmoid function given data that does

I am essentially trying to determine if some gene expression data I have follows as circadian pattern. Currently I have gene expression data for genes know to follow a circadian pattern. I want to ...
0
votes
0answers
7 views

Testing for covariance on a fixed scale?

I'm struggling on finding a way to analyse my data, that may be fixed just with a basic statistics insight. I have a dataset that has a large number of variables (100+) some of which tend to covary ...
3
votes
1answer
40 views

Making two vectors uncorrelated in terms of Kendall Tau correlation

Assume that we have two normalized $n\times 1$ vectors $\bf x$ and $\bf y$. In terms of Pearson correlation, these two signals are uncorrelated if ${\bf x}^T {\bf y} = 0$. Now, assume that ${\bf x}^T ...
0
votes
0answers
18 views

Does Pearson correlation need mutual rated items in collaborative filtering?

I am working on collaborative recommender system and for computing user x user similarity I use Pearson correlation coefficient. My recommender system works with movie ratings. I need to make sure I ...
0
votes
0answers
14 views

How to account for correlation among all observations in my sample?

To illustrate, consider a group of students who take a math exam, but who also cheat during the exam. I wanted to run a regression of whether the students passed the exam (dichotomous dependent ...
2
votes
2answers
40 views

Does higher value of correlation between two values indicate it is good predictor? [duplicate]

Using Boston dataset in MASS library, I'm trying to find correlation of all columns against the column medv. For each column, I'm doing similar to below: ...
2
votes
2answers
104 views

How to compute Kendall tau for more than two variables

I am new to statistics. I decided to use Kendall tau correlation. I want to find correlations between every variable of the data set. (For example, if there are three variables in the data $x, y$ ...
1
vote
1answer
28 views

Where does the correlation come from in the regression coefficient equation for simple regression

In simple linear regression. $\beta = \frac{Cov(x,y)}{s_x^2}$. This is often written as $\beta = r_{xy}(\frac{s_y}{s_x})$ Where does the correlation come from in this equation? From my understanding ...
4
votes
3answers
219 views

In linear regression, what does $\beta_1 = 0$ really mean?

If granted omniscience and we know that $\beta_1$ in a multiple linear regression model is truly 0, what does that mean in words (and math notation)? The model is: $Y = \beta_0 + \beta_1X_1 + ...
0
votes
0answers
24 views

Correlation of Distance Matrix

I have a matrix with 15 samples and ~10,000 data points (all z-scores). I calculated a distance matrix with euclidean distances using R. Is it valid to calculate and present a correlation on this ...
4
votes
1answer
48 views

Generating independent random variables from correlated random variables

I have 2 standard normal, bivariate correlated random variables, $corr \ (X_1, X_2)=\rho$. I want to generate two independent standard normal random variables from these 2. I tried to use what I ...
0
votes
0answers
4 views

How to test whether correlation between Data X and Binary output Y? [duplicate]

I really don't know much about statistics,, and I'm trying to find a method for testing correlation between the independent variables X and the binary output Y.. So for example, lets say I have the ...
13
votes
3answers
443 views

How does the formula for generating correlated random variables work?

If we have 2 normal, uncorrelated random variables $X_1, X_2$ then we can create 2 correlated random variables with the formula $Y=\rho X_1+ \sqrt{1-\rho^2} X_2$ and then $Y$ will have a correlation ...
1
vote
2answers
39 views

How to find which variables are related to a given variable? Can one use PCA?

I have a table which contains about 80 variables. I will give one variable as input and as output I want to have all the variables that are related to the input variable. Example: I have hospital ...
0
votes
0answers
8 views

Correlation between two facets

I need to check psychometric propertis of one test. There are 160 variables in 7 scales/facets. I need to find out if there are any correlations between theese scales ( in SPSS). Can you help me ...
7
votes
2answers
108 views

Approaches for generating synthetic survey data with dependent answers?

I would like to produce synthetic survey data. At the moment I produce independent answers between questions according to an arbitrary discrete distribution as in this question. I want to generate ...
0
votes
0answers
7 views

similarities between two signal using correlation

let say we have two signal ,there is given their time domain plots and also correlation plot and final correlation plot it should be noted that both signal have same spectral structure,they ...