Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Make your voice heard. Take the 2019 Developer Survey now

Questions tagged [correlation]

A measure of the degree of linear association among a pair of variables.

0
votes
0answers
14 views

Correlation coefficient [on hold]

Is it possible to run Pearson correlation on the following two variables? 1. Teachers' metalinguistic awareness (as the teachers' scores from 0 to 30 on a 30-item test) 2. Teacher efficacy (as an ...
1
vote
1answer
29 views

Variance of sum of dependent random variables

Can you guys help me prove the following: $$ Var[\frac{1}{m}\sum_{i=1}^my_i]=\frac{1}{m}(1-\rho)\sigma^2+\rho\sigma^2 $$ where the sampled predictions ($y_is$) have variance $\sigma^2$ and ...
0
votes
0answers
10 views

Multivariate Conditional Entropy as a test of correlation between random variables

I use the word columns to mean the data from which a random variable can be estimated. It is a sample of a random variable. I am working with $N$ columns of weakly correlated data. Furthermore, I ...
0
votes
0answers
31 views

How to determine the right answers from different sources [on hold]

Situation: There are n number of questions Q1..Qn Each question has one and only one correct answer (unknown to me, and when the answer is B, Z is as wrong as C is) I have some answers to those ...
0
votes
0answers
7 views

Ordinary bootstrap preserve rank correlation?

My question is simple: will an ordinary bootstrap preserve rank correlation on the vectors? I.e. I have an m-by-n matrix. M samples of N variables. If I bootstrap, picking M samples uniformly again ...
1
vote
0answers
15 views

Why my correlations are different when they calculated from latent variables (CFA), than when they are calculated from composite scores?

When I am doing a statistical analysis, first I take the measures of each of the constructs I am measuring and I create an average score. With these averages, I calculate the correlation between ...
0
votes
0answers
7 views

How is correlation among repeated measures calculated for ANOVA interaction effect?

I am performing a replication study, of an interaction effect found in repeated measures ANOVA. The study is of a 2X2 design (1 group of subjects, 4 measurements each). To estimate the required ...
0
votes
0answers
12 views

Python: How to plot 2 categorical variables? [on hold]

For example, I have data like this For each data point, it has a categorical feature a in [cat1, cat2, cat3] and a categorical feature ...
0
votes
0answers
10 views

Maximal Information Coefficient (MIC) in practical bioinformatics applications

I am interested in Maximal Information Coefficient (MIC) as an alternative to Pearson correlation when looking at gene co-expression from microarray data. I've read some very good posts on this ...
0
votes
1answer
15 views

Covariance of sums of pairs of correlated variables

Take two vectors of normally-distributed random variables $\mathbf{x} = (x_1, x_2, \ldots x_n)$ $\mathbf{y} = (y_1, y_2, \ldots y_n)$ where the covariance of each pair $(x_i, y_i)$ is known, $\...
1
vote
1answer
64 views

Correlation between two time series - volume and proportion

My company makes widgets. We take a random sample of these widgets every month and count how many are good and bad. Some people think that we are worse at making widgets in the months where we make ...
0
votes
1answer
26 views

What test for a 4-level categorical variable and a continuous variable?

I'm currently doing research on whether image quality affects engagement. Image quality is given in 4 categories (bad, benchmark, semi-professional and commercial level) and engagement is in how many ...
1
vote
2answers
49 views

Do these standardized residuals show heteroskedasticity?

I'm practising in the individuation of heteroskedasticity from the standardized residuals. I know that, if the time series is homoskedastic, the spread of the residuals should be constant and random ...
1
vote
0answers
16 views

Is there a correlation coefficent for “smooth” functions?

There is Pearson's, which measures linear relationships. There is also Spearman's, which can detect monotonic relationships. I am wondering if there is a similar coefficient someone has come up with ...
0
votes
1answer
10 views

correlation with small range of data

I have two count variables. I want to calculate a Pearson correlation between them. However one of the variables only has a rangge of 1-7. Is it inadvisable to use Pearson correlation to study a ...
0
votes
0answers
13 views

Detect Negative Correlation [closed]

In the attached image, if I want to detect negative correlation between these two data sets using machine learning algorithms, which one do I choose? I have indicated the boundaries I first need to ...
0
votes
0answers
21 views

Corelation between two dichotomous variables

I have a data from a research with responses (yes/no) about three questions (A,B,C) and I would to see if the variables are correlated. First of all i compared the proportion of agreement(yes-yes, no-...
0
votes
0answers
12 views

Comparing two groups of time series correlation coefficients?

I have two groups of data, each consisting of 100 correlation coefficients. I am interested in essentially seeing if the correlations in group 1 are stronger than group 2. However, I am unsure of the ...
0
votes
0answers
11 views

Speed up Conditional Variable Importance for Random Forests

I have trained a random forest in R and now I'm calculating the variable importance mesaure unsing the party Package. ...
0
votes
0answers
14 views

What statistical to use - variability between but not within

I have a data set where each line represents a participant's response to a series of questions (e.g., attitudes towards democracy, life satisfaction). Participants were recruited from different ...
-1
votes
0answers
9 views

the influence of supply chain management on customer satisfaction [closed]

I was asked to do a pilot test before giving away the questioner among my customer. my tutor asked me to select ten customers and get do a pilot test and know the alpha value. please let me know how ...
-1
votes
0answers
15 views

How to create correlation matrix [closed]

My project work is on microbiome studies. Here we have OTU table of different persons.How can we create correlation matrix from OTU table?
0
votes
0answers
21 views

Dealing with correlated variables and choosing between models

One of the things I've always been confused about is the framework around model selection in the cases where $n$ predictor variables $x_i$'s are correlated. My thoughts on the different approaches: ...
1
vote
2answers
32 views

Why is correlation not used while splitting in Decision Trees?

Say we have data where we have temperature, humidity, wind and rain for deciding whether we go and play tennis or not. While selecting best feature to split upon initially, we use ID3 or Gini index. ...
0
votes
0answers
6 views

Correlating with average of independent but values of dependent variable?

I want to correlate reef health (independent variable) with fish biomass (dependent variable). I have six location and at each location I determined reef health only once, but fish biomass was ...
0
votes
1answer
27 views

How to evaluate if year-over-year data is correlated and predictive

I'm working with data that is separated by year and classification and want to test if the data is correlated over each year. Essentially, I want to determine if the data is random each year or one ...
0
votes
0answers
3 views

Pair Correlations in R with mixed types in Data Frames [migrated]

If my data frame has mixed types in it (continuous and categorical), and I want to compute pair correlations using, for instance, pairs in R, is there a way to ...
1
vote
0answers
26 views

What is the meaning of correlations between regression coefficients?

I understand that coefficient correlations can arise in the presence of correlated covariates, essentially indicating that our inference for those parameters is coming from information that cannot be ...
1
vote
0answers
8 views

Oversampling to Create Correlated Feature from Existing Feature

For demo purposes in Python, I would like to create an array or pandas df column that shares a high degree of correlation with an existing array. This is not as easy as it sounds and I don't think it ...
0
votes
0answers
7 views

Correlated Data in Model

i have the problem that i am doing a glm quasipoisson in R and some of my Data in the model correlate (0.564 e.g.). Is There a way to deal with this so that i dont get wrong Results? Thanks!
3
votes
0answers
48 views

What is the relationship between an average of correlations and a correlation for an average of the same variables?

I have variables $A, B, C,$ and $X.$ It is common to calculate $r_{AX},$ $r_{BX},$ and $r_{CX}$ correlations; however, sometimes the variables A, B, and C are averaged into one variable, $\...
1
vote
0answers
22 views

Quantifying the observed vs predicted in relation to identity line [duplicate]

I have a model and observed data. I would like to plot the relationship of observed vs predicted values. Predicted values would be generated using the model with optimised parameters. I am ...
0
votes
1answer
24 views

Application of LASSO , Ridge, PLS in feature selection of spectral data

The meatspec data in faraway package is spectral data with 100 features .(215 *101). Use of LASSO over ridge and PLS gives better performance (RMSE based) But none of the features are removed ( no ...
0
votes
1answer
61 views

Why is correlation formula the way it is? Or Say how it formed? [duplicate]

I am really being confused by why the correlation formula is called the correlation of two variables $X$ and $Y$. Also how is it derived? The part where we divide covariance by product of standard ...
0
votes
0answers
16 views

Estimating Pearson correlation for multivariate t distribution using Kendall rank correlation

I had a task to generate a certain amount of samples for $(X_{1},X_{2})$ with a bivariate $t$ distribution $t_{2}(\nu,\mu,\Sigma)$ and then estimate Pearson's correlation coefficient $\rho$ by using ...
6
votes
1answer
175 views

Are all symmetric matrices with diagonal elements 1 and other values between -1 and 1 correlation matrices?

A question for the statisticians and other math lovers: Are all symmetric matrices with diagonal elements 1 and other values between $-1$ and 1 correlation matrices?
1
vote
1answer
57 views

How to separate causality and reverse causality?

For a customer in a grocery store, the greater the number of purchases the longer the shopping path. On the other hand, the longer the shopping path the greater the number of goods to which the ...
0
votes
0answers
14 views

Correct to evaluate the relation between likert type ratings, response time, and age with repeated measures ANOVA and covariates?

I would like to kindly ask you for your point of view on whether my approach is correct. First I will describe my project, then I will ask a question: I conducted approx. 30 surveys in which 28 to ...
1
vote
0answers
16 views

correlation measure in copula

I'm reading about copula but I'm a bit confused about it: we know that the Gaussian copula pdf is equal to : $$C (u_1, u_2, u_3,…, u_n; Σ) = |Σ|^{-1/2} \exp (-1/2 ε^{T}(Σ^{-1} – I)ε)$$ where $Σ$ is ...
0
votes
0answers
11 views

Comparing multiple hurdle to compensatory scoring (of tests)

I plan to do both Monte Carlo and real world research to compare the efficacy of three approaches to combining test scores: (1) fully compensatory scoring, (2) multiple hurdle, and (3) a combination ...
0
votes
0answers
13 views

Sampling theory of correlation; sign of population correlation coefficient & hypothesis design

I'm currently studying 'Estimation of population correlation coefficient from sample correlation coefficient. Source: https://newonlinecourses.science.psu.edu/stat501/node/259/ Test statistic is like ...
0
votes
0answers
15 views

Python: Simulate data correlated with a given array

Although there are some existing methods (e.g. np.random.multivariate_normal) for simulating random correlated variables for given mean, corr and assumed distribution, I couldn't find a way to ...
3
votes
2answers
186 views

Correlation vs Chi Square

Up to now I used correlations to study whether two variables are affected by each other. Now I stumbled over the case that the chi square is doing that too ( http://www.r-tutor.com/elementary-...
0
votes
1answer
19 views

testing whether two correlation coefficients are different from a non zero value

Hello I have 2 bivariate normal populations with correlation coefficients p1 and p2 respectively. I could easily obtain samples from both. I am interested in testing whether the difference in these ...
0
votes
0answers
17 views

Compact way of visualizing heat maps of correlated data

I am trying to visualise the correlation of the Result column with every other column. ...
2
votes
1answer
82 views

Expectation of reciprocal of $(1-r^{2})$

I tried finding expectation from the density function but then realised that I was solving with the density function of $r$ and not it's square. I don't know the density function of $r^{2}$. I am ...
1
vote
0answers
17 views

Uniqueness of partial covariance/corrlation if OLS is not unique

Let $X,Y,Z=(Z_1,...,Z_n)$ be random variables. Define the partial covariance between $X$ and $Y$ given $Z$ as: $$\rho_{X,Y \cdot Z} := cov(\hat{X}-X, \hat{Y}-Y)$$ where $\hat{X}$ and $ \hat{Y}$ are ...
1
vote
3answers
51 views

Is it possible for two independent variables to be correlated, by chance?

I'm reading Introduction to Probability, 2nd Edition by Tsitsiklis and Bertsekas. On page 218, they write "Thus, if X and Y are independent, they are also uncorrelated." Is it possible that there ...
2
votes
1answer
92 views

Linear Relationship vs Correlation

I am new to machine learning, and I'm trying to cover some of the basics. One of the assumptions of linear regression is a linear relationship. However on Reddit I was told today that no machine ...
1
vote
0answers
23 views

Simple mean or weighted mean of Fisher z-transformed correlation coefficients

I need to perform the average of a set of correlation coefficients $\rho_i$ with $i=1,\ldots,m$. I follow the standard prescription: apply the Fisher z-transformation to my $\rho_i$ ($z_i$ are the z-...