Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [correlation]

A measure of the degree of linear association among a pair of variables.

0
votes
0answers
8 views

Corelation between two dichotomous variables

I have a data from a research with responses (yes/no) about three questions (A,B,C) and I would to see if the variables are correlated. First of all i compared the proportion of agreement(yes-yes, no-...
0
votes
0answers
7 views

Comparing two groups of time series correlation coefficients?

I have two groups of data, each consisting of 100 correlation coefficients. I am interested in essentially seeing if the correlations in group 1 are stronger than group 2. However, I am unsure of the ...
0
votes
0answers
18 views

Anti-Correlate Data [on hold]

In the attached image, I want to identify the start and end time of certain periods where those selected variables are anticorrelated using some algorithm (machine learning?), which one do I choose? I ...
0
votes
0answers
11 views

Speed up Conditional Variable Importance for Random Forests

I have trained a random forest in R and now I'm calculating the variable importance mesaure unsing the party Package. ...
0
votes
0answers
14 views

What statistical to use - variability between but not within

I have a data set where each line represents a participant's response to a series of questions (e.g., attitudes towards democracy, life satisfaction). Participants were recruited from different ...
-1
votes
0answers
9 views

the influence of supply chain management on customer satisfaction [on hold]

I was asked to do a pilot test before giving away the questioner among my customer. my tutor asked me to select ten customers and get do a pilot test and know the alpha value. please let me know how ...
-1
votes
0answers
15 views

How to create correlation matrix [on hold]

My project work is on microbiome studies. Here we have OTU table of different persons.How can we create correlation matrix from OTU table?
0
votes
0answers
20 views

Dealing with correlated variables and choosing between models

One of the things I've always been confused about is the framework around model selection in the cases where $n$ predictor variables $x_i$'s are correlated. My thoughts on the different approaches: ...
1
vote
2answers
30 views

Why is correlation not used while splitting in Decision Trees?

Say we have data where we have temperature, humidity, wind and rain for deciding whether we go and play tennis or not. While selecting best feature to split upon initially, we use ID3 or Gini index. ...
0
votes
0answers
6 views

Correlating with average of independent but values of dependent variable?

I want to correlate reef health (independent variable) with fish biomass (dependent variable). I have six location and at each location I determined reef health only once, but fish biomass was ...
0
votes
1answer
26 views

How to evaluate if year-over-year data is correlated and predictive

I'm working with data that is separated by year and classification and want to test if the data is correlated over each year. Essentially, I want to determine if the data is random each year or one ...
0
votes
0answers
3 views

Pair Correlations in R with mixed types in Data Frames [migrated]

If my data frame has mixed types in it (continuous and categorical), and I want to compute pair correlations using, for instance, pairs in R, is there a way to ...
1
vote
0answers
25 views

What is the meaning of correlations between regression coefficients?

I understand that coefficient correlations can arise in the presence of correlated covariates, essentially indicating that our inference for those parameters is coming from information that cannot be ...
1
vote
0answers
7 views

Oversampling to Create Correlated Feature from Existing Feature

For demo purposes in Python, I would like to create an array or pandas df column that shares a high degree of correlation with an existing array. This is not as easy as it sounds and I don't think it ...
0
votes
0answers
7 views

Correlated Data in Model

i have the problem that i am doing a glm quasipoisson in R and some of my Data in the model correlate (0.564 e.g.). Is There a way to deal with this so that i dont get wrong Results? Thanks!
3
votes
0answers
48 views

What is the relationship between an average of correlations and a correlation for an average of the same variables?

I have variables $A, B, C,$ and $X.$ It is common to calculate $r_{AX},$ $r_{BX},$ and $r_{CX}$ correlations; however, sometimes the variables A, B, and C are averaged into one variable, $\...
1
vote
0answers
22 views

Quantifying the observed vs predicted in relation to identity line [duplicate]

I have a model and observed data. I would like to plot the relationship of observed vs predicted values. Predicted values would be generated using the model with optimised parameters. I am ...
0
votes
1answer
22 views

Application of LASSO , Ridge, PLS in feature selection of spectral data

The meatspec data in faraway package is spectral data with 100 features .(215 *101). Use of LASSO over ridge and PLS gives better performance (RMSE based) But none of the features are removed ( no ...
0
votes
1answer
59 views

Why is correlation formula the way it is? Or Say how it formed? [duplicate]

I am really being confused by why the correlation formula is called the correlation of two variables $X$ and $Y$. Also how is it derived? The part where we divide covariance by product of standard ...
0
votes
0answers
15 views

Estimating Pearson correlation for multivariate t distribution using Kendall rank correlation

I had a task to generate a certain amount of samples for $(X_{1},X_{2})$ with a bivariate $t$ distribution $t_{2}(\nu,\mu,\Sigma)$ and then estimate Pearson's correlation coefficient $\rho$ by using ...
6
votes
1answer
175 views

Are all symmetric matrices with diagonal elements 1 and other values between -1 and 1 correlation matrices?

A question for the statisticians and other math lovers: Are all symmetric matrices with diagonal elements 1 and other values between $-1$ and 1 correlation matrices?
1
vote
1answer
57 views

How to separate causality and reverse causality?

For a customer in a grocery store, the greater the number of purchases the longer the shopping path. On the other hand, the longer the shopping path the greater the number of goods to which the ...
0
votes
0answers
14 views

Correct to evaluate the relation between likert type ratings, response time, and age with repeated measures ANOVA and covariates?

I would like to kindly ask you for your point of view on whether my approach is correct. First I will describe my project, then I will ask a question: I conducted approx. 30 surveys in which 28 to ...
1
vote
0answers
16 views

correlation measure in copula

I'm reading about copula but I'm a bit confused about it: we know that the Gaussian copula pdf is equal to : $$C (u_1, u_2, u_3,…, u_n; Σ) = |Σ|^{-1/2} \exp (-1/2 ε^{T}(Σ^{-1} – I)ε)$$ where $Σ$ is ...
0
votes
0answers
10 views

Comparing multiple hurdle to compensatory scoring (of tests)

I plan to do both Monte Carlo and real world research to compare the efficacy of three approaches to combining test scores: (1) fully compensatory scoring, (2) multiple hurdle, and (3) a combination ...
0
votes
0answers
13 views

Sampling theory of correlation; sign of population correlation coefficient & hypothesis design

I'm currently studying 'Estimation of population correlation coefficient from sample correlation coefficient. Source: https://newonlinecourses.science.psu.edu/stat501/node/259/ Test statistic is like ...
0
votes
0answers
15 views

Python: Simulate data correlated with a given array

Although there are some existing methods (e.g. np.random.multivariate_normal) for simulating random correlated variables for given mean, corr and assumed distribution, I couldn't find a way to ...
3
votes
2answers
176 views

Correlation vs Chi Square

Up to now I used correlations to study whether two variables are affected by each other. Now I stumbled over the case that the chi square is doing that too ( http://www.r-tutor.com/elementary-...
0
votes
1answer
19 views

testing whether two correlation coefficients are different from a non zero value

Hello I have 2 bivariate normal populations with correlation coefficients p1 and p2 respectively. I could easily obtain samples from both. I am interested in testing whether the difference in these ...
0
votes
0answers
17 views

Compact way of visualizing heat maps of correlated data

I am trying to visualise the correlation of the Result column with every other column. ...
2
votes
1answer
82 views

Expectation of reciprocal of $(1-r^{2})$

I tried finding expectation from the density function but then realised that I was solving with the density function of $r$ and not it's square. I don't know the density function of $r^{2}$. I am ...
1
vote
0answers
16 views

Uniqueness of partial covariance/corrlation if OLS is not unique

Let $X,Y,Z=(Z_1,...,Z_n)$ be random variables. Define the partial covariance between $X$ and $Y$ given $Z$ as: $$\rho_{X,Y \cdot Z} := cov(\hat{X}-X, \hat{Y}-Y)$$ where $\hat{X}$ and $ \hat{Y}$ are ...
1
vote
3answers
51 views

Is it possible for two independent variables to be correlated, by chance?

I'm reading Introduction to Probability, 2nd Edition by Tsitsiklis and Bertsekas. On page 218, they write "Thus, if X and Y are independent, they are also uncorrelated." Is it possible that there ...
2
votes
1answer
92 views

Linear Relationship vs Correlation

I am new to machine learning, and I'm trying to cover some of the basics. One of the assumptions of linear regression is a linear relationship. However on Reddit I was told today that no machine ...
1
vote
0answers
23 views

Simple mean or weighted mean of Fisher z-transformed correlation coefficients

I need to perform the average of a set of correlation coefficients $\rho_i$ with $i=1,\ldots,m$. I follow the standard prescription: apply the Fisher z-transformation to my $\rho_i$ ($z_i$ are the z-...
8
votes
1answer
97 views

Does correlation between variables in an interaction matter?

Suppose you fit a model $y = x_1 + x_2 + x_1\times x_2$. Are there any practical implications for estimation of the interaction effect if $x_1$ and $x_2$ are correlated? I understand there could be ...
1
vote
1answer
31 views

Generate Variables from Causal Structure for Simulation with Binary/Indicators

I would like to generate a data set of variables from a specific causal structural (a stylized world) for simulation, similar to this answer, but most of the key variables are binary/indicators. ...
1
vote
1answer
39 views

Testing signficance of correlation

I have been trying to study on how to test significance for a correlation coefficient. So far I have seen that we calculate a value t and then either use p-value or critical value from the t-table to ...
3
votes
2answers
44 views

How can I estimate the confidence interval of correlations possibly dependend with time?

I have a multivariate problem (with solar data from different meteorological stations) that I am working on my engineering master thesis. I would like to estimate the correlations of different ...
0
votes
0answers
21 views

Create datasets containing 15 features with normal distributed PCC

I am new to the statistics and currently meeting the difficulty of not familiar with the methods of creating correlated datasets. My task is to assign a normal distributed PCC (Pearson correlation ...
0
votes
1answer
15 views

How to find correlation between categorical independant variable and a continuous dependent variable?

How can i find the correlation between a categorical independant variable and a continuous dependant variable? Traditional correlation metrics dont work in this scenario, should I use any statistical ...
0
votes
0answers
9 views

Correct Correlation Analysis to Use for Multivariate Climate Time Series

I'm faced with an issue of how best to perform a correlation analysis for two time series datasets that have multiple variables. They are both non-stationary (easy to fix on my end), have minimal ...
0
votes
0answers
13 views

Algorithm for anisotropic porous media reconstruction

I have a series of micro-tomography images of a porous material (received from experimental partners) with a resolution 10 μm with total size 1cm × 1cm. Porosity of the porous material lies in the ...
2
votes
1answer
50 views

Why do I need to rounding x and y values to the nearest 0.5 when manually calculating correlation?

I'm trying to calculate correlation using a formula in Statistics 4th Edition by Freedman: r = average of (x in standard units) * (y in standard units) If I try this out ... ...
0
votes
1answer
34 views

how to determine corelation, is there mutual influance between two variable

I want to determine correlation between two variables. Let say we have restaurant, and during a day they make from 1 - 6 pizza per day. We collected data and now we want to determine if there is ...
2
votes
1answer
20 views

Sample size calculation for Kendall's Tau in reproducibility study

Can anyone help with justifying (or rubbishing) a couple of aspects of a sample size calculation for a grant proposal. A pilot study gave significant results for the correlation of a continuous (non-...
1
vote
0answers
19 views

Autocovariance and autocorrelation function of AR(1) process

I'm preparing the exam about AR models, precisely I have this exercise which I have some issues with points "d" and "e". My try was: Knowing that $W_t=X_t-X_{t-1}$, $h=1$ so: d) $\gamma\left(1\right)...
2
votes
2answers
52 views

Knowing correlation, bivariate normality, means and standard deviations, find values probability

After months of study I still do not get it. I apologize (see: Estimating values knowing their Pearson's r and their means and standard deviations) Imagine, for example, I have two bivariate ...
0
votes
1answer
18 views

To standardize or not to standardize in cross correlation

I am new here. My question is simple. Why doesn't the definition of cross-correlation between two time series include a mean centering? Wikipedia defines the cross correlation of two functions as the ...
0
votes
0answers
12 views

Validity of Fisher transformation on autocorrelations?

I'm doing some analysis in which I have to determine if the autocorrelation in one sample is greater than that of another sample. Someone suggested I use the Fisher transform for this purpose, but, ...