Questions tagged [correlation]
A measure of the degree of linear association among a pair of variables.
0
votes
0answers
8 views
Corelation between two dichotomous variables
I have a data from a research with responses (yes/no) about three questions (A,B,C) and I would to see if the variables are correlated.
First of all i compared the proportion of agreement(yes-yes, no-...
0
votes
0answers
7 views
Comparing two groups of time series correlation coefficients?
I have two groups of data, each consisting of 100 correlation coefficients.
I am interested in essentially seeing if the correlations in group 1 are stronger than group 2. However, I am unsure of the ...
0
votes
0answers
18 views
Anti-Correlate Data [on hold]
In the attached image, I want to identify the start and end time of certain periods where those selected variables are anticorrelated using some algorithm (machine learning?), which one do I choose?
I ...
0
votes
0answers
11 views
Speed up Conditional Variable Importance for Random Forests
I have trained a random forest in R and now I'm calculating the variable importance mesaure unsing the party Package.
...
0
votes
0answers
14 views
What statistical to use - variability between but not within
I have a data set where each line represents a participant's response to a series of questions (e.g., attitudes towards democracy, life satisfaction). Participants were recruited from different ...
-1
votes
0answers
9 views
the influence of supply chain management on customer satisfaction [on hold]
I was asked to do a pilot test before giving away the questioner among my customer. my tutor asked me to select ten customers and get do a pilot test and know the alpha value. please let me know how ...
-1
votes
0answers
15 views
How to create correlation matrix [on hold]
My project work is on microbiome studies. Here we have OTU table of different persons.How can we create correlation matrix from OTU table?
0
votes
0answers
20 views
Dealing with correlated variables and choosing between models
One of the things I've always been confused about is the framework around model selection in the cases where $n$ predictor variables $x_i$'s are correlated.
My thoughts on the different approaches:
...
1
vote
2answers
30 views
Why is correlation not used while splitting in Decision Trees?
Say we have data where we have temperature, humidity, wind and rain for deciding whether we go and play tennis or not.
While selecting best feature to split upon initially, we use ID3 or Gini index. ...
0
votes
0answers
6 views
Correlating with average of independent but values of dependent variable?
I want to correlate reef health (independent variable) with fish biomass (dependent variable). I have six location and at each location I determined reef health only once, but fish biomass was ...
0
votes
1answer
26 views
How to evaluate if year-over-year data is correlated and predictive
I'm working with data that is separated by year and classification and want to test if the data is correlated over each year. Essentially, I want to determine if the data is random each year or one ...
0
votes
0answers
3 views
Pair Correlations in R with mixed types in Data Frames [migrated]
If my data frame has mixed types in it (continuous and categorical), and I want to compute pair correlations using, for instance, pairs in R, is there a way to ...
1
vote
0answers
25 views
What is the meaning of correlations between regression coefficients?
I understand that coefficient correlations can arise in the presence of correlated covariates, essentially indicating that our inference for those parameters is coming from information that cannot be ...
1
vote
0answers
7 views
Oversampling to Create Correlated Feature from Existing Feature
For demo purposes in Python, I would like to create an array or pandas df column that shares a high degree of correlation with an existing array.
This is not as easy as it sounds and I don't think it ...
0
votes
0answers
7 views
Correlated Data in Model
i have the problem that i am doing a glm quasipoisson in R and some of my Data in the model correlate (0.564 e.g.). Is There a way to deal with this so that i dont get wrong Results? Thanks!
3
votes
0answers
48 views
What is the relationship between an average of correlations and a correlation for an average of the same variables?
I have variables $A, B, C,$ and $X.$ It is common to calculate $r_{AX},$ $r_{BX},$ and $r_{CX}$ correlations; however, sometimes the variables A, B, and C are averaged into one variable, $\...
1
vote
0answers
22 views
Quantifying the observed vs predicted in relation to identity line [duplicate]
I have a model and observed data. I would like to plot the relationship of observed vs predicted values. Predicted values would be generated using the model with optimised parameters.
I am ...
0
votes
1answer
22 views
Application of LASSO , Ridge, PLS in feature selection of spectral data
The meatspec data in faraway package is spectral data with 100 features .(215 *101). Use of LASSO over ridge and PLS gives better performance (RMSE based)
But none of the features are removed ( no ...
0
votes
1answer
59 views
Why is correlation formula the way it is? Or Say how it formed? [duplicate]
I am really being confused by why the correlation formula is called the correlation of two variables $X$ and $Y$. Also how is it derived?
The part where we divide covariance by product of standard ...
0
votes
0answers
15 views
Estimating Pearson correlation for multivariate t distribution using Kendall rank correlation
I had a task to generate a certain amount of samples for $(X_{1},X_{2})$ with a bivariate $t$ distribution $t_{2}(\nu,\mu,\Sigma)$ and then estimate Pearson's correlation coefficient $\rho$ by using ...
6
votes
1answer
175 views
Are all symmetric matrices with diagonal elements 1 and other values between -1 and 1 correlation matrices?
A question for the statisticians and other math lovers: Are all symmetric matrices with diagonal elements 1 and other values between $-1$ and 1 correlation matrices?
1
vote
1answer
57 views
How to separate causality and reverse causality?
For a customer in a grocery store, the greater the number of purchases the longer the shopping path. On the other hand, the longer the shopping path the greater the number of goods to which the ...
0
votes
0answers
14 views
Correct to evaluate the relation between likert type ratings, response time, and age with repeated measures ANOVA and covariates?
I would like to kindly ask you for your point of view on whether my approach is correct. First I will describe my project, then I will ask a question:
I conducted approx. 30 surveys in which 28 to ...
1
vote
0answers
16 views
correlation measure in copula
I'm reading about copula but I'm a bit confused about it: we know that the Gaussian copula pdf is equal to :
$$C (u_1, u_2, u_3,…, u_n; Σ) = |Σ|^{-1/2} \exp (-1/2 ε^{T}(Σ^{-1} – I)ε)$$
where $Σ$ is ...
0
votes
0answers
10 views
Comparing multiple hurdle to compensatory scoring (of tests)
I plan to do both Monte Carlo and real world research to compare the efficacy of three approaches to combining test scores: (1) fully compensatory scoring, (2) multiple hurdle, and (3) a combination ...
0
votes
0answers
13 views
Sampling theory of correlation; sign of population correlation coefficient & hypothesis design
I'm currently studying 'Estimation of population correlation coefficient from sample correlation coefficient.
Source: https://newonlinecourses.science.psu.edu/stat501/node/259/
Test statistic is like ...
0
votes
0answers
15 views
Python: Simulate data correlated with a given array
Although there are some existing methods (e.g. np.random.multivariate_normal) for simulating random correlated variables for given mean, corr and assumed distribution, I couldn't find a way to ...
3
votes
2answers
176 views
Correlation vs Chi Square
Up to now I used correlations to study whether two variables are affected by each other. Now I stumbled over the case that the chi square is doing that too ( http://www.r-tutor.com/elementary-...
0
votes
1answer
19 views
testing whether two correlation coefficients are different from a non zero value
Hello I have 2 bivariate normal populations with correlation coefficients p1 and p2 respectively. I could easily obtain samples from both. I am interested in testing whether the difference in these ...
0
votes
0answers
17 views
Compact way of visualizing heat maps of correlated data
I am trying to visualise the correlation of the Result column with every other column.
...
2
votes
1answer
82 views
Expectation of reciprocal of $(1-r^{2})$
I tried finding expectation from the density function but then realised that I was solving with the density function of $r$ and not it's square. I don't know the density function of $r^{2}$. I am ...
1
vote
0answers
16 views
Uniqueness of partial covariance/corrlation if OLS is not unique
Let $X,Y,Z=(Z_1,...,Z_n)$ be random variables. Define the partial covariance between $X$ and $Y$ given $Z$ as:
$$\rho_{X,Y \cdot Z} := cov(\hat{X}-X, \hat{Y}-Y)$$ where $\hat{X}$ and $ \hat{Y}$ are ...
1
vote
3answers
51 views
Is it possible for two independent variables to be correlated, by chance?
I'm reading Introduction to Probability, 2nd Edition by Tsitsiklis and Bertsekas.
On page 218, they write "Thus, if X and Y are independent, they are also uncorrelated."
Is it possible that there ...
2
votes
1answer
92 views
Linear Relationship vs Correlation
I am new to machine learning, and I'm trying to cover some of the basics. One of the assumptions of linear regression is a linear relationship.
However on Reddit I was told today that no machine ...
1
vote
0answers
23 views
Simple mean or weighted mean of Fisher z-transformed correlation coefficients
I need to perform the average of a set of correlation coefficients $\rho_i$ with $i=1,\ldots,m$. I follow the standard prescription:
apply the Fisher z-transformation to my $\rho_i$ ($z_i$ are the z-...
8
votes
1answer
97 views
Does correlation between variables in an interaction matter?
Suppose you fit a model $y = x_1 + x_2 + x_1\times x_2$. Are there any practical implications for estimation of the interaction effect if $x_1$ and $x_2$ are correlated?
I understand there could be ...
1
vote
1answer
31 views
Generate Variables from Causal Structure for Simulation with Binary/Indicators
I would like to generate a data set of variables from a specific causal structural (a stylized world) for simulation, similar to this answer, but most of the key variables are binary/indicators.
...
1
vote
1answer
39 views
Testing signficance of correlation
I have been trying to study on how to test significance for a correlation coefficient. So far I have seen that we calculate a value t and then either use p-value or critical value from the t-table to ...
3
votes
2answers
44 views
How can I estimate the confidence interval of correlations possibly dependend with time?
I have a multivariate problem (with solar data from different meteorological stations) that I am working on my engineering master thesis. I would like to estimate the correlations of different ...
0
votes
0answers
21 views
Create datasets containing 15 features with normal distributed PCC
I am new to the statistics and currently meeting the difficulty of not familiar with the methods of creating correlated datasets.
My task is to assign a normal distributed PCC (Pearson correlation ...
0
votes
1answer
15 views
How to find correlation between categorical independant variable and a continuous dependent variable?
How can i find the correlation between a categorical independant variable and a continuous dependant variable? Traditional correlation metrics dont work in this scenario, should I use any statistical ...
0
votes
0answers
9 views
Correct Correlation Analysis to Use for Multivariate Climate Time Series
I'm faced with an issue of how best to perform a correlation analysis for two time series datasets that have multiple variables. They are both non-stationary (easy to fix on my end), have minimal ...
0
votes
0answers
13 views
Algorithm for anisotropic porous media reconstruction
I have a series of micro-tomography images of a porous material (received from experimental partners) with a resolution 10 μm with total size 1cm × 1cm. Porosity of the porous material lies in the ...
2
votes
1answer
50 views
Why do I need to rounding x and y values to the nearest 0.5 when manually calculating correlation?
I'm trying to calculate correlation using a formula in Statistics 4th Edition by Freedman:
r = average of (x in standard units) * (y in standard units)
If I try this out ...
...
0
votes
1answer
34 views
how to determine corelation, is there mutual influance between two variable
I want to determine correlation between two variables. Let say we have restaurant, and during a day they make from 1 - 6 pizza per day. We collected data and now we want to determine if there is ...
2
votes
1answer
20 views
Sample size calculation for Kendall's Tau in reproducibility study
Can anyone help with justifying (or rubbishing) a couple of aspects of a sample size calculation for a grant proposal.
A pilot study gave significant results for the correlation of a continuous (non-...
1
vote
0answers
19 views
Autocovariance and autocorrelation function of AR(1) process
I'm preparing the exam about AR models, precisely I have this exercise which I have some issues with points "d" and "e".
My try was:
Knowing that $W_t=X_t-X_{t-1}$, $h=1$ so:
d) $\gamma\left(1\right)...
2
votes
2answers
52 views
Knowing correlation, bivariate normality, means and standard deviations, find values probability
After months of study I still do not get it. I apologize (see: Estimating values knowing their Pearson's r and their means and standard deviations)
Imagine, for example, I have two bivariate ...
0
votes
1answer
18 views
To standardize or not to standardize in cross correlation
I am new here. My question is simple. Why doesn't the definition of cross-correlation between two time series include a mean centering? Wikipedia defines the cross correlation of two functions as the ...
0
votes
0answers
12 views
Validity of Fisher transformation on autocorrelations?
I'm doing some analysis in which I have to determine if the autocorrelation in one sample is greater than that of another sample. Someone suggested I use the Fisher transform for this purpose, but, ...
