Questions tagged [correlation]

A measure of the degree of linear association among a pair of variables.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Testing for Trends in Sales Data, of 2 Products(Independent data of each Product)

The Data is structured as: Date of Sale | Order Amt | Price of product | Qty. Product A was sold independent of Product B(thus, 2 datasets), so dates do not match at certain instances, and entries are ...
user avatar
0 votes
0 answers
24 views

Linking features via correlations

I am comparing two tables with different size of features (rows), but the same number of samples (columns). I know that a significant part of the features are the same in the two tables, but they are ...
user avatar
  • 1,640
0 votes
1 answer
12 views

How to test the effect between income and economic growth in spss?

am going to test the impact and relationship between income, which is an independent variable, and economic growth, which is a dependent variable, over 10 years in spss? Thanks
user avatar
0 votes
1 answer
13 views

Transitivity of Pearson's Correlation relationship - when does it (not) occur? [duplicate]

Let us assume three quantitative variables: A, B, and C. So far my experience in determining the correlation has been such that: if A correlates positively with B and A correlates positively with C, ...
user avatar
0 votes
1 answer
27 views

Correlation Analysis (Numerical vs Categorical data) in Google Sheets

I'm trying to find the correlation between a categorical and numerical data by using google sheets. The result of CORREL function is 0.041. Could anyone help how to proceed from here to understand if ...
user avatar
0 votes
0 answers
5 views

Likert Scale for 5 independent variables, one yes/no question for dependent variable

I'm seeking to find out whether certain variables (5, total) influence individuals' decision/intention to invest for their own retirement. Whether they already do invest or plan to doesn't matter -- I ...
user avatar
  • 1
0 votes
0 answers
28 views

Compare correlation coefficients or use a mixed effect model?

I have a dataset of documents from 13 government organisations published between 2000 and 2020. The dataset contains between 18 and 20 million words. I have a measure that tells me how unique a word ...
user avatar
  • 13
2 votes
1 answer
199 views

Why is the sum of individual Spearman's rho squared less than 1 as opposed to Pearson's r in a synthetic example?

A relatively low number of iid random vectors of a relatively high dimension (10,000) are added up together element wise: $$\sum_{i=1}^{n}X_i=Y$$ where $dim(X_i)=dim(X_j)=dim(Y),\forall i,j$ and $dim(...
user avatar
  • 261
0 votes
0 answers
24 views

Matrix Completion function in r [closed]

I want to write a matrix completion function for missing values. I am wondering why the correlation of the output with the true missing values is decreasing when I am increasing the number of ...
user avatar
  • 1
1 vote
0 answers
20 views

Significance testing for correlations

Say I have a set of predictions $X$ which predict some variable $Y$. The correlation between $X$ and $Y$ is $r$. How can I determine if $r$ is significant? My intuition would be to do a monte carlo ...
user avatar
2 votes
0 answers
17 views

Linear correlation and machine learning methods

I am reading a paper that has a wide number of explanatory variables. In order to reduce this number, the author first computes the (Pearson) correlation between each pair and keeps just one of the ...
user avatar
0 votes
1 answer
17 views

Correlation betwen lines and columns in a non-mutually exclusive contingency table

I am interested in testing whether there is a correlation between a line and one or several rows in a contingency table. Let's imagine this dummy dataset: In rows, I have the types of meal that you ...
user avatar
-1 votes
0 answers
17 views

Can "Heavily Skewed" Predictor Variables be "Useful" to a Model?

In school, we were always told some general/informal "heuristics" about statistical modelling. In particular, we were advised about the following point: If most of the variable contains ...
user avatar
  • 5,892
1 vote
0 answers
9 views

How can I analyze the relationship between a quantitative and a categorical variable in a within-subject design?

For my bachelor thesis I'm conducting a study on the relationship between eye-movements and memory. One of the hypotheses is that the number of fixations made during the viewing of a movie clip will ...
user avatar
0 votes
0 answers
22 views

Correlation within groups

I have a dataset with 200 variables. Each row of my data represents the quarter results of a company. I want to know if variable X is linearly related to variable Y. I computed the correlation matrix ...
user avatar
0 votes
2 answers
20 views

Is this a white noise? Can I use ARCH/GARCH models on this?

I am trying to find out if I can use ARCH/GARCH models. To my knowledge, to use ARCH/GARCH models you should have autocorrelation and this correlogram should not be a white noise. How can I know if ...
user avatar
  • 1
1 vote
1 answer
30 views

Verify relationship between a categorical and numerical variable

I have got some data about movie directors and number of cuts per minutes in their movies. I whould like to verify if there is any relation between this two variables. I have tried looking at the mean ...
user avatar
  • 109
0 votes
0 answers
15 views

Simple slope of interaction between 2 continues variable and 1 categorical variable in MLM

I have a mixed model with one categorical predictor (X1) and 2 continues predictors (X2, X3) model_3c_P = lmer(Y ~ X1*X2*X3 +(1+X1|Subject),data,REML=TRUE) Here is ...
user avatar
  • 1
0 votes
0 answers
7 views

Running individual semi-partial correlation coefficient tests to get R squared for each individual covariate?

I'm trying to obtain the individual R squared for each coefficient in my model. I understand from other posts on here ( How to split r-squared between predictor variables in multiple regression?), ...
user avatar
  • 13
2 votes
1 answer
42 views

assessing which random sample agrees more with a preferred ranking

I am not a mathematician or a statistician. But, I think the question I have is related to statistics. I will start with a made up example. If I can grade apples into,say four grades from 1 to 4, one ...
user avatar
  • 23
0 votes
0 answers
13 views

Mechanical Relationship between two variables

Recently I attended a conference in which one of the authors was talking about the mechanical relationship between variables "X" and "Y" (I don't remember exactly the variables) ...
user avatar
  • 1
5 votes
1 answer
127 views

Upper bound for random correlation

I recently had a conversation with a college of mine and this question popped into my mind and I am unsure how to solve it. Let's say I have 100 variables. Some of the variables have a pairwise ...
user avatar
1 vote
2 answers
43 views

Understanding the correlations in pairplots

I was using pairplots and I'm stuck in understanding the pair wise relation between those features that have vertical or horizontal lines in the pair plots. (Image source - source) The correlation ...
user avatar
0 votes
0 answers
33 views

How do I find out which variables influence a certain variable [closed]

This question seems trivial but I want to know how to find which variables have a certain impact on a specific other variable in a R data set. I used regressions (in my case multilevel regressions) ...
user avatar
0 votes
0 answers
38 views

What robust test should I use in order to find the correlation between two numeric vectors in R?

I have 2 numeric vectors: ...
user avatar
0 votes
0 answers
12 views

Concordance Correlation Coefficient for negative correlations

A question regarding the Concordance correlation coefficient (CCC). I would like to implement it in Python. Following the Wikipedia entry, one of the possible implementations would be ...
user avatar
0 votes
0 answers
44 views

Finding the vectors/modes "maximally responsible" for the fluctutation of a quantity?

I have a question which I fear might be simple but i am completely unable to figure out. Lets say we have a time-series of a vector of coordinates which define a molecule ( column vector of x,y,z) ...
user avatar
  • 1
0 votes
0 answers
7 views

How to calculate the most likely time spent per customer category based on this data set?

I know how many customers of each customer-size group each sales representative handles on an annual basis. Is it possible to calculate the likely time/effort required by each customer size based on ...
user avatar
1 vote
1 answer
17 views

Is the expected value of a correlation in a randomly permuted sample zero?

Let's say we have two real valued data sets $x$ and $y$, both of length $n$. I do not want to make any further assumptions regarding these data sets. We're interested in their correlation. For testing ...
user avatar
1 vote
1 answer
35 views

How to calculate p-value from the expected correlation coefficient using randomisation in R?

Here, I want to estimate the p-value using randomization. Tests of correlations involving patterns like Y (a function of X) vs. X involve a shared term and the X term. Here I want to randomize (999 ...
user avatar
2 votes
0 answers
21 views

Intra-class correlation coefficients with R

I have recorded the heart rate of a number of participants with two methods: ECG rPPG I am trying to demonstrate that the rPPG is a reliable measure and I think I can use an approach similar to this ...
user avatar
1 vote
1 answer
39 views

Simulating "Realistic Data" for Statistical Problems [closed]

My friend is working on his Sociology Thesis and is currently waiting for the researchers to finish running their experiments and collecting the data that will be used in his project (socio-...
user avatar
  • 5,892
0 votes
1 answer
57 views

Reverse engineering device RGB data

I'm working with a device that maps certain RGB colors to a 7 bit value (0-127): I want to reverse the process, i.e. given any RGB triplet, what is the (closest) corresponding color index (0-127)? ...
user avatar
  • 111
3 votes
2 answers
369 views

Can spurious correlations exist in the (theoretical) population?

Is it possible that spurious correlations exist between two random variables X and Y in the (theoretical) population (here I mean purely by chance, not due to missing confounders in a model, etc.)?
user avatar
  • 173
2 votes
4 answers
124 views

When not to look at p-value

I am trying to indentify correlations between the columns of a dataset I have. I am using the pearson correlation index, and am looking at the p-value I get from the "correlation test" of R. ...
user avatar
  • 109
3 votes
1 answer
55 views

Why is the correlation between two coordinate axes measured by cosine?

In factor analysis, I believe, the correlation between two factors is measured by the cosine of degree between the factors. For instance, the correlation between two factors below is cosine θ, the ...
user avatar
  • 65
0 votes
0 answers
20 views

Why the correlation between factors are penalized when weight is large negative in Oblimin rotation?

I'm facing difficulties in interpreting the criterion for the Oblimin rotation. In my knowledge, the following criterion shall be minimized in oblimin rotation. $$\sum_{ij} (\sum_{v}{l_i}^2{l_j}^2 - \...
user avatar
  • 65
7 votes
2 answers
307 views

How can I generate random observations from a concrete copula?

Let us assume that we have two continuous random variables $X$, $Y$, with known distributions (not necessarily normal), connected/related via a concrete copula. What is a procedure to generate random ...
user avatar
  • 719
1 vote
1 answer
12 views

How can I model, and train a Bayesian Netwok from a given dataset for predicting cause-effect scenario?

I have a dataset that I used to train my ML model for prediction. I created my prediction models using XGBoost and Neural Networks. Now I want to create another model that can give me the causal ...
user avatar
  • 412
5 votes
2 answers
74 views

Interpretation of correlation coefficient between two binary variables

I have a dataset with binary variables for mitigation measures (0= a measure is not implemented, 1 = a measure is not implemented). I now want to know how often a certain measure is put in place ...
user avatar
  • 53
1 vote
1 answer
34 views

Obtain minimum-variance hedge ratio from a copula-GARCH model

Let $r_{s, t}$ and $r_{f, t}$ be the return rates of the spot and futures of a commodity at time $t$. The hedging ratio based on variance minimization is calculated by finding the minimum of the ...
user avatar
1 vote
0 answers
37 views

Expectation of the product of multiple correlated 1-D normal variables [duplicate]

First question: is it possible to have a set of $k$ random variables $\left\{X_i\right\}$ s.t. each $X_i \sim N(0,1)$ individually, and $\text{Corr}(X_i,X_j)=\rho$, $∀i\neq j$? If those conditions are ...
user avatar
3 votes
1 answer
39 views

Generating data in the desired correlation structure from the multivariate normal distribution in R

I want to derive 200 variables from the multivariate normal distribution. I will divide these 200 variables into 3 blocks. The 1st block will consist of 40 variables and will be low-moderately ...
user avatar
0 votes
0 answers
10 views

Canonical Correlation Analysis Significance Test for non-normal data distribution

I am running a number of Canonical Correlation Analysis methods on some time series data on variables that are not normally distributed. Some of the methods I intend to apply are: Classical CCA ...
user avatar
  • 101
0 votes
0 answers
61 views

Estimating Successful Combinations of Basketball Players

Suppose we have to analyze sports data (from our school 's informal basketball data). We have data for all games from the 2019 season, for each game we know which players were playing and which ...
user avatar
  • 5,892
0 votes
0 answers
14 views

Analyzing / comparing two trajectories that have an unequal number of elements

I'm doing an experiment where a user tries to draw a shown trajectory. So, we have two trajectories - one 'template' trajectory with a fixed number of elements (3d coordinates) and a trajectory with ...
user avatar
  • 1
0 votes
0 answers
34 views

Ensuring Correlation Between Groups of Variables

Correlation is usually defined between 2 variables - for example, height and weight measurements in athletes might be correlated, or height and salary might also be correlated. Suppose I want to ...
user avatar
  • 5,892
1 vote
1 answer
33 views

Is there a clear interpretation of Corr(X, X+Y) in research?

Consider a case of $Corr(X,Z)$, often found to be high; where later, it was found that it holds exactly $Z = X + Y$. In effect, the previously found correlations were equal to $Corr(X, X+Y)$. How can ...
user avatar
  • 85
0 votes
0 answers
27 views

Siamese networks for comparison of a dataset with multiple features (text, numeric, and ordinal)

Lets assume I have a dataset where each row has ordinal, numerical, and free text columns/features. Some of the rows are grouped together i.e.: ...
user avatar
  • 521
0 votes
0 answers
44 views

Simulating Correlated Categorical Data

I am trying to simulate "correlated categorical data". For instance, consider the following example: Suppose there are 10 players (p1, p2, ...p10) - each day, a random combination of these ...
user avatar
  • 5,892

1
2 3 4 5
117