Questions tagged [correlation]
A measure of the degree of linear association among a pair of variables.
5,787
questions
0
votes
0
answers
8
views
Pearson correlation and VIF with categorical variables
All columns' values are class labels. For example: value "1" for feature1 is <50. Namely, all features were classified.
When I apply Pearson correlation with continuous variables (before ...
- 1
1
vote
1
answer
18
views
How are bivariate lognormal parameters related to the underlying normal parameters?
If $Y∼N(μ,σ^2)$ is normally distributed, then $X=\exp(Y)$ is lognormally distributed. The parameters for a univariate distribution, $\mu$ and $\sigma$ of this lognormal distribution are given by
$$\mu{...
- 11
0
votes
0
answers
9
views
Sampling from a Poisson Process
I am trying to simulate a one dimension correlated random walk. In this algorithm, the direction of a particle’s next step is correlated with the direction of it’s previous step. The particle’s step ...
0
votes
1
answer
33
views
Correlation of multivariate distributions without "slope"
Wikipedia has this image showing different correlations:
It says:
The correlation reflects the noisiness and direction of a linear relationship (top row), but not the slope of that relationship (...
- 547
0
votes
0
answers
12
views
How to find correlation between continuous variable and yes/no question? [duplicate]
I'm trying to find out if there is correlation between website clicks and sales. In order to do this, I'm trying to find the correlation between a nominal question (Yes and No) and a continuous ...
0
votes
0
answers
13
views
Correlation between binary (true/false) variable(s) and a continuous variable (a real number)
I have four types of binary variables (true or false) and a single continuous variable. I want to analyze the correlation between any of those binary variables with the continuous variable. My dataset ...
- 51
0
votes
0
answers
23
views
What is the correct definition of the correlation of two random variables?
In the book Signal Processing for Communications by Prandoni and Vetterli, correlation of two variables is defined as the expected value of their product, while the covariance is refered to as the &...
- 247
-2
votes
0
answers
18
views
Electron-electron interaction term in Hatsugai-Kohmoto model [closed]
Hatsugai-Kohmoto (HK) model contains interaction between electrons that is not local in position, but is local in momentum space. Moreover, it is an interaction that conserves center of mass of ...
1
vote
1
answer
26
views
Permutation testing: Which variable should be shuffled?
When conducting a permutation test, which variable should be shuffled?
Let's say I have two variables X and Y, and my test statistic is the correlation coefficient between the two variables. To test ...
- 21
0
votes
0
answers
25
views
model.matrix correlations of discrete
I have the following full factorial design and I would like to evaluate the factors independency.
...
- 115
-2
votes
0
answers
14
views
Nature of electron-electron interaction in Hatsugai-Kohmoto model [closed]
In Hatsugai-Kohmoto model of strongly correlated electrons, there is an interaction term that tells us that electron-electron interaction is local only in momentum space. After Fourier transform, we ...
1
vote
1
answer
31
views
Non-positive definite matrix problem for desired correlation structure
I want to derive a correlation matrix such that block1 is 0.1 within itself, block2 is 0.1 within itself and 0.7 with block1, and the remaining variables are 0.01 within itself and with other blocks ...
- 23
1
vote
2
answers
43
views
Why would quartile plotted against percentile not be a straight line?
In the famous Dunning-Kruger paper (ungated) perceived and actual scores are plotted, with quartile on the x axis and percentile on the y axis. The message is the perceived scores, and the actual ...
- 111
2
votes
0
answers
25
views
If there is a high correlation between two variables, should we expect the high linear regression coefficient?
I have a data set with multiple features, let suppose $x_1,x_2,x_3,x_4$ and my dependent variable is $y$, when I compute the correlation matrix for $y,x_1,x_2,x_3,x_4$ then imagine the correlation ...
- 133
0
votes
0
answers
8
views
Assumptions for correlation disattenuation
I am familiar with the Spearman disattenuation formula for two random variables $X^\prime$ and $Y^\prime$ measured as $X$ and $Y$ with measured correlation $r_{xy}$ and reliabilities $r_{xx}$ and $r_{...
1
vote
0
answers
25
views
When does the Instrumental Variable (IV) method fail?
Consider the following ARMAX model:
$$
y[k]=\alpha y[k-1]+\beta u[k-1]+e[k]-0.7e[k-1]
$$
where $e[k]$ is a white noise. One can see that $e[k]-0.7e[k-1]$ is a filtered noise and thus using ordinary ...
- 111
0
votes
0
answers
21
views
Practical correlation metric for a large number of vectors
I am dealing with a timeseries consisting of input flow sampled every 5 minutes over 441 days. My aim is to find any possible correlation from data coming from:
The same day of the week
The same ...
- 11
0
votes
0
answers
5
views
Validate correlations with multiple influence factors
I am having a problem with validating my correlations.
To explain the situation:
Dataframe 1: the number of observations for each day in 2 years.
Dataframe 2: multiple weather and air quality ...
- 101
0
votes
1
answer
37
views
Distribution of the Spearman rank correlation coefficient under the assumption of non-zero correlation
There are some papers and some R packages providing exact calculations for the CDF and the inverse-CDF of the (sample) Spearman rank correlation coefficient.
My question is: How difficult would it be ...
- 649
0
votes
1
answer
31
views
Distribution function of the linear combination of standardized student-t quantiles [duplicate]
Assume to observe 2 quantiles, x and y, associated with the z% probability. These quantiles are generated by 2 non-independent standardized student-t distributions X and Y.
In case of linear ...
- 1
0
votes
0
answers
18
views
Interpretation of binomial regression in R
I'm running a regression to see if theres a positive or negative relationship between cvdrst and smkcigst, but I'm not sure how to interpret this regression also why is the data for smkcigst NA, even ...
- 1
2
votes
0
answers
27
views
Spearman Correlation for multiple variables
I want to characterize the correlation of a set of variables with a target variable using Spearman correlation since I expect the relationship to be monotonic but not necessarily linear. Specifically, ...
- 121
1
vote
1
answer
22
views
How do I test that two samples don’t differ significantly by their demographics?
I am conducting a study where I am randomly assigning people to different conditions. Because I am recruiting from a representative sample in the US, there will be some variability in the demographics ...
- 125
0
votes
0
answers
4
views
Serial correlation fix
I have some question about positive serial correlation:
What happens to the standard error of the model once transformed using generalized differencing/Cochrane-Orcutt? Is it higher than the original ...
- 1
0
votes
0
answers
15
views
How to perform variable/feature selection before random forest in R?
I have a phyloseq object with 4000+ taxa and 300+ samples.
What methods/packages can I use to perform feature selection to reduce the number of taxa/OTUs prior to performing random forest? I was ...
0
votes
1
answer
24
views
Is the correlation coefficient of the residuals related to the correlation coefficient of the original distirbution?
I have 10 different 2D datasets which each have 200 datapoints. Each data point has a 'quality' property ($Q$) which somewhat depends on the area ($A$) of that datapoint. There is a (relatively) ...
- 11
0
votes
0
answers
16
views
Testing equality of coefficients from two different samples
I have the regression statistics for the same regression run on two different samples, and am asked to explain whether it is possible to test for equality of the coefficents, $\beta_1$and $\beta_2$ ...
0
votes
1
answer
51
views
Conditional probability for autocorrelated binary random variables
In this answer, @whuber outlines conditional probabilities for correlated binary random variables. In some recent exploration, I have been playing with inference for sequences of binary random ...
- 25.8k
0
votes
1
answer
31
views
Correlations significant at individual level, but not at group level?
For my research I have to do correlation tests to see how different measures of animal welfare are related to each other. When i run the test using the data at individual level, it shows a weak ...
- 101
0
votes
0
answers
13
views
What is the meaning of the inner product between two regression variables?
I have been analyzing the effect of design matrix columns on the contour line of the least squares regression. These contours obviously are ellipses when only two columns $\phi_1$ and $\phi_2$ are ...
- 11
0
votes
1
answer
25
views
Correlations between variables based on gender in R
I have a data set in which female is coded as 0 and male as 1 and then I have different testing scores; writing, reading, math. I want to determine whether the correlation between math and writing ...
- 11
0
votes
0
answers
18
views
Relationship between normally distributed errors (bivariate OLS regression) and bivariate normality of variables (Pearson’s r)
I have forty quite diverse statistical text books at hand but I cannot find a reliable answer.
I’ve studied the following and associated threads, but they didn’t got into it that deep: here and here. ...
0
votes
0
answers
21
views
Cumulative Distribution function of ratio of 2 correlated gamma distributed random variables
Let $X \sim \Gamma(k_x, \theta_x)$, $Y\sim \Gamma(k_y, \theta_y)$ be correlated with correlation coefficient $\rho$. The shape and scale parameters are different. i.e. $k_x \neq k_y$ and $\theta_x \...
2
votes
1
answer
34
views
What is the best way to represent Spearman’s correlation graphically?
For my thesis research I have to do Spearman’s correlation tests on a set of ordinal variables. I would like to represent the correlations graphically to include them in the report and give a clearer ...
- 101
0
votes
0
answers
7
views
statistical criteria for grouping questions in questionnaire data analysis and treat averaging the respond on these groups as continuous variable?
In questionnaire data analysis,
if you as a statistician received data after designing and filling the questionnaire by respondents, and encounter pre-determined groups of questions that are claimed ...
- 11
0
votes
0
answers
25
views
Change in Correlation after Imputation of Missing Outcome Data for Machine Learning Prediction Task
I am attempting to generate a predictive model for a continuous outcome using machine learning models. However, some observations in the original dataset have missing outcome data (and missing ...
- 5
1
vote
1
answer
20
views
Prediction based on correlation
I know -- correlation does not mean causation.
However, I need a "best guess" estimate for the following scenario:
Variable A = Starting Point: 4,5 (SD=0,5)
Variable B = Starting Point: 300 (...
1
vote
0
answers
17
views
Cross Correlation Techniques for Time Series Data [closed]
I am working with time series data and am currently using cross correlation on my data. I can use different techniques like spearman, pearson, and kendall.
I was wondering if beyond that, there are ...
- 11
1
vote
0
answers
22
views
How to calculate the value (wind direction) of "maximal effect" in a linear-circular correlation?
I am trying to evaluate the effect of wind direction (circular variable) on a dependent linear variable. I have used circular-linear regression to find the correlation coefficient between the two ...
- 21
0
votes
0
answers
25
views
Correlation matrix from pairwise correlations with specified structure
I need to simulate multivariate normal samples with a pre-specified correlation structure. The structure is such that the bigger the (GPS) distance between two points, the smaller the correlation (...
- 184
0
votes
0
answers
11
views
Should p.values be adjusted when comparing correlations of differently sized groups?
I have performed a series of correlations between several variables (all of them are continuous). Some of these correlations were conducted with the entire sample (which is composed of two independent ...
- 31
0
votes
0
answers
12
views
How to interpret an autocorrelation function (ACF) plot of residuals? [closed]
I have an autocorrelation plot of residuals. Does it show correlation? It is for a time series with monthly data.
Lexxie
1
vote
0
answers
50
views
Difference between y against x, x^2 and sqrt(y) against x in a linear model
Suppose that we have a response variable y which is known to have a quadratic relationship
with a predictor variable x. What are the differences between fitting a linear model of y
against x and x^2, ...
- 11
0
votes
1
answer
47
views
Categorical variable to be predicted from continuous variables with an idea: "maximise boxplots distance"
I am not sure the title is easy to understand, but this is because I'm struggling to find out if my question is machine-learning related, or just correlation-related, or anything else.
Context: I have ...
1
vote
0
answers
9
views
How to analysis the correlation of one value corresponding to multiple values
For example, there is a school teacher's answering score corresponding to multiple students' answering scores (which like 1 for unlikely, and 5 for very likely). For analyzing the correlation of this ...
- 11
0
votes
1
answer
31
views
Point-Biserial Correlation vs Pearson's Correlation
I know that when looking at the correlation between a binary and a continuous variable we should use point biserial correlation.
Today I was looking at some data and mistakenly used Pearson's. When I ...
- 49.4k
0
votes
0
answers
23
views
How to find what categorical variables explain a continuous dependent variable?
I am currently working on a statistical analysis for a project, I have a dependent variable ($y$) that represents delays in days, it's continuous and take integer values.
I also have a set of over 90 ...
- 1
0
votes
0
answers
15
views
Parametric or non-parametric if a variable is distributed normally and another non-normally distributed
I have 2 variables that I'd like to know if one (a concentration of a substance in a biological fluid) is dependent on the other (the percentage of inhibition of this fluid over the substance). I'd ...
1
vote
0
answers
22
views
Interpreting fisher’s Z transformation and correlations
Part of my planned analysis in a paper is testing the difference in strength of correlations between depression and other things like social relationships and quality of life. This test is set up with ...
4
votes
3
answers
126
views
Determine whether two binary sequences are correlated
I have two arrays of 2048 binary values (1 and 0) and I'd like to find out if their content can be considered correlated. I'd like to have less than 1:1000000 chance of a false positive in determining ...
- 93