Questions tagged [correlation]

A measure of the degree of linear association among a pair of variables.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Pearson correlation and VIF with categorical variables

All columns' values are class labels. For example: value "1" for feature1 is <50. Namely, all features were classified. When I apply Pearson correlation with continuous variables (before ...
user avatar
  • 1
1 vote
1 answer
18 views

How are bivariate lognormal parameters related to the underlying normal parameters?

If $Y∼N(μ,σ^2)$ is normally distributed, then $X=\exp(Y)$ is lognormally distributed. The parameters for a univariate distribution, $\mu$ and $\sigma$ of this lognormal distribution are given by $$\mu{...
user avatar
0 votes
0 answers
9 views

Sampling from a Poisson Process

I am trying to simulate a one dimension correlated random walk. In this algorithm, the direction of a particle’s next step is correlated with the direction of it’s previous step. The particle’s step ...
user avatar
0 votes
1 answer
33 views

Correlation of multivariate distributions without "slope"

Wikipedia has this image showing different correlations: It says: The correlation reflects the noisiness and direction of a linear relationship (top row), but not the slope of that relationship (...
user avatar
  • 547
0 votes
0 answers
12 views

How to find correlation between continuous variable and yes/no question? [duplicate]

I'm trying to find out if there is correlation between website clicks and sales. In order to do this, I'm trying to find the correlation between a nominal question (Yes and No) and a continuous ...
user avatar
0 votes
0 answers
13 views

Correlation between binary (true/false) variable(s) and a continuous variable (a real number)

I have four types of binary variables (true or false) and a single continuous variable. I want to analyze the correlation between any of those binary variables with the continuous variable. My dataset ...
user avatar
0 votes
0 answers
23 views

What is the correct definition of the correlation of two random variables?

In the book Signal Processing for Communications by Prandoni and Vetterli, correlation of two variables is defined as the expected value of their product, while the covariance is refered to as the &...
user avatar
-2 votes
0 answers
18 views

Electron-electron interaction term in Hatsugai-Kohmoto model [closed]

Hatsugai-Kohmoto (HK) model contains interaction between electrons that is not local in position, but is local in momentum space. Moreover, it is an interaction that conserves center of mass of ...
user avatar
1 vote
1 answer
26 views

Permutation testing: Which variable should be shuffled?

When conducting a permutation test, which variable should be shuffled? Let's say I have two variables X and Y, and my test statistic is the correlation coefficient between the two variables. To test ...
user avatar
  • 21
0 votes
0 answers
25 views

model.matrix correlations of discrete

I have the following full factorial design and I would like to evaluate the factors independency. ...
user avatar
  • 115
-2 votes
0 answers
14 views

Nature of electron-electron interaction in Hatsugai-Kohmoto model [closed]

In Hatsugai-Kohmoto model of strongly correlated electrons, there is an interaction term that tells us that electron-electron interaction is local only in momentum space. After Fourier transform, we ...
user avatar
1 vote
1 answer
31 views

Non-positive definite matrix problem for desired correlation structure

I want to derive a correlation matrix such that block1 is 0.1 within itself, block2 is 0.1 within itself and 0.7 with block1, and the remaining variables are 0.01 within itself and with other blocks ...
user avatar
1 vote
2 answers
43 views

Why would quartile plotted against percentile not be a straight line?

In the famous Dunning-Kruger paper (ungated) perceived and actual scores are plotted, with quartile on the x axis and percentile on the y axis. The message is the perceived scores, and the actual ...
user avatar
  • 111
2 votes
0 answers
25 views

If there is a high correlation between two variables, should we expect the high linear regression coefficient?

I have a data set with multiple features, let suppose $x_1,x_2,x_3,x_4$ and my dependent variable is $y$, when I compute the correlation matrix for $y,x_1,x_2,x_3,x_4$ then imagine the correlation ...
user avatar
  • 133
0 votes
0 answers
8 views

Assumptions for correlation disattenuation

I am familiar with the Spearman disattenuation formula for two random variables $X^\prime$ and $Y^\prime$ measured as $X$ and $Y$ with measured correlation $r_{xy}$ and reliabilities $r_{xx}$ and $r_{...
user avatar
1 vote
0 answers
25 views

When does the Instrumental Variable (IV) method fail?

Consider the following ARMAX model: $$ y[k]=\alpha y[k-1]+\beta u[k-1]+e[k]-0.7e[k-1] $$ where $e[k]$ is a white noise. One can see that $e[k]-0.7e[k-1]$ is a filtered noise and thus using ordinary ...
user avatar
  • 111
0 votes
0 answers
21 views

Practical correlation metric for a large number of vectors

I am dealing with a timeseries consisting of input flow sampled every 5 minutes over 441 days. My aim is to find any possible correlation from data coming from: The same day of the week The same ...
user avatar
  • 11
0 votes
0 answers
5 views

Validate correlations with multiple influence factors

I am having a problem with validating my correlations. To explain the situation: Dataframe 1: the number of observations for each day in 2 years. Dataframe 2: multiple weather and air quality ...
user avatar
  • 101
0 votes
1 answer
37 views

Distribution of the Spearman rank correlation coefficient under the assumption of non-zero correlation

There are some papers and some R packages providing exact calculations for the CDF and the inverse-CDF of the (sample) Spearman rank correlation coefficient. My question is: How difficult would it be ...
user avatar
  • 649
0 votes
1 answer
31 views

Distribution function of the linear combination of standardized student-t quantiles [duplicate]

Assume to observe 2 quantiles, x and y, associated with the z% probability. These quantiles are generated by 2 non-independent standardized student-t distributions X and Y. In case of linear ...
user avatar
0 votes
0 answers
18 views

Interpretation of binomial regression in R

I'm running a regression to see if theres a positive or negative relationship between cvdrst and smkcigst, but I'm not sure how to interpret this regression also why is the data for smkcigst NA, even ...
user avatar
  • 1
2 votes
0 answers
27 views

Spearman Correlation for multiple variables

I want to characterize the correlation of a set of variables with a target variable using Spearman correlation since I expect the relationship to be monotonic but not necessarily linear. Specifically, ...
user avatar
  • 121
1 vote
1 answer
22 views

How do I test that two samples don’t differ significantly by their demographics?

I am conducting a study where I am randomly assigning people to different conditions. Because I am recruiting from a representative sample in the US, there will be some variability in the demographics ...
user avatar
  • 125
0 votes
0 answers
4 views

Serial correlation fix

I have some question about positive serial correlation: What happens to the standard error of the model once transformed using generalized differencing/Cochrane-Orcutt? Is it higher than the original ...
user avatar
0 votes
0 answers
15 views

How to perform variable/feature selection before random forest in R?

I have a phyloseq object with 4000+ taxa and 300+ samples. What methods/packages can I use to perform feature selection to reduce the number of taxa/OTUs prior to performing random forest? I was ...
user avatar
0 votes
1 answer
24 views

Is the correlation coefficient of the residuals related to the correlation coefficient of the original distirbution?

I have 10 different 2D datasets which each have 200 datapoints. Each data point has a 'quality' property ($Q$) which somewhat depends on the area ($A$) of that datapoint. There is a (relatively) ...
user avatar
0 votes
0 answers
16 views

Testing equality of coefficients from two different samples

I have the regression statistics for the same regression run on two different samples, and am asked to explain whether it is possible to test for equality of the coefficents, $\beta_1$and $\beta_2$ ...
user avatar
0 votes
1 answer
51 views

Conditional probability for autocorrelated binary random variables

In this answer, @whuber outlines conditional probabilities for correlated binary random variables. In some recent exploration, I have been playing with inference for sequences of binary random ...
user avatar
0 votes
1 answer
31 views

Correlations significant at individual level, but not at group level?

For my research I have to do correlation tests to see how different measures of animal welfare are related to each other. When i run the test using the data at individual level, it shows a weak ...
user avatar
  • 101
0 votes
0 answers
13 views

What is the meaning of the inner product between two regression variables?

I have been analyzing the effect of design matrix columns on the contour line of the least squares regression. These contours obviously are ellipses when only two columns $\phi_1$ and $\phi_2$ are ...
user avatar
  • 11
0 votes
1 answer
25 views

Correlations between variables based on gender in R

I have a data set in which female is coded as 0 and male as 1 and then I have different testing scores; writing, reading, math. I want to determine whether the correlation between math and writing ...
user avatar
  • 11
0 votes
0 answers
18 views

Relationship between normally distributed errors (bivariate OLS regression) and bivariate normality of variables (Pearson’s r)

I have forty quite diverse statistical text books at hand but I cannot find a reliable answer. I’ve studied the following and associated threads, but they didn’t got into it that deep: here and here. ...
user avatar
0 votes
0 answers
21 views

Cumulative Distribution function of ratio of 2 correlated gamma distributed random variables

Let $X \sim \Gamma(k_x, \theta_x)$, $Y\sim \Gamma(k_y, \theta_y)$ be correlated with correlation coefficient $\rho$. The shape and scale parameters are different. i.e. $k_x \neq k_y$ and $\theta_x \...
user avatar
2 votes
1 answer
34 views

What is the best way to represent Spearman’s correlation graphically?

For my thesis research I have to do Spearman’s correlation tests on a set of ordinal variables. I would like to represent the correlations graphically to include them in the report and give a clearer ...
user avatar
  • 101
0 votes
0 answers
7 views

statistical criteria for grouping questions in questionnaire data analysis and treat averaging the respond on these groups as continuous variable?

In questionnaire data analysis, if you as a statistician received data after designing and filling the questionnaire by respondents, and encounter pre-determined groups of questions that are claimed ...
user avatar
0 votes
0 answers
25 views

Change in Correlation after Imputation of Missing Outcome Data for Machine Learning Prediction Task

I am attempting to generate a predictive model for a continuous outcome using machine learning models. However, some observations in the original dataset have missing outcome data (and missing ...
user avatar
  • 5
1 vote
1 answer
20 views

Prediction based on correlation

I know -- correlation does not mean causation. However, I need a "best guess" estimate for the following scenario: Variable A = Starting Point: 4,5 (SD=0,5) Variable B = Starting Point: 300 (...
user avatar
1 vote
0 answers
17 views

Cross Correlation Techniques for Time Series Data [closed]

I am working with time series data and am currently using cross correlation on my data. I can use different techniques like spearman, pearson, and kendall. I was wondering if beyond that, there are ...
user avatar
1 vote
0 answers
22 views

How to calculate the value (wind direction) of "maximal effect" in a linear-circular correlation?

I am trying to evaluate the effect of wind direction (circular variable) on a dependent linear variable. I have used circular-linear regression to find the correlation coefficient between the two ...
user avatar
0 votes
0 answers
25 views

Correlation matrix from pairwise correlations with specified structure

I need to simulate multivariate normal samples with a pre-specified correlation structure. The structure is such that the bigger the (GPS) distance between two points, the smaller the correlation (...
user avatar
  • 184
0 votes
0 answers
11 views

Should p.values be adjusted when comparing correlations of differently sized groups?

I have performed a series of correlations between several variables (all of them are continuous). Some of these correlations were conducted with the entire sample (which is composed of two independent ...
user avatar
  • 31
0 votes
0 answers
12 views

How to interpret an autocorrelation function (ACF) plot of residuals? [closed]

I have an autocorrelation plot of residuals. Does it show correlation? It is for a time series with monthly data.
user avatar
1 vote
0 answers
50 views

Difference between y against x, x^2 and sqrt(y) against x in a linear model

Suppose that we have a response variable y which is known to have a quadratic relationship with a predictor variable x. What are the differences between fitting a linear model of y against x and x^2, ...
user avatar
0 votes
1 answer
47 views

Categorical variable to be predicted from continuous variables with an idea: "maximise boxplots distance"

I am not sure the title is easy to understand, but this is because I'm struggling to find out if my question is machine-learning related, or just correlation-related, or anything else. Context: I have ...
user avatar
1 vote
0 answers
9 views

How to analysis the correlation of one value corresponding to multiple values

For example, there is a school teacher's answering score corresponding to multiple students' answering scores (which like 1 for unlikely, and 5 for very likely). For analyzing the correlation of this ...
user avatar
0 votes
1 answer
31 views

Point-Biserial Correlation vs Pearson's Correlation

I know that when looking at the correlation between a binary and a continuous variable we should use point biserial correlation. Today I was looking at some data and mistakenly used Pearson's. When I ...
user avatar
  • 49.4k
0 votes
0 answers
23 views

How to find what categorical variables explain a continuous dependent variable?

I am currently working on a statistical analysis for a project, I have a dependent variable ($y$) that represents delays in days, it's continuous and take integer values. I also have a set of over 90 ...
user avatar
0 votes
0 answers
15 views

Parametric or non-parametric if a variable is distributed normally and another non-normally distributed

I have 2 variables that I'd like to know if one (a concentration of a substance in a biological fluid) is dependent on the other (the percentage of inhibition of this fluid over the substance). I'd ...
user avatar
1 vote
0 answers
22 views

Interpreting fisher’s Z transformation and correlations

Part of my planned analysis in a paper is testing the difference in strength of correlations between depression and other things like social relationships and quality of life. This test is set up with ...
user avatar
4 votes
3 answers
126 views

Determine whether two binary sequences are correlated

I have two arrays of 2048 binary values (1 and 0) and I'd like to find out if their content can be considered correlated. I'd like to have less than 1:1000000 chance of a false positive in determining ...
user avatar

1
2 3 4 5
116