Questions tagged [correlation]
A measure of the degree of linear association among a pair of variables.
5,780
questions
0
votes
0
answers
4
views
Why would quartile plotted against percentile not be a straight line?
In the famous Dunning-Kruger paper perceived and actual scores are plotted, with quartile on the x axis and percentile on the y axis. The message is the perceived scores, and the actual scores are ...
- 101
1
vote
0
answers
18
views
If there is a high correlation between two variables, should we expect the high linear regression coefficient?
I have a data set with multiple features, let suppose $x_1,x_2,x_3,x_4$ and my dependent variable is $y$, when I compute the correlation matrix for $y,x_1,x_2,x_3,x_4$ then imagine the correlation ...
- 123
0
votes
0
answers
6
views
Assumptions for correlation disattenuation
I am familiar with the Spearman disattenuation formula for two random variables $X^\prime$ and $Y^\prime$ measured as $X$ and $Y$ with measured correlation $r_{xy}$ and reliabilities $r_{xx}$ and $r_{...
1
vote
0
answers
17
views
When does the Instrumental Variable (IV) method fail?
Consider the following ARMAX model:
$$
y[k]=\alpha y[k-1]+\beta u[k-1]+e[k]-0.7e[k-1]
$$
where $e[k]$ is a white noise. One can see that $e[k]-0.7e[k-1]$ is a filtered noise and thus using ordinary ...
- 111
0
votes
0
answers
48
views
auto-correlation and cross-correlation of multivariate random processes
Assume $N$ observations of a multivariate random process $X(t)$ of dimension $q$ arranged in a matrix as it follows, where each column represents a time-series and the $i$-th row is a realization of ...
- 101
0
votes
0
answers
21
views
Practical correlation metric for a large number of vectors
I am dealing with a timeseries consisting of input flow sampled every 5 minutes over 441 days. My aim is to find any possible correlation from data coming from:
The same day of the week
The same ...
- 11
0
votes
0
answers
5
views
Validate correlations with multiple influence factors
I am having a problem with validating my correlations.
To explain the situation:
Dataframe 1: the number of observations for each day in 2 years.
Dataframe 2: multiple weather and air quality ...
- 101
0
votes
1
answer
37
views
Distribution of the Spearman rank correlation coefficient under the assumption of non-zero correlation
There are some papers and some R packages providing exact calculations for the CDF and the inverse-CDF of the (sample) Spearman rank correlation coefficient.
My question is: How difficult would it be ...
- 649
0
votes
1
answer
29
views
Distribution function of the linear combination of standardized student-t quantiles [duplicate]
Assume to observe 2 quantiles, x and y, associated with the z% probability. These quantiles are generated by 2 non-independent standardized student-t distributions X and Y.
In case of linear ...
- 1
0
votes
0
answers
17
views
Interpretation of binomial regression in R
I'm running a regression to see if theres a positive or negative relationship between cvdrst and smkcigst, but I'm not sure how to interpret this regression also why is the data for smkcigst NA, even ...
- 1
2
votes
0
answers
23
views
Spearman Correlation for multiple variables
I want to characterize the correlation of a set of variables with a target variable using Spearman correlation since I expect the relationship to be monotonic but not necessarily linear. Specifically, ...
- 121
0
votes
1
answer
15
views
The correlation coefficient with a priori intercept and slope [closed]
What does the formula for the product moment correlation coefficient look like if the intercept and the slope are given a priori?
- 1
1
vote
1
answer
21
views
How do I test that two samples don’t differ significantly by their demographics?
I am conducting a study where I am randomly assigning people to different conditions. Because I am recruiting from a representative sample in the US, there will be some variability in the demographics ...
- 125
0
votes
0
answers
3
views
Serial correlation fix
I have some question about positive serial correlation:
What happens to the standard error of the model once transformed using generalized differencing/Cochrane-Orcutt? Is it higher than the original ...
- 1
0
votes
0
answers
13
views
How to perform variable/feature selection before random forest in R?
I have a phyloseq object with 4000+ taxa and 300+ samples.
What methods/packages can I use to perform feature selection to reduce the number of taxa/OTUs prior to performing random forest? I was ...
0
votes
1
answer
24
views
Is the correlation coefficient of the residuals related to the correlation coefficient of the original distirbution?
I have 10 different 2D datasets which each have 200 datapoints. Each data point has a 'quality' property ($Q$) which somewhat depends on the area ($A$) of that datapoint. There is a (relatively) ...
- 11
0
votes
0
answers
16
views
Testing equality of coefficients from two different samples
I have the regression statistics for the same regression run on two different samples, and am asked to explain whether it is possible to test for equality of the coefficents, $\beta_1$and $\beta_2$ ...
0
votes
1
answer
51
views
Conditional probability for autocorrelated binary random variables
In this answer, @whuber outlines conditional probabilities for correlated binary random variables. In some recent exploration, I have been playing with inference for sequences of binary random ...
- 25.8k
0
votes
1
answer
30
views
Correlations significant at individual level, but not at group level?
For my research I have to do correlation tests to see how different measures of animal welfare are related to each other. When i run the test using the data at individual level, it shows a weak ...
- 91
0
votes
0
answers
12
views
What is the meaning of the inner product between two regression variables?
I have been analyzing the effect of design matrix columns on the contour line of the least squares regression. These contours obviously are ellipses when only two columns $\phi_1$ and $\phi_2$ are ...
- 11
0
votes
1
answer
25
views
Correlations between variables based on gender in R
I have a data set in which female is coded as 0 and male as 1 and then I have different testing scores; writing, reading, math. I want to determine whether the correlation between math and writing ...
- 1
0
votes
0
answers
18
views
Relationship between normally distributed errors (bivariate OLS regression) and bivariate normality of variables (Pearson’s r)
I have forty quite diverse statistical text books at hand but I cannot find a reliable answer.
I’ve studied the following and associated threads, but they didn’t got into it that deep: here and here. ...
0
votes
0
answers
21
views
Cumulative Distribution function of ratio of 2 correlated gamma distributed random variables
Let $X \sim \Gamma(k_x, \theta_x)$, $Y\sim \Gamma(k_y, \theta_y)$ be correlated with correlation coefficient $\rho$. The shape and scale parameters are different. i.e. $k_x \neq k_y$ and $\theta_x \...
2
votes
1
answer
34
views
What is the best way to represent Spearman’s correlation graphically?
For my thesis research I have to do Spearman’s correlation tests on a set of ordinal variables. I would like to represent the correlations graphically to include them in the report and give a clearer ...
- 91
0
votes
0
answers
7
views
statistical criteria for grouping questions in questionnaire data analysis and treat averaging the respond on these groups as continuous variable?
In questionnaire data analysis,
if you as a statistician received data after designing and filling the questionnaire by respondents, and encounter pre-determined groups of questions that are claimed ...
- 11
0
votes
0
answers
25
views
Change in Correlation after Imputation of Missing Outcome Data for Machine Learning Prediction Task
I am attempting to generate a predictive model for a continuous outcome using machine learning models. However, some observations in the original dataset have missing outcome data (and missing ...
- 5
1
vote
1
answer
20
views
Prediction based on correlation
I know -- correlation does not mean causation.
However, I need a "best guess" estimate for the following scenario:
Variable A = Starting Point: 4,5 (SD=0,5)
Variable B = Starting Point: 300 (...
1
vote
0
answers
16
views
Cross Correlation Techniques for Time Series Data [closed]
I am working with time series data and am currently using cross correlation on my data. I can use different techniques like spearman, pearson, and kendall.
I was wondering if beyond that, there are ...
- 11
1
vote
0
answers
21
views
How to calculate the value (wind direction) of "maximal effect" in a linear-circular correlation?
I am trying to evaluate the effect of wind direction (circular variable) on a dependent linear variable. I have used circular-linear regression to find the correlation coefficient between the two ...
- 21
0
votes
0
answers
25
views
Correlation matrix from pairwise correlations with specified structure
I need to simulate multivariate normal samples with a pre-specified correlation structure. The structure is such that the bigger the (GPS) distance between two points, the smaller the correlation (...
- 184
0
votes
0
answers
11
views
Should p.values be adjusted when comparing correlations of differently sized groups?
I have performed a series of correlations between several variables (all of them are continuous). Some of these correlations were conducted with the entire sample (which is composed of two independent ...
- 31
0
votes
0
answers
12
views
How to interpret an autocorrelation function (ACF) plot of residuals? [closed]
I have an autocorrelation plot of residuals. Does it show correlation? It is for a time series with monthly data.
Lexxie
1
vote
0
answers
50
views
Difference between y against x, x^2 and sqrt(y) against x in a linear model
Suppose that we have a response variable y which is known to have a quadratic relationship
with a predictor variable x. What are the differences between fitting a linear model of y
against x and x^2, ...
- 11
0
votes
1
answer
47
views
Categorical variable to be predicted from continuous variables with an idea: "maximise boxplots distance"
I am not sure the title is easy to understand, but this is because I'm struggling to find out if my question is machine-learning related, or just correlation-related, or anything else.
Context: I have ...
1
vote
0
answers
9
views
How to analysis the correlation of one value corresponding to multiple values
For example, there is a school teacher's answering score corresponding to multiple students' answering scores (which like 1 for unlikely, and 5 for very likely). For analyzing the correlation of this ...
- 11
0
votes
1
answer
31
views
Point-Biserial Correlation vs Pearson's Correlation
I know that when looking at the correlation between a binary and a continuous variable we should use point biserial correlation.
Today I was looking at some data and mistakenly used Pearson's. When I ...
- 49.4k
0
votes
0
answers
22
views
How to find what categorical variables explain a continuous dependent variable?
I am currently working on a statistical analysis for a project, I have a dependent variable ($y$) that represents delays in days, it's continuous and take integer values.
I also have a set of over 90 ...
- 1
0
votes
0
answers
15
views
Parametric or non-parametric if a variable is distributed normally and another non-normally distributed
I have 2 variables that I'd like to know if one (a concentration of a substance in a biological fluid) is dependent on the other (the percentage of inhibition of this fluid over the substance). I'd ...
1
vote
0
answers
20
views
Interpreting fisher’s Z transformation and correlations
Part of my planned analysis in a paper is testing the difference in strength of correlations between depression and other things like social relationships and quality of life. This test is set up with ...
4
votes
3
answers
120
views
Determine whether two binary sequences are correlated
I have two arrays of 2048 binary values (1 and 0) and I'd like to find out if their content can be considered correlated. I'd like to have less than 1:1000000 chance of a false positive in determining ...
- 93
0
votes
0
answers
23
views
How to perform a breakpoint analysis on these two variables (Survey Score and Total # of Outages)?
Here is a snippet of the data:
I believe both of these are discrete variables. I want to first determine if a correlation exists. Would I utilize Kendall’s Rank Correlation? Second, I want to create ...
1
vote
0
answers
13
views
Time series that are correlated in levels but not first differences
Is it correct to say that the two time series are correlated in levels, but not first differences?
$$x_{t}=\phi t+u_{t}\\ y_{t}=\rho t+\epsilon_{t}$$
where $\epsilon_{t}$ and $u_{t}$ are mutually ...
- 131
0
votes
0
answers
38
views
Which correlation test should I utilize to compare score data and total number of an event?
I have two fields in my dataset. The first field has records that return a final score (0-5) from the customer. The second field has records that show the total number of customer power outages.
I ...
0
votes
0
answers
13
views
Multivariate Time Series Clustering with Affinity Propagation
I'm trying to implement the clustering procedure described in the paper that follows at the link https://www.hindawi.com/journals/wcmc/2021/9915315/, but I get to the algorithm 2 described and I get ...
- 1
0
votes
0
answers
11
views
Estimate variable based on correlated random generated variable
I have two random variables $X$ (price of a commodity) and $Y$ (default rate of that commodity), that are correlated through $\rho$. $X$ follows a log-normal distribution and $Y$ has not enough ...
0
votes
0
answers
10
views
Is ARIMA the right model to use for the questions I am trying to answer?
I am looking for guidance or suggestions on the best model or method to solve the questions below.
My dataset is a time series that contains date, number of orders, and number of Customer Service (CS) ...
0
votes
0
answers
6
views
Correlation measure for ordinal values (values are ratings of different items)
I'm looking for the correct correlation metric for following problem:
Scale is from: 0-3 (0,1,2,3)
There are n items to rate by k raters (n and k are both small). I want to compare their ratings to ...
- 121
2
votes
0
answers
35
views
Can PCA provide contextual correlation between features and a target label?
I was exploring some of the projects I've done years ago in high school, and my current understanding of Principal Component Analysis differs from what I have performed back then.
Experiment: Our ...
- 221
1
vote
1
answer
28
views
Is it wrong to include the rows with all zero values to calculate correlation between two variables?
I have two variables: age and weight. I have a collection of documents in which these two variables may or may not appear. My dataset is two lists, one for each variable with values 0 or 1 to indicate ...
- 13
0
votes
0
answers
17
views
Relationship between regression coefficients in regression vs. reverse regression (with controls)
Question: How do the coefficients in a multiple linear regression (Regress y on x and controls Z) relate to the coefficients in the reverse regression (Regress x on y and controls Z)?
Simple Linear ...
