Questions tagged [correlation]

A measure of the degree of linear association among a pair of variables.

Filter by
Sorted by
Tagged with
0 votes
0 answers
4 views

Why would quartile plotted against percentile not be a straight line?

In the famous Dunning-Kruger paper perceived and actual scores are plotted, with quartile on the x axis and percentile on the y axis. The message is the perceived scores, and the actual scores are ...
user avatar
  • 101
1 vote
0 answers
18 views

If there is a high correlation between two variables, should we expect the high linear regression coefficient?

I have a data set with multiple features, let suppose $x_1,x_2,x_3,x_4$ and my dependent variable is $y$, when I compute the correlation matrix for $y,x_1,x_2,x_3,x_4$ then imagine the correlation ...
user avatar
  • 123
0 votes
0 answers
6 views

Assumptions for correlation disattenuation

I am familiar with the Spearman disattenuation formula for two random variables $X^\prime$ and $Y^\prime$ measured as $X$ and $Y$ with measured correlation $r_{xy}$ and reliabilities $r_{xx}$ and $r_{...
1 vote
0 answers
17 views

When does the Instrumental Variable (IV) method fail?

Consider the following ARMAX model: $$ y[k]=\alpha y[k-1]+\beta u[k-1]+e[k]-0.7e[k-1] $$ where $e[k]$ is a white noise. One can see that $e[k]-0.7e[k-1]$ is a filtered noise and thus using ordinary ...
user avatar
  • 111
0 votes
0 answers
48 views

auto-correlation and cross-correlation of multivariate random processes

Assume $N$ observations of a multivariate random process $X(t)$ of dimension $q$ arranged in a matrix as it follows, where each column represents a time-series and the $i$-th row is a realization of ...
user avatar
  • 101
0 votes
0 answers
21 views

Practical correlation metric for a large number of vectors

I am dealing with a timeseries consisting of input flow sampled every 5 minutes over 441 days. My aim is to find any possible correlation from data coming from: The same day of the week The same ...
user avatar
  • 11
0 votes
0 answers
5 views

Validate correlations with multiple influence factors

I am having a problem with validating my correlations. To explain the situation: Dataframe 1: the number of observations for each day in 2 years. Dataframe 2: multiple weather and air quality ...
user avatar
  • 101
0 votes
1 answer
37 views

Distribution of the Spearman rank correlation coefficient under the assumption of non-zero correlation

There are some papers and some R packages providing exact calculations for the CDF and the inverse-CDF of the (sample) Spearman rank correlation coefficient. My question is: How difficult would it be ...
user avatar
  • 649
0 votes
1 answer
29 views

Distribution function of the linear combination of standardized student-t quantiles [duplicate]

Assume to observe 2 quantiles, x and y, associated with the z% probability. These quantiles are generated by 2 non-independent standardized student-t distributions X and Y. In case of linear ...
user avatar
0 votes
0 answers
17 views

Interpretation of binomial regression in R

I'm running a regression to see if theres a positive or negative relationship between cvdrst and smkcigst, but I'm not sure how to interpret this regression also why is the data for smkcigst NA, even ...
user avatar
  • 1
2 votes
0 answers
23 views

Spearman Correlation for multiple variables

I want to characterize the correlation of a set of variables with a target variable using Spearman correlation since I expect the relationship to be monotonic but not necessarily linear. Specifically, ...
user avatar
  • 121
0 votes
1 answer
15 views

The correlation coefficient with a priori intercept and slope [closed]

What does the formula for the product moment correlation coefficient look like if the intercept and the slope are given a priori?
user avatar
  • 1
1 vote
1 answer
21 views

How do I test that two samples don’t differ significantly by their demographics?

I am conducting a study where I am randomly assigning people to different conditions. Because I am recruiting from a representative sample in the US, there will be some variability in the demographics ...
user avatar
  • 125
0 votes
0 answers
3 views

Serial correlation fix

I have some question about positive serial correlation: What happens to the standard error of the model once transformed using generalized differencing/Cochrane-Orcutt? Is it higher than the original ...
user avatar
0 votes
0 answers
13 views

How to perform variable/feature selection before random forest in R?

I have a phyloseq object with 4000+ taxa and 300+ samples. What methods/packages can I use to perform feature selection to reduce the number of taxa/OTUs prior to performing random forest? I was ...
user avatar
0 votes
1 answer
24 views

Is the correlation coefficient of the residuals related to the correlation coefficient of the original distirbution?

I have 10 different 2D datasets which each have 200 datapoints. Each data point has a 'quality' property ($Q$) which somewhat depends on the area ($A$) of that datapoint. There is a (relatively) ...
user avatar
0 votes
0 answers
16 views

Testing equality of coefficients from two different samples

I have the regression statistics for the same regression run on two different samples, and am asked to explain whether it is possible to test for equality of the coefficents, $\beta_1$and $\beta_2$ ...
user avatar
0 votes
1 answer
51 views

Conditional probability for autocorrelated binary random variables

In this answer, @whuber outlines conditional probabilities for correlated binary random variables. In some recent exploration, I have been playing with inference for sequences of binary random ...
user avatar
0 votes
1 answer
30 views

Correlations significant at individual level, but not at group level?

For my research I have to do correlation tests to see how different measures of animal welfare are related to each other. When i run the test using the data at individual level, it shows a weak ...
user avatar
  • 91
0 votes
0 answers
12 views

What is the meaning of the inner product between two regression variables?

I have been analyzing the effect of design matrix columns on the contour line of the least squares regression. These contours obviously are ellipses when only two columns $\phi_1$ and $\phi_2$ are ...
user avatar
  • 11
0 votes
1 answer
25 views

Correlations between variables based on gender in R

I have a data set in which female is coded as 0 and male as 1 and then I have different testing scores; writing, reading, math. I want to determine whether the correlation between math and writing ...
user avatar
0 votes
0 answers
18 views

Relationship between normally distributed errors (bivariate OLS regression) and bivariate normality of variables (Pearson’s r)

I have forty quite diverse statistical text books at hand but I cannot find a reliable answer. I’ve studied the following and associated threads, but they didn’t got into it that deep: here and here. ...
user avatar
0 votes
0 answers
21 views

Cumulative Distribution function of ratio of 2 correlated gamma distributed random variables

Let $X \sim \Gamma(k_x, \theta_x)$, $Y\sim \Gamma(k_y, \theta_y)$ be correlated with correlation coefficient $\rho$. The shape and scale parameters are different. i.e. $k_x \neq k_y$ and $\theta_x \...
user avatar
2 votes
1 answer
34 views

What is the best way to represent Spearman’s correlation graphically?

For my thesis research I have to do Spearman’s correlation tests on a set of ordinal variables. I would like to represent the correlations graphically to include them in the report and give a clearer ...
user avatar
  • 91
0 votes
0 answers
7 views

statistical criteria for grouping questions in questionnaire data analysis and treat averaging the respond on these groups as continuous variable?

In questionnaire data analysis, if you as a statistician received data after designing and filling the questionnaire by respondents, and encounter pre-determined groups of questions that are claimed ...
user avatar
0 votes
0 answers
25 views

Change in Correlation after Imputation of Missing Outcome Data for Machine Learning Prediction Task

I am attempting to generate a predictive model for a continuous outcome using machine learning models. However, some observations in the original dataset have missing outcome data (and missing ...
user avatar
  • 5
1 vote
1 answer
20 views

Prediction based on correlation

I know -- correlation does not mean causation. However, I need a "best guess" estimate for the following scenario: Variable A = Starting Point: 4,5 (SD=0,5) Variable B = Starting Point: 300 (...
user avatar
1 vote
0 answers
16 views

Cross Correlation Techniques for Time Series Data [closed]

I am working with time series data and am currently using cross correlation on my data. I can use different techniques like spearman, pearson, and kendall. I was wondering if beyond that, there are ...
user avatar
1 vote
0 answers
21 views

How to calculate the value (wind direction) of "maximal effect" in a linear-circular correlation?

I am trying to evaluate the effect of wind direction (circular variable) on a dependent linear variable. I have used circular-linear regression to find the correlation coefficient between the two ...
user avatar
0 votes
0 answers
25 views

Correlation matrix from pairwise correlations with specified structure

I need to simulate multivariate normal samples with a pre-specified correlation structure. The structure is such that the bigger the (GPS) distance between two points, the smaller the correlation (...
user avatar
  • 184
0 votes
0 answers
11 views

Should p.values be adjusted when comparing correlations of differently sized groups?

I have performed a series of correlations between several variables (all of them are continuous). Some of these correlations were conducted with the entire sample (which is composed of two independent ...
user avatar
  • 31
0 votes
0 answers
12 views

How to interpret an autocorrelation function (ACF) plot of residuals? [closed]

I have an autocorrelation plot of residuals. Does it show correlation? It is for a time series with monthly data.
user avatar
1 vote
0 answers
50 views

Difference between y against x, x^2 and sqrt(y) against x in a linear model

Suppose that we have a response variable y which is known to have a quadratic relationship with a predictor variable x. What are the differences between fitting a linear model of y against x and x^2, ...
user avatar
0 votes
1 answer
47 views

Categorical variable to be predicted from continuous variables with an idea: "maximise boxplots distance"

I am not sure the title is easy to understand, but this is because I'm struggling to find out if my question is machine-learning related, or just correlation-related, or anything else. Context: I have ...
user avatar
1 vote
0 answers
9 views

How to analysis the correlation of one value corresponding to multiple values

For example, there is a school teacher's answering score corresponding to multiple students' answering scores (which like 1 for unlikely, and 5 for very likely). For analyzing the correlation of this ...
user avatar
0 votes
1 answer
31 views

Point-Biserial Correlation vs Pearson's Correlation

I know that when looking at the correlation between a binary and a continuous variable we should use point biserial correlation. Today I was looking at some data and mistakenly used Pearson's. When I ...
user avatar
  • 49.4k
0 votes
0 answers
22 views

How to find what categorical variables explain a continuous dependent variable?

I am currently working on a statistical analysis for a project, I have a dependent variable ($y$) that represents delays in days, it's continuous and take integer values. I also have a set of over 90 ...
user avatar
0 votes
0 answers
15 views

Parametric or non-parametric if a variable is distributed normally and another non-normally distributed

I have 2 variables that I'd like to know if one (a concentration of a substance in a biological fluid) is dependent on the other (the percentage of inhibition of this fluid over the substance). I'd ...
user avatar
1 vote
0 answers
20 views

Interpreting fisher’s Z transformation and correlations

Part of my planned analysis in a paper is testing the difference in strength of correlations between depression and other things like social relationships and quality of life. This test is set up with ...
user avatar
4 votes
3 answers
120 views

Determine whether two binary sequences are correlated

I have two arrays of 2048 binary values (1 and 0) and I'd like to find out if their content can be considered correlated. I'd like to have less than 1:1000000 chance of a false positive in determining ...
user avatar
0 votes
0 answers
23 views

How to perform a breakpoint analysis on these two variables (Survey Score and Total # of Outages)?

Here is a snippet of the data: I believe both of these are discrete variables. I want to first determine if a correlation exists. Would I utilize Kendall’s Rank Correlation? Second, I want to create ...
user avatar
1 vote
0 answers
13 views

Time series that are correlated in levels but not first differences

Is it correct to say that the two time series are correlated in levels, but not first differences? $$x_{t}=\phi t+u_{t}\\ y_{t}=\rho t+\epsilon_{t}$$ where $\epsilon_{t}$ and $u_{t}$ are mutually ...
user avatar
  • 131
0 votes
0 answers
38 views

Which correlation test should I utilize to compare score data and total number of an event?

I have two fields in my dataset. The first field has records that return a final score (0-5) from the customer. The second field has records that show the total number of customer power outages. I ...
user avatar
0 votes
0 answers
13 views

Multivariate Time Series Clustering with Affinity Propagation

I'm trying to implement the clustering procedure described in the paper that follows at the link https://www.hindawi.com/journals/wcmc/2021/9915315/, but I get to the algorithm 2 described and I get ...
user avatar
  • 1
0 votes
0 answers
11 views

Estimate variable based on correlated random generated variable

I have two random variables $X$ (price of a commodity) and $Y$ (default rate of that commodity), that are correlated through $\rho$. $X$ follows a log-normal distribution and $Y$ has not enough ...
user avatar
0 votes
0 answers
10 views

Is ARIMA the right model to use for the questions I am trying to answer?

I am looking for guidance or suggestions on the best model or method to solve the questions below. My dataset is a time series that contains date, number of orders, and number of Customer Service (CS) ...
user avatar
0 votes
0 answers
6 views

Correlation measure for ordinal values (values are ratings of different items)

I'm looking for the correct correlation metric for following problem: Scale is from: 0-3 (0,1,2,3) There are n items to rate by k raters (n and k are both small). I want to compare their ratings to ...
user avatar
  • 121
2 votes
0 answers
35 views

Can PCA provide contextual correlation between features and a target label?

I was exploring some of the projects I've done years ago in high school, and my current understanding of Principal Component Analysis differs from what I have performed back then. Experiment: Our ...
user avatar
  • 221
1 vote
1 answer
28 views

Is it wrong to include the rows with all zero values to calculate correlation between two variables?

I have two variables: age and weight. I have a collection of documents in which these two variables may or may not appear. My dataset is two lists, one for each variable with values 0 or 1 to indicate ...
user avatar
0 votes
0 answers
17 views

Relationship between regression coefficients in regression vs. reverse regression (with controls)

Question: How do the coefficients in a multiple linear regression (Regress y on x and controls Z) relate to the coefficients in the reverse regression (Regress x on y and controls Z)? Simple Linear ...
user avatar

1
2 3 4 5
116