For plot 1, I can test the association between x and y by doing a simple correlation.
For plot 2, where the relationship is nonlinear yet there is a clear relation between x and y, how can I test the association and label its nature?
$\begingroup$
$\endgroup$
Add a comment
|
5 Answers
$\begingroup$
$\endgroup$
8
...the relationship is nonlinear yet there is a clear relation between x and y, how can I test the association and label its nature?
One way of doing this would be to fit $y$ as a semi-parametrically estimated function of $x$ using, for example, a generalized additive model and testing whether or not that functional estimate is constant, which would indicate no relationship between $y$ and $x$. This approach frees you from having to do polynomial regression and making sometimes arbitrary decisions about the order of the polynomial, etc.
Specifically, if you have observations, $(Y_i, X_i)$, you could fit the model:
$$ E(Y_i | X_i) = \alpha + f(X_i) + \varepsilon_i $$
and test the hypothesis $H_{0} : f(x) = 0, \ \forall x$. In R, you can do this using the gam() function. If y is your outcome and x is your predictor, you could type:
library(mgcv)
g <- gam(y ~ s(x))
Typing summary(g) will give you the result of the hypothesis test above. As far as characterizing the nature of the relationship, this would be best done with a plot. One way to do this in R (assuming the code above has already been entered)
plot(g,scheme=2)
If your response variable is discrete (e.g. binary), you can accommodate that within this framework by fitting a logistic GAM (in R, you'd add family=binomial to your call to gam). Also, if you have multiple predictors, you can include multiple additive terms (or ordinary linear terms), or fit multivariable functions, e.g. $f(x,z)$ if you had predictors x, z. The complexity of the relationship is automatically selected by cross validation if you use the default methods, although there is a lot of flexibility here - see the gam help file if interested.
-
1$\begingroup$ I prefer this approach to the two separate rank correlations either side of $x=a$ because it examines the relation as whole. It's also better than the parametric model, so I've accepted this instead. $\endgroup$ Sep 9, 2012 at 17:40
-
$\begingroup$ @user1447630 This is a model for the relationship. Polynomial linear regression or nonlinear regression as well as additive models are ways to characterize a functionsl relationship. I could have mentioned any of those. But you asked for a measure of association, so I gave you other possible forms of correlatiom. As nice as Macro's answer might be it fits a functional relationship, but does not provide a measure of association. $\endgroup$ Sep 10, 2012 at 1:51
-
1$\begingroup$ @Michael, I don't know which question you were reading but the OP asked how to test for an association, not for a measure of association. In any case, as nice as your answer may be, (and ignoring the fact that it requires rather strong a priori knowledge of where the change point occurs) I think it is overly tailored to the specific plot in this question, instead of the general "nonlinear association" problem. $\endgroup$– MacroSep 10, 2012 at 12:26
-
3$\begingroup$ @Michael, your interpretation ("the question was about how to generalize the concept of correlation..") and the statistical "argument" you give (btw, "testing for siginicance of the coefficients and the model" in a linear model this is the same as testing a correlation) don't make any sense, given the facts, so I'm not going to address them. But, "..after you gave yours mine was unaccepted and yours was accepted ... I just felt that was not right" is campaigning for your answer which only reflects a fixation on rep points, not anything useful to the community. $\endgroup$– MacroSep 10, 2012 at 15:53
-
5$\begingroup$ @Macro and Michael to me fitting a model of the relation between $x$ and $y$ in a semi/non-parametric way is one way of testing the association between the two. Such a test could be extended by measuring the extent of association with the different ways you've each suggested. I think both answers and the follow-up here have been quite useful to me, sans the ad hominem. However, since my question did include how we could "label its nature", which could be interpreted as model-fitting, I'm going to stick with Macro's answer. $\endgroup$ Sep 10, 2012 at 16:28
$\begingroup$
$\endgroup$
5
If the nonlinear relationship had been monotonic rank correlation (Spearman's rho) would be appropriate. In your example there is a clear small region where the curve changes from monotoncally increasing to montonically decreasing like a parabola would do at the point where the first derivative equals $0$.
I think if you have some modeling knowledge (beyond the empiricial information) where that change point occurs (say at $x=a$) then you can characterize the correlation as positive and use Spearman's rho on the set of $(x,y)$ pairs where $x < a$ to provide an estimate of that correlation and use another estimate of Spearman's correlation for $x>a$ where the correlation is negative. These two estimates then characterize the correlation structure between $x$ and $y$ and unlike a correlation estimate that would be near $0$ when estimated using all the data these estimates will both be large and opposite in sign.
Some might argue that just the empirical information (i.e. the observed $(x,y)$ pairs is enough to justify this.
answered Sep 7, 2012 at 23:36
-
1$\begingroup$ Essentially, I'd be splitting the x~y relation into two parts. Below x=a, the correlation by Spearman's rho is positive. Above x=a, the correlation by Spearman's rho is negative. I like this approach. However, is there also some way of parametrically testing whether the relationship between x and y fits an inverse parabola, i.e. $y = ax^2 + bx + c$, where $a$ is negative. Perhaps, this requires a custom statistical test? $\endgroup$ Sep 8, 2012 at 12:04
-
1$\begingroup$ For that you fit the quadratic model using say OLS linear regression and do a standard statistical test that the coefficient a is greater than 0 (say a one-tailed t test). $\endgroup$ Sep 8, 2012 at 12:15
-
1$\begingroup$ @Michael, regarding your last comment, if $x$ is not symmetrically distributed around zero, there could be large collinearity between $x$ and $x^2$. Since this will probably affect the standard $t$-test for a regression coefficient, a likelihood ratio test may be more appropriate, right? $\endgroup$– MacroSep 8, 2012 at 15:07
-
$\begingroup$ @Macro I guess if there is a high degree of correlation between x and x$_2$ your suggestion may be good. But I do think it is common to apply individual t tests on coefficients even though covariates X$_1$ and X$_2$ may be correlated. High collinearity makes the coefficients very unstable because identifiability is almost lost. In such cases it may not make sense to do any kind of test on the coefficients of the covariates. $\endgroup$ Sep 8, 2012 at 15:24
-
$\begingroup$ how to measure both linear and non-linear relationship, and see the contributions of each, e.g. 40% linear, 60% non-linear? $\endgroup$ Aug 23, 2020 at 22:01
$\begingroup$
$\endgroup$
3
You can test any kind of dependence by using distance correlation tests. See here for more informations about the distance correlation: Understanding distance correlation computations
And here the original paper: https://arxiv.org/pdf/0803.4101.pdf
In R this is implemented in the energy package with the dcor.test function.
-
$\begingroup$ can this measure both linear and non-linear relationship, and discern the contributions of each, e.g. 40% linear, 60% non-linear? $\endgroup$ Aug 23, 2020 at 22:00
-
$\begingroup$ Yes, it also identifies non-linear contributions: en.wikipedia.org/wiki/Distance_correlation It does not discern the contributions as directly as you mentioned. $\endgroup$ Aug 24, 2020 at 12:48
-
$\begingroup$ which section of that long article talks about separating the two contributions? Edit to your edit: ok so it doesn't $\endgroup$ Aug 24, 2020 at 12:49
$\begingroup$
$\endgroup$
1
Someone correct me if my understanding is wrong here but one way to deal with non- linear variables is to use a linear approximation. So, for example, taking log of exponential distribution should allow you to treat the variable as normal distribution. It may then be used to solve the problem like any linear regression.
-
8$\begingroup$ I don't think this really answers the question. Taking logs isn't the same thing as doing a linear approximation. Moreover, even if you do take logs, the distribution for which the log of the original variable is a normally distributed variable isn't the exponential distribution but the lognormal distribution. However, neither the independent nor the dependent variable needs to be normally distributed for linear regression to be appropriate - the issue here is the relationship between the variables, not their marginal distributions. $\endgroup$ Aug 29, 2016 at 20:34
$\begingroup$
$\endgroup$
I used to implement the general additive model to detect the non-linear relationship between two variables, but recently I've found out about the non-linear correlation implemented via nlcor package in R, you can implement this method in the same way as Pearson correlation, the correlation coefficient is between 0 and 1 and not -1 and 1 as in Pearson correlation. A higher correlation coefficient implies the existence of a strong non-linear relationship. Let's assume two time series x2 and y2, the nonlinear correlation between the two time series is tested as follows
install.packages("devtools")
library(devtools)
install_github("ProcessMiner/nlcor")
library(nlcor)
c <- nlcor(x2, y2, plt = T)
c$cor.estimate
[1] 0.897205
The two variables seem to be strongly correlated via nonlinear relationship, you can also obtain the adjusted p-value for the correlation coefficient
c$adjusted.p.value
[1] 0
You can also plot the results
print(c$cor.plot)
You can view this link for more details
