Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 12 down vote favorite

For plot 1, I can test the association between x and y by doing a simple correlation. plot 1

For plot 2, where the relationship is nonlinear yet there is a clear relation between x and y, how can I test the association and label its nature? plot 2

share|improve this question

2 Answers

up vote 9 down vote accepted

...the relationship is nonlinear yet there is a clear relation between x and y, how can I test the association and label its nature?

One way of doing this would be to fit $y$ as a semi-parametrically estimated function of $x$ using, for example, a generalized additive model and testing whether or not that functional estimate is constant, which would indicate no relationship between $y$ and $x$. This approach frees you from having to do polynomial regression and making sometimes arbitrary decisions about the order of the polynomial, etc.

Specifically, if you have observations, $(Y_i, X_i)$, you could fit the model:

$$ E(Y_i | X_i) = \alpha + f(X_i) + \varepsilon_i $$

and test the hypothesis $H_{0} : f(x) = 0, \ \forall x$. In R, you can do this using the gam() function. If y is your outcome and x is your predictor, you could type: 

library(mgcv) 
g <- gam(y ~ s(x))

Typing summary(g) will give you the result of the hypothesis test above. As far as characterizing the nature of the relationship, this would be best done with a plot. One way to do this in R (assuming the code above has already been entered)

plot(g,scheme=2)

If your response variable is discrete (e.g. binary), you can accommodate that within this framework by fitting a logistic GAM (in R, you'd add family=binomial to your call to gam). Also, if you have multiple predictors, you can include multiple additive terms (or ordinary linear terms), or fit multivariable functions, e.g. $f(x,z)$ if you had predictors x, z. The complexity of the relationship is automatically selected by cross validation if you use the default methods, although there is a lot of flexibility here - see the gam help file if interested.

share|improve this answer
I prefer this approach to the two separate rank correlations either side of $x=a$ because it examines the relation as whole. It's also better than the parametric model, so I've accepted this instead. – user1447630 Sep 9 '12 at 17:40
@user1447630 This is a model for the relationship. Polynomial linear regression or nonlinear regression as well as additive models are ways to characterize a functionsl relationship. I could have mentioned any of those. But you asked for a measure of association, so I gave you other possible forms of correlatiom. As nice as Macro's answer might be it fits a functional relationship, but does not provide a measure of association. – Michael Chernick Sep 10 '12 at 1:51
1  
@Michael, I don't know which question you were reading but the OP asked how to test for an association, not for a measure of association. In any case, as nice as your answer may be, (and ignoring the fact that it requires rather strong a priori knowledge of where the change point occurs) I think it is overly tailored to the specific plot in this question, instead of the general "nonlinear association" problem. – Macro Sep 10 '12 at 12:26
2  
@Michael, your interpretation ("the question was about how to generalize the concept of correlation..") and the statistical "argument" you give (btw, "testing for siginicance of the coefficients and the model" in a linear model this is the same as testing a correlation) don't make any sense, given the facts, so I'm not going to address them. But, "..after you gave yours mine was unaccepted and yours was accepted ... I just felt that was not right" is campaigning for your answer which only reflects a fixation on rep points, not anything useful to the community. – Macro Sep 10 '12 at 15:53
4  
@Macro and Michael to me fitting a model of the relation between $x$ and $y$ in a semi/non-parametric way is one way of testing the association between the two. Such a test could be extended by measuring the extent of association with the different ways you've each suggested. I think both answers and the follow-up here have been quite useful to me, sans the ad hominem. However, since my question did include how we could "label its nature", which could be interpreted as model-fitting, I'm going to stick with Macro's answer. – user1447630 Sep 10 '12 at 16:28
show 2 more comments
up vote 4 down vote

If the nonlinear relationship had been monotonic rank correlation (Spearman's rho) would be appropriate. In your example there is a clear small region where the curve changes from monotoncally increasing to montonically decreasing like a parabola would do at the point where the first derivative equals $0$.

I think if you have some modeling knowledge (beyond the empiricial information) where that change point occurs (say at $x=a$) then you can characterize the correlation as positive and use Spearman's rho on the set of $(x,y)$ pairs where $x < a$ to provide an estimate of that correlation and use another estimate of Spearman's correlation for $x>a$ where the correlation is negative. These two estimates then characterize the correlation structure between $x$ and $y$ and unlike a correlation estimate that would be near $0$ when estimated using all the data these estimates will both be large and opposite in sign.

Some might argue that just the empirical information (i.e. the observed $(x,y)$ pairs is enough to justify this.

share|improve this answer
Essentially, I'd be splitting the x~y relation into two parts. Below x=a, the correlation by Spearman's rho is positive. Above x=a, the correlation by Spearman's rho is negative. I like this approach. However, is there also some way of parametrically testing whether the relationship between x and y fits an inverse parabola, i.e. $y = ax^2 + bx + c$, where $a$ is negative. Perhaps, this requires a custom statistical test? – user1447630 Sep 8 '12 at 12:04
1  
For that you fit the quadratic model using say OLS linear regression and do a standard statistical test that the coefficient a is greater than 0 (say a one-tailed t test). – Michael Chernick Sep 8 '12 at 12:15
1  
@Michael, regarding your last comment, if $x$ is not symmetrically distributed around zero, there could be large collinearity between $x$ and $x^2$. Since this will probably affect the standard $t$-test for a regression coefficient, a likelihood ratio test may be more appropriate, right? – Macro Sep 8 '12 at 15:07
@Macro I guess if there is a high degree of correlation between x and x$_2$ your suggestion may be good. But I do think it is common to apply individual t tests on coefficients even though covariates X$_1$ and X$_2$ may be correlated. High collinearity makes the coefficients very unstable because identifiability is almost lost. In such cases it may not make sense to do any kind of test on the coefficients of the covariates. – Michael Chernick Sep 8 '12 at 15:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.