For plot 1, I can test the association between x and y by doing a simple correlation.
For plot 2, where the relationship is nonlinear yet there is a clear relation between x and y, how can I test the association and label its nature?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this communityFor plot 1, I can test the association between x and y by doing a simple correlation.
For plot 2, where the relationship is nonlinear yet there is a clear relation between x and y, how can I test the association and label its nature?
...the relationship is nonlinear yet there is a clear relation between x and y, how can I test the association and label its nature?
One way of doing this would be to fit $y$ as a semi-parametrically estimated function of $x$ using, for example, a generalized additive model and testing whether or not that functional estimate is constant, which would indicate no relationship between $y$ and $x$. This approach frees you from having to do polynomial regression and making sometimes arbitrary decisions about the order of the polynomial, etc.
Specifically, if you have observations, $(Y_i, X_i)$, you could fit the model:
$$ E(Y_i | X_i) = \alpha + f(X_i) + \varepsilon_i $$
and test the hypothesis $H_{0} : f(x) = 0, \ \forall x$. In R
, you can do this using the gam()
function. If y
is your outcome and x
is your predictor, you could type:
library(mgcv)
g <- gam(y ~ s(x))
Typing summary(g)
will give you the result of the hypothesis test above. As far as characterizing the nature of the relationship, this would be best done with a plot. One way to do this in R
(assuming the code above has already been entered)
plot(g,scheme=2)
If your response variable is discrete (e.g. binary), you can accommodate that within this framework by fitting a logistic GAM (in R
, you'd add family=binomial
to your call to gam
). Also, if you have multiple predictors, you can include multiple additive terms (or ordinary linear terms), or fit multivariable functions, e.g. $f(x,z)$ if you had predictors x, z
. The complexity of the relationship is automatically selected by cross validation if you use the default methods, although there is a lot of flexibility here - see the gam
help file if interested.
If the nonlinear relationship had been monotonic rank correlation (Spearman's rho) would be appropriate. In your example there is a clear small region where the curve changes from monotoncally increasing to montonically decreasing like a parabola would do at the point where the first derivative equals $0$.
I think if you have some modeling knowledge (beyond the empiricial information) where that change point occurs (say at $x=a$) then you can characterize the correlation as positive and use Spearman's rho on the set of $(x,y)$ pairs where $x < a$ to provide an estimate of that correlation and use another estimate of Spearman's correlation for $x>a$ where the correlation is negative. These two estimates then characterize the correlation structure between $x$ and $y$ and unlike a correlation estimate that would be near $0$ when estimated using all the data these estimates will both be large and opposite in sign.
Some might argue that just the empirical information (i.e. the observed $(x,y)$ pairs is enough to justify this.
You can test any kind of dependence by using distance correlation tests. See here for more informations about the distance correlation: Understanding distance correlation computations
And here the original paper: https://arxiv.org/pdf/0803.4101.pdf
In R this is implemented in the energy
package with the dcor.test
function.
Someone correct me if my understanding is wrong here but one way to deal with non- linear variables is to use a linear approximation. So, for example, taking log of exponential distribution should allow you to treat the variable as normal distribution. It may then be used to solve the problem like any linear regression.
I used to implement the general additive model to detect the non-linear relationship between two variables, but recently I've found out about the non-linear correlation implemented via nlcor
package in R, you can implement this method in the same way as Pearson correlation, the correlation coefficient is between 0 and 1 and not -1 and 1 as in Pearson correlation. A higher correlation coefficient implies the existence of a strong non-linear relationship. Let's assume two time series x2
and y2
, the nonlinear correlation between the two time series is tested as follows
install.packages("devtools")
library(devtools)
install_github("ProcessMiner/nlcor")
library(nlcor)
c <- nlcor(x2, y2, plt = T)
c$cor.estimate
[1] 0.897205
The two variables seem to be strongly correlated via nonlinear relationship, you can also obtain the adjusted p-value for the correlation coefficient
c$adjusted.p.value
[1] 0
You can also plot the results
print(c$cor.plot)
You can view this link for more details