2
$\begingroup$

I have 3 variables in my data set.

(i) My gut feel says variable1 and variable2 are correlated, only when variable3 >= threshold3. What is the technique I can use to see if this holds true, to statistical significance (simple regression based/machine learning)?

eg. Taking variables in Stock Market, I think stock price move is correlated to Price Momentum over the last 3o minutes, but only when Volatility Index is above a certain threshold. I do not know the threshold value and also do not know if the statement, that the above correlation is significant only when the threshold is met, holds. Looking for a technique that I can use to formally investigate.

(ii) The other scenario: My gut feel says variable1 and variable2 are correlated, after variable2 >= threshold2. What would be the technique to investigate this?

eg. Similar to the above example, when the correlation between two variables is significant only when one of the 2 variables is above a threshold in its space, looking to investigate if that is true and also estimate the threshold.

$\endgroup$
  • $\begingroup$ Can you say anything more about your situation and your data? Do you know the thresholds, or are the thresholds something you need to estimate from your data? $\endgroup$ – Silverfish Sep 17 '16 at 16:31
  • $\begingroup$ I am trying to estimate the thresholds, if infact there exists a significant correlation, above the threshold. Say, I take the example of some variables in stock market and think, price is correlated to volume, only when the volatility index is above a certain value, I am trying to estimate if there exists a threshold above which this correlation is significant. $\endgroup$ – user2956863 Sep 18 '16 at 4:30
  • 1
    $\begingroup$ It's best to edit clarifications into the question rather than post them as a comment. I've tried editing your last comment into the question for you $\endgroup$ – Silverfish Sep 19 '16 at 20:11
1
$\begingroup$

You could investigate your data visually with a conditioning plot, in R this is named coplot. There is an example in Best method to visualize large interaction between two factors or run example(coplot) within R.

Say your tree variables are $x,y,z$ and you suspect the relationship between $y$ and $x$ depends on the values of $z$. Then you would use the following command in R:

coplot(y ~ x | z, data=yourdataframe)

assuming your variables are in data frame yourdataframe.

Below an example of an conditioning plot produced by R:

example conditioning plot using R swiss dataset

The R code used was:

data(swiss)
with(swiss, coplot(Fertility ~ Education | Agriculture))

To read the plot: The order of the subplots is from bottom and from left, so the plot corresponding to the lowest vales of variable Agriculture is bottom left. Other examples of coplots can be found here: What is the physical significance of cumulative correlation coefficient? and Can I analyze or model a conditional correlation?

As for your second scenario: Here there is only two variables, so a usual scatterplot will do. You can add some nonparametric smooth to the plot, for a more formal approach see How to use segmented package to fit a piecewise linear regression with one breakpoint? and search this site for the tag .

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.