# How to decide if means of two sets are statistically significantly different given that sets are obtained by a split that maximises the difference?

I have a data set consisting of some number of pairs of real numbers. For example:

(1.2, 3.4), (3.2, 2.7), ..., (4.2, 1.0)


or

(x1, y1), (x2, y2), ..., (xn, yn)


I want to know if the second variable depends on the first one (it is known in advance that if there is a dependency, it is very weak, so it is hard to detect).

I split the data set into two parts using the first number (Xs). Then I use the mean of Ys for the first and the second sub-sets as "predictions". If find such a split that the squared deviation between the predictions and real values of Ys is minimal. Basically I do what is done by decision trees.

Now I wont to know if the found split and the corresponding difference between the two means is significant. I could use some standard test to check if the means of two sets are statistically significantly different but, I think, it would be incorrect because we did the split that maximise this difference. What would be the way to count for that?

• It is nearly always inadvisable to split, even moreso when dependency is weak, since it loses information, inflates variances and biases effects – Glen_b Apr 9 at 2:10
• @Glen_b, I need to split to have a "predictive model". With this simple "model" and can tell something about Y given X. However, in the end, I need to know if the model does something (if results are significant). – Roman Apr 9 at 7:18
• It sounds like you have 2 variables, $X$ and $Y$, with $n$ total observations, and you wish to uncover a hypothesiszed association between them. Is this correct ? – Robert Long Apr 11 at 8:48
• " the question is specifically about "predictive power" of the step function" - your question doesn't mention the step function at all. You question states "I want to know if the second variable depends on the first one" but then you talk about splitting the data without any reasoning why you do so. As stated by @Glen_b, this rarely makes sense. I think you need to add a lot more detail into the question about what you are doing, and why. – Robert Long Apr 12 at 10:12
• I would call that, subsetting, not a step function, anyway, thanks for clarifying that, but I note you have chosen not to explain why you are doing this. If you can provide the motivation behind the question it is easier for people to understand and give advice. – Robert Long Apr 12 at 11:11