7
$\begingroup$

I analyze a set of multivariate measurements. It is known that several pairs of independent variables show high linear correlation. The graph below shows a scatterplot of one such pair (X and Y, upper pane), as well the residuals as a function of Y (lower left pane) and the histogram of these residuals (lower right pane)

data example

As one can see, there is a strange peak in the residual histogram. Many of the remaining linearly-dependent variable pairs from the same data set have a similar peak. I have double checked, and I'm sure there are no duplicate records in the data set. What might be the reason for such a behaviour?

PS Please don't ask me to elaborate on the problem domain, I'm not allowed to.

$\endgroup$
3
  • 2
    $\begingroup$ I am tempted to say that this strange peak is the more related to poor binning algorithm of histogram than to the features of data. Besides that since no context is given there is a lot of possible answers, but writing them up would not be interesting. $\endgroup$
    – mpiktas
    May 4, 2011 at 12:14
  • $\begingroup$ @mpiktas: actually, the binning algorithm is not guilty here. Wolfgang was right, there was a strong bias in one category of measurements. Problem solved $\endgroup$ May 4, 2011 at 12:32
  • $\begingroup$ cheers, you got lucky :) $\endgroup$
    – mpiktas
    May 4, 2011 at 13:38

1 Answer 1

9
$\begingroup$

What is the value of the residual that shows such a high count? It does not appear to be zero (slightly to the right of 0), so maybe 1? In any case, there may be something about that value that may provide you with some insight about the underlying mechanism. For example, if X and Y are measurements taken by observers, some of them may have a tendency to follow a certain pattern (i.e., Joe thinks: "everybody knows that Y is always 1 point higher than X" and "observes" accordingly), leading to such results. Without domain-level knowledge though, it is difficult to guess what is going on here.

$\endgroup$
1
  • $\begingroup$ Wolfgang, you were as close to the problem as possible without knowing the problem domain. Thank you $\endgroup$ May 4, 2011 at 12:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.