2
$\begingroup$

I'm trying to calculate the correlation of two variables, but the array is disjointed in the middle - but I'm trying to obtain one correlation coefficient.

See the excel file I uploaded.

Because the disjoint in the middle, there is somewhat a jump in value and when I correlate the whole set I obtain a correlation coefficient that is not really reflective of the relationship between the variables.

How would I be able to calculate one correlation coefficient? I can calculate two separate correlations but is there any way to sum two correlations?

$\endgroup$
  • 2
    $\begingroup$ Disclaimer: I didn't look at the file. Your link obliges download of an executable, and I don't want to do that. But you seem to have answered your question at the same time as asking it. You can and may calculate a correlation but in so far as it appears not to reflect the structure of the data, you are looking in the wrong direction. The way forward is to seek a model of the data that somehow uses the group structure that is evident to you. No single summary can do justice to such data. $\endgroup$ – Nick Cox Dec 19 '13 at 16:52
  • $\begingroup$ There seems to be something very odd about your link. $\endgroup$ – Glen_b Jan 21 '14 at 14:17
2
$\begingroup$

I am writing this assuming that fitting two separate correlations and combining them is an appropriate thing to do in the first place. I'm not sure whether or not it is, and the answer may depend on the specifics of your problem.

If you do two separate regressions, I would report the correlation coefficient, and slope for both separately. It might be best for you to end your analysis here, but since you asked I will give a tentative suggestion about how to combine them.

There is no way to get a global correlation coefficient, $r$, but it seems to me that you can calculate a coefficient of determination, $R^2$, which is probably what you are interested in anyways. In a typical data set (i.e. not disjoint):

$$R^2 = 1 - \frac{\sum_i (y_i -\hat{y}_i)^2}{\sum_i (y_i -\bar{y})^2} $$

Here, $\bar{y}$ is the mean of your dependent variable, $\hat{y}_i$ is what your model predicts for the dependent variable each observation, and $y_i$ is what you actually observed the dependent variable to be. The numerator in the above equation is the sum of squared residuals, which measures the variability that remains unaccounted for by your model. The denominator is the total sum of squares, which measures the total variability in the dataset before your model was considered.

In your case, it seems reasonable to me to calculate $R^2$ as:

$$R^2 = 1 - \frac{\sum_i (y_i -\hat{y}_i)^2 + \sum_j(y_j -\hat{y}_j)^2}{\sum_k (y_k -\bar{y})^2} $$

Essentially the total variability in your dataset is the same as before, but now you have two different models to consider. The first sum in the numerator is the sum of squared residuals for the first model (e.g. T1, T2, T3, in your excel file), and the second sum corresponds to the other set of data. That is, the variable $i$ indexes the first subset of your data, $j$ indexes the second subset, and $k$ indexes everything.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.