1
$\begingroup$

I am trying to work out how good my particular model is at explaining some observed data. The problem here is that the observed data takes the form of averaged (mean) values for each of my predictive scores. When performing a simple correlation, I get a really high R-squared value (and this is repeated for independent data sets), which I am assuming means that there is a consistent relationship between my predictive value and the observed data. However, if I want to estimate how much of the variance in observed data I explain with my predictive score, how would I do this with averaged observed values? My underlying concern is that although I have a good correlation in all cases, there may be many underlying factors that also drive my observed data that are simply being 'averaged out' within each category, and as such, my r-squared value is meaningless.

Just as an example, if my underlying observed data was: x<-c(3,4,3,2,1,6,5,7,5,4,9,7,8,10,11) And my predictive score for these values is: y<-c(1,1,1,1,1,2,2,2,2,2,3,3,3,3,3)

I get r-squared of 0.911. However, if I average my observed data (which is all I can measure in my case), I would get: x1<-c((13/5),(27/5),9) y1<-c(1,2,3) R-squared = 0.997

Thus telling me that my predictive score explains almost all of the variation in the observed data, when a better representation above (x vs y) tells me I am explaining 91% of the variation.

Thanks.

$\endgroup$
4
  • $\begingroup$ Not that it impacts on your question but I think you are quoting r, not $R^2$. $\endgroup$ Feb 25, 2013 at 19:01
  • $\begingroup$ Do you mean to say that $x$ is an actual or observed value and $y$ is it's prediction. Further, are you interested in how well $y$ predicts $x$? Otherwise, I can't conceive of why you'd aggregate $x$ over values of $y$ and not the other way around. $\endgroup$
    – AdamO
    Feb 25, 2013 at 20:07
  • $\begingroup$ Peter - yes, sorry, I read the wrong part of the read out, but yes the question remains the same! AdamO - x is observed, and y is predicted, sorry that's not too conventional is it?! I am interested in how well the bottom vector predicts the top vector. The problem here is that I can't actually get at x as it is described in the first scenario, I can only get at it as an average across a number of my different predicted y values. $\endgroup$
    – Alan
    Feb 26, 2013 at 14:44
  • $\begingroup$ I guess an easy way to state it is that (I'll change the notation now to follow convention) my observed results are actually 0's and 1's across a large number of positions, so what I'm really comparing is an observed rate for each given predicted value: x<-c(1,1,1,1,1,1,2,2,2,2,2,3,3,3,3,3,3,3,4,4,4,4,4,4) - predicted, y<-c(0,0,0,1,0,0,0,0,1,1,0,0,0,0,0,1,1,0,1,0,0,0,0,0) - observed, I then calculate the rate that a 1 occurs in y for each given x value (hence what is essentially an average). Is there a way to then calculate how much of y is explained by x? $\endgroup$
    – Alan
    Feb 26, 2013 at 14:48

0

Browse other questions tagged or ask your own question.