0
$\begingroup$

I am performing a correlation analysis between LST values captured from Landsat 8, and combinations of MODIS Terra and Aqua observations. I am also comparing the MODIS datasets with LST values observed from 11 ground weather stations. I calculated the daily mean LST over an ROI for both sets of data (after preprocessing for high-quality pixels), plotted the scatter plot, fitted a linear regression model, and generated the coefficient of determination (r2) and Root Mean Squared Difference (RMSD) for all pairs of data. My question is, that I am getting p-values very close to zero ( < 0.01 ) in almost all regressions. In terms of the satellite data, I am getting a correlation of around 0.35 (+-1) for all sets of data, and for weather station comparison I am getting a correlation of 0.6-0.8.

How come the p-value is low even though the degree of correlation between the sets of satellite data is small? The number of data points is on average 300 higher than the weather station LST values.

Another question is what other tests can I implement to solidify the significance of the low p-values in both sets of data? What tests can I perform to check the accuracy of these statistics? I know this is a somewhat broad question but any point in the right direction will be appreciated.

Improve this question
$\endgroup$
6
  • $\begingroup$ You are doing something wrong, although it's hard to say what. Correlations range between -1 and 1. Also, I'm not clear what you mean by "the number of data points is 300 higher" - to do correlation you need equal numbers of points. $\endgroup$
    – Peter Flom
    Commented Aug 15, 2023 at 11:39
  • $\begingroup$ Yes as far as I understand that is Pearson's correlation. I am talking about the coefficient of determination R^2 which is the square of that correlation. In terms of the data points, I mean to say that for the satellite data, the total number 'n' of observations used for the linear regression model is higher than 'n' of the other comparison between LST and weather station data. For instance, the correlation of 'Terra Day - Terra Night' with LST had 1300 points, whereas with weather stations it had around 1000 points. This difference is almost same for all the combinations of Terra and Aqua. $\endgroup$
    – Ali Turab Hani
    Commented Aug 15, 2023 at 11:51
  • $\begingroup$ Well, $R^2$ ranges from 0 to 1, so, that's still a problem. $\endgroup$
    – Peter Flom
    Commented Aug 15, 2023 at 11:53
  • 1
    $\begingroup$ I'm sorry if I wasn't clear in my question. I have made a mistake in the values and I will edit the question. By 3.5 I mean 0.35 and by 6 -8 I mean 0.6-0.8. $\endgroup$
    – Ali Turab Hani
    Commented Aug 15, 2023 at 11:56
  • $\begingroup$ And let me clarify what I mean by sets of data as well. There are 11 combinations of mean MODS Terra and Aqua LST observations reduced over an ROI (E.g Terra Day -Terra Night, Terra Day-Aqua NIght, TerraDay-TerraNight-AquaDay and so on). I also have LST values recorded from weather stations. So, I take the LST values from one station, calculate the correlation with all combinations of terra and aqua (Joining them first by observation date) and repeat for all stations. Therefore I get 11 sets of correlation values and linear regression equations for each weather station. $\endgroup$
    – Ali Turab Hani
    Commented Aug 15, 2023 at 12:02

1 Answer 1

Reset to default
0
$\begingroup$

The reason is that you have a fairly large N (from your comments, it is ~1000) and, with that size sample, even a moderate correlation is highly significant. Here is some R code demonstrating this. (Anything after a # is a comment)

set.seed(1234)
    
x <- rnorm(1000)
y <- x + rnorm(1000,0, 2)
cor(x,y) # 0.49 sort of midway of your examples
cor.test(x,y)  # p = 2.2 e-16 (or 0.000000000000000016)
Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you. And is there any way to gauge the accuracy of the results I mentioned? Note: I used Python for the analysis so I can implement it over large datasets. Thanks $\endgroup$
    – Ali Turab Hani
    Commented Aug 15, 2023 at 12:08
  • $\begingroup$ You can get the standard errors of the correlations or the regression coefficients. I don't know Python at all, but I would be amazed if it can't do this. $\endgroup$
    – Peter Flom
    Commented Aug 15, 2023 at 12:10
  • $\begingroup$ Yes, I have generated the SE values as well. I wanted to ask how I can interpret them. If the SE and RMSD values are consistent does that signify anything? Also, any other advice that you would give keeping in mind the nature of the data? Sorry to flog you with questions but I have a deadline soon and I'm not really experienced in statistics. Thanks $\endgroup$
    – Ali Turab Hani
    Commented Aug 15, 2023 at 12:13
  • $\begingroup$ @AliTurabHani Please post your new questions as new questions. Unless some corners of the internet, Cross Validated is strictly Q&A, not a discussion forum. $\endgroup$
    – Dave
    Commented Aug 15, 2023 at 12:16
  • $\begingroup$ Okay. Anyway, thankyou for your help. $\endgroup$
    – Ali Turab Hani
    Commented Aug 15, 2023 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.