6
$\begingroup$

thanks in advance for anyone taking the time to read/answer this.

I am comparing a ground-sourced dataset versus a satellite-sourced dataset for weather conditions, such as temperature. Both sets are time series data (ground takes a reading every 15 minutes while satellite is every 30 minutes, so there are twice as many data points from the ground data). I want to compare the difference between these two sets at each entry, to see if my ground data fits my satellite data to a statistical significance of whatever.

For example, a few entries for temperature would look like:
enter image description here

Basically, I am not sure which statistical test/method to use to determine if my ground data is 'good', or close enough to my satellite data, or how 'different' they are between each other to the point of inaccurate data.

The reason I ask is that the satellite data is much more reliable versus the ground data from some of the sites I am looking at, where sensor malfunctions plague my ground data sets. In essence, I want to automate this process in mathematica so that it will tell me if a dataset is worth using or not.

Thanks for all your help!

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Is all of your temperature data for a single location or are there multiple locations in your dataset and you were just showing a few of the variables above? $\endgroup$
    – StatsStudent
    Commented Feb 26, 2016 at 23:05
  • $\begingroup$ Yes all my data is from from one location over a 4 year period. I was just showing a small sample of the sat an ground data over an hour period of the same location. $\endgroup$
    – sanjayr
    Commented Feb 26, 2016 at 23:08
  • $\begingroup$ i think you can use a CDF for the values, i am making the same thing with data of Temperature and radiaton, i hope this work for you.. Best regards $\endgroup$
    – Ana Sophia Altamirano
    Commented Apr 21, 2019 at 5:43

1 Answer 1

Reset to default
2
$\begingroup$

If you don't have concern about the accuraccy degrading over time or don't have concerns that the time of day results in less accurate measurements then I would advocate simplicity here through the use of a paired-sample t-test. You have completely missing data for the :15 and :45 intervals, so I'd throw those measurements away as you have nothing to compare them against from Satellite measurements. Then, with the remaining data, take the differences between the satellite measurement and the ground measurements, $y_{diff}=y_{satellite}-y_{ground}$. Then do a simple t-test on $y_{diff}$ to determine if $H_0:y_{diff}=0$ can be rejected at your desired level of confidence.

If there are temporal concerns, I'd take a look at building time-series type model for analyses.

Improve this answer
$\endgroup$
7
  • $\begingroup$ What do I do if there is a difference of 3.5 between the means of the datasets? $\endgroup$
    – sanjayr
    Commented Feb 29, 2016 at 21:27
  • $\begingroup$ 3.5 is meaningless without some sort of variance measure. Did you actually perform a statistical test? What were the results? $\endgroup$
    – StatsStudent
    Commented Feb 29, 2016 at 22:14
  • $\begingroup$ Sorry about that. For the paired t-test, I got a p-value of 1.014 x 10^-22 and a statistic of 9.824. $\endgroup$
    – sanjayr
    Commented Feb 29, 2016 at 23:43
  • $\begingroup$ Wow. That's a very small p-value which definitely rejects the null hypothesis and suggests strong evidence that there the two measurements are quite different. I would stick with the more reliable of the two measurements in this case (the satellite data). You should also double-check the calculations. $\endgroup$
    – StatsStudent
    Commented Feb 29, 2016 at 23:45
  • $\begingroup$ Another question that is important to answer here is how tolerable can your error be. Just because the results is statistically significant doesn't mean the results are practically useful. $\endgroup$
    – StatsStudent
    Commented Feb 29, 2016 at 23:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.