I'm comparing 2 data sets from nearby locations to see if they differ significantly in air quality. I cannot perform any paired tests because the data at one location does not necessarily correspond to the data at the other location in time (i.e. one sample at location A was taken from Monday to Friday, another sample at location B was taken at from Tuesday to Saturday). In addition, I have some duplicate samples (2 samples taken over the same few days at the same location, but separate physical samples, not duplicate analyses of one sample) I have been advised to compare the CDF of the two locations. Is this good advice? If so, I can use the k-s test here. Are there any other tests I could use?
$\begingroup$
$\endgroup$
7
-
4$\begingroup$ This is terrible advice in general because at most locations air quality changes dramatically throughout the day. So, although you could compare the CDFs, the comparison would likely be meaningless or woefully misleading. Special circumstances could rescue you, though, depending on how these sample times were chosen and how they compare between the two datasets. Perhaps you could tell us more about that? $\endgroup$– whuber ♦Commented Jul 26, 2012 at 20:01
-
$\begingroup$ Edited to better describe the samples. $\endgroup$– rnorbergCommented Jul 26, 2012 at 20:10
-
$\begingroup$ Thanks. Are you now saying that these are time-composite samples, so that e.g. sample A represents a Monday-Friday total (or average)? Or is it that location A is sampled daily M-F and location B is sampled daily T-S? Also, what kinds of duplicates are these? Do they constitute physically separate samples taken at (essentially) the same location during the same time interval, or are they simply duplicate measurements of the same physical sample? $\endgroup$– whuber ♦Commented Jul 26, 2012 at 20:14
-
1$\begingroup$ The samples are composite samples taken over a few days that result in an average measurement for those days. The duplicate samples are separate physical specimens, not duplicate analyses of the same sample. $\endgroup$– rnorbergCommented Jul 26, 2012 at 20:21
-
1$\begingroup$ Sigh...That's good information but it means you're in a complicated situation. What you can do will depend on some of the details of your data, including how much you have and whether the composites always cover the same sets of days of the week or whether they cover varying days. In the first case your data will be difficult to compare but in the second you might be able to tease out the information you need. Would it be possible to post a small representative excerpt of your dataset? $\endgroup$– whuber ♦Commented Jul 26, 2012 at 21:06
|
Show 2 more comments
1 Answer
$\begingroup$
$\endgroup$
15
Bill Huber is an expert with spatial data and I think has given you good advice. Comparing CDFs may be too simplistic with spatial and temporal effects present and possibly different at the two locations. But there is also a certain amount of aggregation.
Having worked in industry for many years i know that if the bosses want things a certain way sometimes you have no choice but to give it to them that way. Just be careful to provide all the important caveats so that they don't misinterpret the results. Now your basic questions can be answered without getting into the nitty gritty details of the data.
If you have two data set there are a number of tests called empirical cdf tests because they compare the two sample cdfs and look for specific differences. The Kolmogorov-Smirnov test is perhaps the most well-known. It looks at the maximum absolute difference between the two cdfs over the entire range of the data. You can also create histograms of the data constructing the same bins for both data sets. There is a form of the chi-square test that can be used to see in the frequencies in the bins for one group is similar the the frequencies in the other. This can be done using contingency tables. For the contingency table approach there are exact permutation tests (e.g. Fisher's exact test) that also can be used.
answered Jul 26, 2012 at 21:36
-
$\begingroup$ I appreciate your faith in me :-). My biggest concern here is that air quality, especially near urban areas, undergoes a weekly cycle: weekends can be substantially cleaner than weekdays. The lack of comparability, then, of a M-F average with a T-S average can be important. I maintain--and believe the ASA supports this--that the statistician has an ethical duty to inform their boss/client of such concerns and their implications, even in some cases going so far as to say "it's not right to do the analysis you request." (I'm not even remotely suggesting your advice is unethical, though.) $\endgroup$– whuber ♦Commented Jul 26, 2012 at 21:43
-
$\begingroup$ @whuber Right on Bill! I wouldn't say that I was really giving advice. I really wanted to give a direct answer to a simple question that has an answer. The advice part of my answer was to provide the important caveats. I meant to present that in strong terms. Maybe he could even do a sensitivity analysis to show how the distributions could differ alot due to the confounding factors. Such as the M-F average versus the T-S. Maintaining ethics in the face of office pressure is important. But not providing the work your supervisor requests can cost you your job. $\endgroup$ Commented Jul 26, 2012 at 22:03
-
$\begingroup$ I think there could be several ways to handle this short of refusing to do the job that could lead to outcomes that are both satisfactory and ethical. $\endgroup$ Commented Jul 26, 2012 at 22:04
-
4$\begingroup$ You're right, there are. The important point to make is that we must look beyond just getting a result. The first step is to assess the situation to determine what can be accomplished and whether our clients' actions and decisions will be sensitive to these issues of data comparability and quality. A statistician who proposes procedures too early in this process, before the question and data are sufficiently well understood, potentially sidesteps all these considerations and may ultimately lose all involvement when the client runs off and implements those procedures themselves. $\endgroup$– whuber ♦Commented Jul 26, 2012 at 22:10
-
2$\begingroup$ (+1), and even more so because of the discussion following the answer. I'd love to see a question about this kind of statistical "dilemma" on CV. $\endgroup$– MånsTCommented Jul 27, 2012 at 7:39