I conducted a pilot study in which I measured a variable on 20 different days in order to document how time passage affects this variable. I want to plot the uncertainty for each day, but I have a very small, n = 6, sample. I have been told how pointless it is to make estimates from such a small sample, but I only want to take this as an exploratory non-definitive, however thorough, analysis of my data. Actually, each of the six data points that I have per day represents a summary statistic (i.e., median) of 16 observations collected daily per subject. My question is: if I take into account that each data point is constituted by several observations, Is it possible to add somehow weight or certainty to each data point (so that I might make estimates of the group more reliably)? Can you guide me to a method to do so? Or, else, is it exactly the same as if each data point was constituted by a single observation? I will be very grateful of any answer you provide.
-
$\begingroup$ Summarizing 6 observation from each day by taking the median will have made your daily numbers: (a) more nearly normal, (b) less variable. Not clear why you want to plot uncertainty for each day. As for any graphic, think about what truth you want to communicate to those who look at it. Maybe drop idea of showing daily data in detail, or just have an 'error bar' that goes from daily min to daily max. $\endgroup$– BruceETCommented Jul 23, 2018 at 17:27
-
$\begingroup$ Hi, @BruceET. Actually, I want to compare two factors. That is why I want to plot uncertainty. As an example, in session No. 12 the measured values for X factor were 2.9, 3.9, 3.4, 3.5, 2.17, and 4.3 for subjects 1, 2, 3, 4, 5, and 6, respectively; in the same session for the same subjects, values for Y factor were 1.9, .8, 2.8, 1.5, 1.1, and .81. I feel that plotting ranges (i.e., min-max), although more straightforward, would saturate much of the plot (particularly in earlier measurements). That is why I'm seeking for more narrow descriptives to account for variation. $\endgroup$– LepismatidaeCommented Jul 23, 2018 at 17:59
1 Answer
Plot the twenty X medians against time (days); connect the points with line segments. On the same axes, do the same for the twenty Y medians. If the segmented line for the X's lies (mainly) above the segmented line for the Y's, or if the two lines show clearly different trends, then maybe you have a graph that makes your main point without further embellishment. You can explain any anomalies by saying that each point is the median of six observations and thus subject to error. That is my recommendation. (I guess that means I think the answer to the question in the title is No--especially if "mitigate" means "hide.")
If you insist on some sort of vertical error bars around each median, then you are going to have a more cluttered graph no matter what scheme you use to get the error bars.
However, it does not seem completely impossible to give somewhat informative error bars based on only six X-observations (or Y-observations) each day. Anyhow, that is the impression I got from the bit of data you showed (Day 12).
sort(x); summary(x)
[1] 2.17 2.90 3.40 3.50 3.90 4.30
Min. 1st Qu. Median Mean 3rd Qu. Max.
2.170 3.025 3.450 3.362 3.800 4.300
The Wilcoxon one-sample signed rank test can give a 95% confidence interval (CI)
for the median: In R, wilcox.test(x, conf.int=T) returns (among other results)
the CI $(2.17, 4.30).$
sort(y); summary(y)
[1] 0.80 0.81 1.10 1.50 1.90 2.80
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.8000 0.8825 1.3000 1.4850 1.8000 2.8000
The Wilcoxon interval is $(0.8, 2.8).$
So you might use Wilcoxon CIs as error bars. This would require an explanation that each CI carries 95% confidence, so one has less than 95% confidence in conclusions reached looking at more than one CI. Roughly, you might have about 90% confidence comparing the two CIs for any one day. And there are likely to be many days for which the two CIs overlap--as they do for Day 12.
In the Day 12 data, I see no strong evidence that the data are not normal. Presumably, you used medians instead of means because you have some data for other days that 'misbehave'. If data were normal, then you could use t CIs. on Day 12, these are approximately $(0.7, 2.5)$ for Y and $(2.6, 4.2)$ for X (no overlap). The same precautions about drawing conclusions from comparisons of two or more CIs hold here.
Another possible scheme might be to show bars for each median that go from Q1 to Q3. For X on Day 12 this would be $(3.0, 3.8)$ and for Y $(.9, 1.8).$ You would have to stress that such bars are only a rough indication of variability and not confidence intervals. Perhaps these error bars would have a less "saturated" appearance. (Perhaps they might be misleading.)
One can always hope for less variation in measurements. But if you ever get it, that comes from taking more observations or from using less-variable methods, not from "fudging" the report at the end. So if you are going to use some sort of error bars, you will have to take care to explain how you got them and what they really mean.
I know error bars are very popular. But I have to say that, whether by design or happenstance, most of the error bars I have seen around measurements on graphs (in papers on psychology, education, environmental science, physics, etc.) give a false impression of variability.
