Choosing a method for random sampling a small sample size economic dataset in R: fitting probability dist function or sample()

Question

I have a dataset of energy demand with a small sample size:

>dput(dat.demand2050.unique)    
c(79, 56, 69, 61, 53, 73, 72, 86, 75, 68, 74.2, 80, 65.6, 60, 54)  

with a density function that looks like this:

The data comes from a number of studies and can be separated into two "regimes" - low and high, which can be seen in the density function. However, I know that the data is biased toward a particular group of studies.

I want to generate a large number of random samples of data that will be an input to a simulation study. I am debating between a choice of three approaches:

1.Model the density function using the normalmixEM() function from the mixtools() package, which gives the following result: (related post here https://stackoverflow.com/questions/17924976/fitting-multimodal-distributions-in-r-generating-new-values-from-fitted-distrib)

the caveat being that I am forcing a strict normality assumption on my underlying process.

OR

2.Generating a random sample from a uniform distribution i.e assuming each observation is equi-probable:

NN = 1000 #a number  
set.seed(99)  
dat.demand2050.random <-    runif(NN,min=min(dat.demand2050.unique),max=max(dat.demand2050.unique))  

which gives the following density function:

densityplot(dat.demand2050.random)  

OR

3.Use sample() with replacement:

dat.demand2050.random2 <- sample(dat.demand2050.unique,NN,replace=TRUE)  
densityplot(dat.demand2050.random2)  

So my question is, what are the pros and cons of these approaches and what criteria should I apply for choosing the correct one?

Answer 1

You have 15 values here, and there are going to be different judgements here on what is sensible with such a small dataset, but here are some personal opinions.

Starting with the graphics:

• The first histogram can be decoded as showing bins with frequency 1, 2 or 3. Showing a density scale is not wrong in any absolute sense, but I'd want histograms for sample sizes this small to show frequencies directly. That's a matter of clear and simple communication rather than statistical logic.

• A density function being estimated here from 15 points using kernel estimation also falls into the category of not wrong, but likely to be regarded by technical readers as a real stretch with so few data. In any case, there is never a single possible kernel density estimate, even if a program produced one for you, so you need to worry more about the effects of different kernel choices, or not use the method.

• To the point here, the kernel density you show based on the raw data is unimodal, and really doesn't match the story you have of two regimes. On the other hand, that based on sampling with replacement is bimodal. The two really should be telling the same story! My guess is that you are relying on program default choices, which produces the anomaly. But there can't be more information in the bulked out data than you have originally.

• By far the easiest display to think about is the dot plot, strip plot or rug in the last graph above.

• Mixture modeling here is also a real stretch with this sample size. How to interpret your graph is a little unclear as I'd expect two normal density curves and a combined one.

I realise that these are (mostly) side-issues for you. My answer to your main question is that unless you have independent evidence for uniform or mixed normal distributions, any such assumption is arbitrary. You would need to spend space and time discussing either assumption. Bulking out the data by sampling with replacement is the least bad option if you really need to get much bigger samples. But it still seems dubious: bootstrapping what you have seems a much better way of exploring uncertainty.

All that said, there is some sampling process that led to these 15 values, but that is not discussed here beyond an admission that there is bias. As you know, none of your solutions removes that bias, not there is any obvious alternative.