# Ways to overcome small number of survey responses

For one of my academic project I ran a sample survey. But unfortunately the respondents were not very proactive & I get just 25 responses for my study. With this small response I guess I can't do any statistical studies (e.g. anova, regressions, etc).

So, I was wondering if based on the sample's mean & standard deviation I can simulate a larger dataset & then do my statistical analysis.

I'll also be grateful if somebody can tell me the method to do simulation with given mean & sd in R software.

Regards,
Ari

• You can do statistical studies with any amount of data, even just one response. The important question is whether there is any valid way to draw conclusions beyond the sample itself. When non-response rates are high, one usually cannot say much, if anything, no matter how sophisticated or extensive the analyses of the data may be. The usual dodge for academic projects is to acknowledge this and then explicitly to pretend the sample is representative and conduct the planned analyses anyway, regardless of how small the dataset may be. – whuber Jul 25 '11 at 14:20
• Also see stats.stackexchange.com/questions/726/… :-) – whuber Jul 25 '11 at 14:54
• @whuber: Thanks for your comment & the link. The link was really funny :) I read it before, but reading again was refreshing. I just want to add one ques regarding your comment. Suppose I have 3 data points & 4 variables & I was to do a regression analysis. Can I still do the analysis within this setting? I'm not worried about interpretation of the result. But is this analysis feasible? Hope I made clear myself! – Beta Jul 26 '11 at 10:18
• When there are more variables (including the constant) than data points you will usually get a perfect fit--assuming your software can handle collinearities, which is a mathematical, not a statistical, problem--so something can be done but it's not very informative and yields no information about error. Either collect more data or restrict the number of variables. – whuber Jul 26 '11 at 13:27
• Thanks Whuber!Your comment made things bit more clearer. – Beta Jul 26 '11 at 14:36

Simulating new data with the same mean/sd as your data and doing analysis on that is effectively the same as arbitrarily increasing your sample size while keeping your point estimates the same, thus increasing power, all while ignoring the fact that the mean/sd you've matched was based on only 25 observations - not a very diligent practice.

First off, if your data is normally distributed, then the $p$-values from ANOVA/regression are valid, since the sampling distribution of your estimates will still be normal, regardless of the sample size. More generally, you could consider non-parametric bootstrap resampling to get confidence intervals:

http://en.wikipedia.org/wiki/Bootstrapping_%28statistics%29

Essentially, you would re-sample from your data with and re-estimate a parameter (e.g. regression coefficients) repeatedly (say 1000 times) and treat these parameter estimates as draws from the sampling distribution of $\hat{\beta}$. You can then use the empirical quantiles (e.g. using the 2.5th and 97.5th percentiles) of this sample to construct confidence intervals.

As a side note, if the effects are large, and you can make a rational story from them you shouldn't over-emphasize the ability to formally "prove" your claims statistically just because your data is non-normal or your sample size is insufficient to invoke the central limit theorem.

• Thanks Macro! I took time to reply to your answer as I'm still grappling with the fact that bootstrapping really solve my problem. Once I understand the bootstraping & how it's solving my problem I'll mark yours (& Henry's) answer as correct answer. – Beta Jul 26 '11 at 10:21