# How to determine sample size for non-normal distributed data with continuous variables?

TLDR: I need to determine the minimum sample size for measuring the energy consumption of code.

Hello, I'm trying to determine how many samples (N) I need in an experiment to be confident that my data's mean and standard deviation are representatives of the population. Note, I'm not well versed in statistics, hence why I'm struggling to find the correct formula to use.

The experiment: I'm measuring the energy consumption of running a piece of code repeatably. My data set will contain N amount of samples, where each sample is a continuous variable (energy in joules). However, I need to calculate how many N samples I should acquire. If necessary, I can do an initial experiment to get an estimation of the standard deviation, mean, and such. Then use that to calculate whether I have enough samples or I need to acquire more.

I have previously tried to use Cochran's formula. However, I used the Shapiro-Wilk test and found out that some of my data is not normally distributed (I actually have multiple data sets with different pieces of code and the same code with different measuring instruments, but I would like to use a formula that works on the non-normally distributed data). If my understanding is correct, that means I should not use Cochran's formula.

I can (within reason) acquire as many samples as I want and as such am not limited by the population size or the difficulty in getting more samples, I would just like to calculate the minimum required samples, as to be able to argue why some I picked to run my experiments N times.

Goal/Plan: I'm comparing measurement of the same code, but measured by different measuring instruments as to compare them. I'm probably going to use the Mann-Whitney U Test to test if the test case measurement are independent of each other. Then I'm going to do a non-parametric rank correlation (Kendall's Tau coefficient) to calculate the correlation between the different measuring instruments and also our ground truth.

Non-normal distribution: I don't know what specific distributions the non-normal distributions are. I have done a normalcy test and discovered that while some of the data sets are normally distributed, several are not.

• "and found out that some of my data is not normally distributed", what does this "some" mean? Mar 2, 2023 at 13:40
• I have several experiments testing different code, i.e. several data sets. Some of the data sets are not normally distributed, while some of them are. However, I would like to use the same sample size formula on all the data sets if possible. Hope that cleared it up. Mar 2, 2023 at 13:43
• why not just run your experiment until you have your answers to a required level of precision? A sample size calculation requires that you specify that precision needed anyway, so since you don't have any practical limits I can't see the value in working it out beforehand. Mar 2, 2023 at 13:59
• The idea is to run an initial experiment for each setup, and using the initial data to calculate how many more samples are needed. Then use the maximum amount of needed samples for all the experiments. I wanted to this because I'm going to run a lot of automated experiments and I want to be able to argue why I chose to run the experiments N times. Mar 2, 2023 at 14:08
• It would help a lot if you could describe what your analysis is likely to be, and how you will interpret the data. Also what you mean by 'not normally distributed'. When we do sample size calculations we are trying to anticipate your analysis and predict what the uncertainty in your estimates is going to be. So the information about the analysis plan and the distribution of the pilot data is important. If you're worried about distributional assumptions you can run a sample size calc by simulation/bootstrapping. But as I said above I'm not sure you need one at all. Mar 2, 2023 at 16:14