How many measurements are needed to 'baseline' a measurement?

Question

When we are monitoring movements of structures we normally install monitoring points onto the structure before we do any work which might cause movement. This gives us chance to take a few readings before we start doing the work to 'baseline' the readings.

Quite often the data is quite variable (the variations in the reading can easily be between 10 and 20% of the fianl movement). The measurements are also often affected by the environment in which they are taken so one set of measurements taken on one project may not have the same accuracy as measurements on another project.

Is there any statisitcal method, or rule of thumb that can be applied to say how many baseline readings need to be taken to give a certain accuracy before the first reading is taken? Are there any rules of humb that can be applied to this situation?

Answer 1

I think you should look at power calculations. These are often used to decide the sample size of survey or clinical trial. Taken from wikipedia:

A priori power analysis is conducted prior to the research study, and is typically used to determine an appropriate sample size to achieve adequate power.

Answer 2

It really depends on the amount of variance relative to the size of a measurement that you care about. If you need to be able to tell the difference between a mean of 2 and a mean of 0, and your data look like this:

-4.4 3.8 -2.0 -5.1 0.2 7.1 0.9 -5.4 2.8 0.5

Then you're going to need a lot more data! But if you only care if the average is less than or more than 10, then that much data is adequate.

@cgillespie gives the technically correct response. You need to have some idea what size of effect you care about, as well as some idea how much variance your measurements have. If the equations of power analysis are more than you can deal with, you can always use random numbers in an Excel spreadsheet! Generate columns of random numbers with a normal distribution and various means and variances, then figure out whether the confidence intervals (2 * standard deviation / sqrt(N)) around the mean of each set of numbers include differences that you might are about. Do that a bunch of times. That'll give you a good idea of how many measurements you need, and it's not too hard to explain to others.

Answer 3

OK, so your data is very expensive to get. If you have some indication of the shape of the data then perhaps a bootstrap / bayesian / optimization (sticking in keywords :)) approach would work best. See the optim command in R as an example. You would need to know some things though. For example, could we assume something like normality? If so then fitting a small number of data points to the normal distribution will likely give you a much better estimate of your parameters than simple mean and sdev values.