Return to Answer

I didn't check your calculations but finding widely different standard deviations/sample size estimates from very small pilot studies is not surprising. Just like the sample mean, the sample standard deviation is a noisy estimate of its population counterpart. That's why it's not such a good idea to use a small pilot study to compute power or decide on sample size (even if it is, as you noted, standard advice). I discussed this further in these comments and, indirectly, in this answer to an unrelated question.

As to whether or not the pilot data would be included in the final data set, the usual answer would be negative but thinking about it I am not sure if I can see a compelling reason, at least not if you are not using the mean or difference but only the standard deviation in your sample size calculations. On a related note, collecting data and stopping when your CI reaches a certain width is in fact sound (whereas repeatedly testing and stopping when the mean is significantly different from some value is a big no-no). It seems related to the accuracy in parameter estimation approach, which might be relevant to you.

PS: Note that some of your statements about confidence intervals seems a little unclear at times. For example, by construction, 95% of them should contain the true mean, what increasing the sample size does is reduce their width, not change this frequency.

I didn't check your calculations but finding widely different standard deviations/sample size estimates from very small pilot studies is not surprising. Just like the sample mean, the sample standard deviation is a noisy estimate of its population counterpart. That's why it's not such a good idea to use a small pilot study to compute power or decide on sample size (even if it is, as you noted, standard advice). I discussed this further in these comments and, indirectly, in this answer to an unrelated question.

As to whether or not the pilot data would be included in the final data set, the usual answer would be negative but thinking about it I am not sure if I can see a compelling reason, at least not if you are not using the mean or difference but only the standard deviation in your sample size calculations. On a related note, collecting data and stopping when your CI reaches a certain width is in fact sound (whereas repeatedly testing and stopping when the mean is significantly different from some value is a big no-no). It seems related to the accuracy in parameter estimation approach, which might be relevant to you.

PS: Note that some of your statements about confidence intervals seems a little unclear at times. For example, by construction, 95% of them should contain the true mean, what increasing the sample size does is reduce their width, not change this frequency.

I didn't check your calculations but finding widely different standard deviations/sample size estimates from very small pilot studies is not surprising. Just like the sample mean, the sample standard deviation is a noisy estimate of its population counterpart. That's why it's not such a good idea to use a small pilot study to compute power or decide on sample size (even if it is, as you noted, standard advice). I discussed this further in these comments and, indirectly, in this answer to an unrelated question.

As to whether or not the pilot data would be included in the final data set, the usual answer would be negative but thinking about it I am not sure if I can see a compelling reason, at least not if you are not using the mean or difference but only the standard deviation. On a related note, collecting data and stopping when your CI reaches a certain width is in fact sound (whereas stopping when the mean is significantly different from some value is a big no-no). It seems related to the accuracy in parameter estimation approach, which might be relevant to you.

PS: Note that some of your statements about confidence intervals seems a little unclear at times. For example, by construction, 95% of them should contain the true mean, what increasing the sample size does is reduce their width, not change this frequency.

I didn't check your calculations but finding widely different standard deviations/sample size estimates from very small pilot studies is not surprising. Just like the sample mean, the sample standard deviation is a noisy estimate of its population counterpart. That's why it's not such a good idea to use a small pilot study to compute power or decide on sample size (even if it is, as you noted, standard advice). I discussed this further in these comments and, indirectly, in this answer to an unrelated question.

As to whether or not the pilot data would be included in the final data set, the usual answer would be negative but thinking about it I am not sure if I can see a compelling reason, at least not if you are not using the mean or difference but only the standard deviation in your sample size calculations. On a related note, collecting data and stopping when your CI reaches a certain width is in fact sound (whereas repeatedly testing and stopping when the mean is significantly different from some value is a big no-no). It seems related to the accuracy in parameter estimation approach, which might be relevant to you.

PS: Note that some of your statements about confidence intervals seems a little unclear at times. For example, by construction, 95% of them should contain the true mean, what increasing the sample size does is reduce their width, not change this frequency.

I didn't check your calculations but finding widely different standard deviations/sample size estimates from very small pilot studies is not surprising. Just like that sample mean, the sample standard deviation is a noisy estimate of its population counterpart. That's why it's not such a good idea to use them to compute power or plan a study (even if it is, as you noted, standard advice). I discussed this further in these comments and, indirectly, in this answer to an unrelated question.

As to whether or not the pilot data would be included in the final data set, the usual answer would be negative but thinking about it I am not sure if I can see a compelling reason, at least not if you are not using the mean or difference but only the standard deviation. On a related note, collecting data and stopping when your CI reaches a certain width is in fact sound (whereas stopping when the mean is significantly different from some value is a big no-no). It seems related to the accuracy in parameter estimation approach, which might be relevant to you.

PS: Note that some of your statements about confidence intervals seems a little unclear at times. For example, by construction, 95% of them should contain the true mean, what increasing the sample size does is reduce their width, not change this frequency.

I didn't check your calculations but finding widely different standard deviations/sample size estimates from very small pilot studies is not surprising. Just like the sample mean, the sample standard deviation is a noisy estimate of its population counterpart. That's why it's not such a good idea to use a small pilot study to compute power or decide on sample size (even if it is, as you noted, standard advice). I discussed this further in these comments and, indirectly, in this answer to an unrelated question.

As to whether or not the pilot data would be included in the final data set, the usual answer would be negative but thinking about it I am not sure if I can see a compelling reason, at least not if you are not using the mean or difference but only the standard deviation. On a related note, collecting data and stopping when your CI reaches a certain width is in fact sound (whereas stopping when the mean is significantly different from some value is a big no-no). It seems related to the accuracy in parameter estimation approach, which might be relevant to you.

PS: Note that some of your statements about confidence intervals seems a little unclear at times. For example, by construction, 95% of them should contain the true mean, what increasing the sample size does is reduce their width, not change this frequency.