1
$\begingroup$

My model identifies objects with a specific quality in a large population. I tested it on a sample of 500 and found that it fails to identify about 7% of the objects (false negatives) due to some difficulties with the data which I cannot correct. This is a systematic measurement error, which should not affect regression analysis. But I also need to estimate the number of objects with this quality in the population. It seems logical to apply my model and increase the estimate by 7%. But this would introduce an error, as 7% was an estimate itself, and I do not know how to measure it. What would be the best way to do this?

$\endgroup$

1 Answer 1

1
$\begingroup$

You can create many, say $n$, test samples, each of size e.g. 500, like yours, and measure the error for each of them. This gives you a sample of size $n$ of your systematic error and from that, you can obtain, via the mean, an estimate of the systematic error (you currently have 7%), and the standard deviation will give you an estimate of the error of the systematic error. If you don't have many different samples but only the single one of size 500, you should use a bootstrapping approach.

$\endgroup$
2
  • $\begingroup$ Thank you. Suppose I take n samples of size m. In my example, m was 500, but I can change it. Is there a way to connect the error (I mean the error in measuring the systematic error) with n and m? $\endgroup$
    – Mikhail
    Aug 4 at 21:32
  • $\begingroup$ Presuming that those systematic errors are approximately normally distributed, the standard error of the mean reduces with $\sqrt{n}$. $\endgroup$
    – frank
    Aug 5 at 3:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.