0
$\begingroup$

I have developed a machine-vision program that calculates the diameter of trees from Google Streetview images and GPS data. In order to verify the accuracy of my results, I went out and physically measured the diameter of 18 trees and ran my program against the same trees.

Now, I'm looking to analyze these data. One idea that I had was to calculate the differences between the expected (from my measurements) and the experimental (from my program) diameters and use that difference to calculate the percentile accuracy for each sample point. From that, I calculated the mean and median accuracies.

I would appreciate any suggestions for further analysis that might be done on my data in order to test its accuracy!

$\endgroup$

1 Answer 1

1
$\begingroup$

First of all, beware that your 18 measurements sample looks small to get meaningful results from most of the analysis you might want to perform.

The answer should depend on what are you going to do with your data and your program. To use it in real world situations, I think the main goal is to be able to state the accuracy of future measurements.

To do this, I suggest:

  • Take a bigger sample of measurements, including trees of different sizes.
  • Compute absolute and percentile accuracy for each measurement.
  • Try to find out how accuracy is related to tree diameter or other relevant factor. Plotting percentile and absolute error against tree trunk might show that one of them is quite independent of tree size (I'd guess percentile accuracy) or a more complex relation would arise (that would need a bit more advanced tools).
  • If percentile (or absolute) accuracy don't change with tree diameter, you should find a bound for accuracy - e.g. our goal is to end with a statement like 99% of times our program measures tree diameter with an error of less than X%.
  • To find which accuracy bounds 99% of your errors, you could:
    • Compute it directly: take a big sample (e.g. 200 trees or more) and, find the 1% measurements with greatest errors. The minimum of that 1% bounds the 99% of the errors.
    • Or you could try to estimate the bound: Check if your errors are normally distributed (any statistical package could do that), and if they are, you can estimate that 3 (or 2.58) times standard deviation bounds 99% of errors.

There are more analysis you could perform in order to gather information to improve your model, like testing if the mean of your measurements matches the mean size of real trees, or analysing regression to find systematic biases.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.