I developed a procedure to measure the geometric accuracy of 3D building models based on the similarity to a 3D point cloud. Therefore I created mainly two quality criteria. The result of my automatic method is an index for each building how well the model fits the point cloud. I am interested in how to validate this accuracy result and if this is necessary.
Is it possible to alter a test dataset that fits the point cloud according to the quality criteria so that it contains errors? I then would want to use this "faulty" test dataset to see whether the errors are correctly detected. I am afraid that this method could be argued as too biased because I am introducing the errors. Error creation would be done either manually by hand or automatically e.g. by a shifting algorithm that changes the models position.
Edit for clarification:
- 3D point cloud means a set of points, each with x-, y-, z-coordinate
- 3D-Model is a mesh model, represented by vertices, which are also points with x- y-, z-coordinate
I am looking on the building from the top view and arrange the outline inside a regular grid. For each grid cell, I am calculating the mean/median of z-coordinates inside. For the points of the building model as well as for the points of the point cloud. After I have this 2D representations in the xy-plane, I am calculating the differences between each cell. These are my "distance" measures. I now apply a threshold to the calculated distances to decide whether they are significant in terms of "too high". Without going more into details, each building gets assigned a percentage value, how many cells contain differences considered as erroneous. In fact I am using some more criteria, but those are not relevant for the question.
What I want to know: If I have a set of 3D building models, that are all correct in terms of my accuracy check (all differences below threshold). Is it valid to change the correct building models so they contain errors in terms of my threshold and use them to test my accuracy check? I want to validate my developed procedure if it is able to detect the erroneous buildings. Otherwise my percentage values are being calculated but it is not clear if the procedure is the right way to do it.