I recently had a client come to me to do a bootstrap analysis because an FDA reviewer said that their error-in-variables regression was invalid because when pooling data from sites the analysis include pooling data from three sites where two sites included some samples that were the same.

The client had a new assaying method they want to show is "equivalent" to an existing approved method. Their approach was to compare the results of both methods applied to the same samples. Three sites were used to do the testing. Error-in-variables (Deming regression) was applied to the data at each site. The idea is that if the regression showed the slope parameter to be close to 1 and the intercept near 0 this would show that the two assaying techniques gave nearly the same results and hence the new method shhould be approved. At site 1 they had 45 samples given them 45 paired observations. Site 2 had 40 samples and site 3, 43 samples. They did three separate Deming regressions (assuming a ratio of 1 for the measurement errors for the two methods). So the algorithm minimizes the sum of squared perpendicular distances. Separate regressions were done at each site and a pooled regression was also done using all the data from all three sites.

At that point since the client did not know how to do bootstrap I was brought in. The term interference was strange and I was not sure exactly what the reviewer was getting at. I assumed that the point really was that the becuase the pooled data had common samples there would be correlation for the common samples and hence the model error terms would not all be independent.

With the results being so similar at all three sites pooling the site data seemed reasonable. The client did a pooled Deming regression which also lead to similar results. Given these results I wrote a report for the client disputing the claim that the regressions were invalid. My argument is that because there are similar measurement errors in both variables th client was right to use Deming regression as a way to show agreement/disagreement. The individual site regressions had no problems of correlated errors because no samples were repeated within a given site. Pooling data to get tighter confidence intervals. The pooling which included use of the the common samples twice might produce positive correlation between the residuals for those common samples which would mean that the confidence intervals for the regression parameters would be too narrow (the estimated model residual variance biased on the low side).

This difficulty could be remedied by simply pooling the data with the common samples from site 1 say left out. Also the three individual site models do not have the problem and are valid. This seems to me to provide strong evidence of agreement even without the pooling. Furthermore the measurements were taken independently at sites 1 and 2 for the common sites. So I think that even the pooled analysis using all the data is valid because the measurement errors for a sample at site 1 are not correlated with the measurement errors in the corresponding sample at site 2. This really just amounts to repeating a point in the design space which should not be a problem. It does not create correlation/"interference".

In my report I wrote that a bootstrap analysis was unnecessary because their is no correlation to adjust for. The three site models were valid (no possible "interference" within sites) and a pooled analysis could be done removing the common samples at site 1 when doing the pooling. Such a pooled analysis could not have an interference problem. A bootstrap adjustment would not be necessary because there is not bias to adjust for.

A) Do you agree with (1) My analysis of the clients results and (2) my argument that the bootstrap is unnecessary.

EDIT: Given the suggestion of Bill Huber I plan to look at bounds on the error-in-variables regression by regression both y on x and x on y. We already know that for one version of OLS the answer is essentially the same as error-in-variables when the two error variances are assumed to be equal. If the is true for the other regression then I think that will show that the Deming regression gives an appropriate solution. Do you agree??

I plan to use the R program that whuber suggested in his answer to enable me to bootstrap the Deming regression. I am not very familiar with R but I think I can do it. I have R installed along with R Studio. Will that make it easy enough for a novice like me?

I recently had a client come to me to do a bootstrap analysis because an FDA reviewer said that their errors-in-variables regression was invalid because when pooling data from sites the analysis include pooling data from three sites where two sites included some samples that were the same.

The client had a new assaying method they wanted to show was "equivalent" to an existing approved method. Their approach was to compare the results of both methods applied to the same samples. Three sites were used to do the testing. Errors-in-variables (Deming regression) was applied to the data at each site. The idea is that if the regression showed the slope parameter to be close to 1 and the intercept near 0 this would show that the two assaying techniques gave nearly the same results and hence the new method should be approved. At site 1 they had 45 samples giving them 45 paired observations. Site 2 had 40 samples and site 3, 43 samples. They did three separate Deming regressions (assuming a ratio of 1 for the measurement errors for the two methods). So the algorithm minimized the sum of squared perpendicular distances. Separate regressions were done at each site and a pooled regression was also done using all the data from all three sites.

At that point since the client did not know how to do the bootstrap I was brought in. The term interference was strange and I was not sure exactly what the reviewer was getting at. I assumed that the point really was that the because the pooled data had common samples there would be correlation for the common samples and hence the model error terms would not all be independent.

With the results being so similar at all three sites pooling the site data seemed reasonable. The client did a pooled Deming regression which also lead to similar results. Given these results I wrote a report for the client disputing the claim that the regressions were invalid. My argument is that because there are similar measurement errors in both variables the client was right to use Deming regression as a way to show agreement  / disagreement. The individual site regressions had no problems of correlated errors because no samples were repeated within a given site. Pooling data to get tighter confidence intervals. The pooling which included use of the the common samples twice might produce positive correlation between the residuals for those common samples which would mean that the confidence intervals for the regression parameters would be too narrow (the estimated model residual variance biased on the low side).

This difficulty could be remedied by simply pooling the data with the common samples from site 1 say left out. Also the three individual site models do not have the problem and are valid. This seems to me to provide strong evidence of agreement even without the pooling. Furthermore the measurements were taken independently at sites 1 and 2 for the common sites. So I think that even the pooled analysis using all the data is valid because the measurement errors for a sample at site 1 are not correlated with the measurement errors in the corresponding sample at site 2. This really just amounts to repeating a point in the design space which should not be a problem. It does not create correlation  / "interference".

In my report I wrote that a bootstrap analysis was unnecessary because there is no correlation to adjust for. The three site models were valid (no possible "interference" within sites) and a pooled analysis could be done removing the common samples at site 1 when doing the pooling. Such a pooled analysis could not have an interference problem. A bootstrap adjustment would not be necessary because there is no bias to adjust for.

A) Do you agree with (1) My analysis of the client's results and (2) my argument that the bootstrap is unnecessary.

EDIT: Given the suggestion of Bill Huber I plan to look at bounds on the errors-in-variables regression by regression both y on x and x on y. We already know that for one version of OLS the answer is essentially the same as errors-in-variables when the two error variances are assumed to be equal. If this is true for the other regression then I think that will show that the Deming regression gives an appropriate solution. Do you agree?

I plan to use the R program that @whuber suggested in his answer to enable me to bootstrap the Deming regression. I am not very familiar with R but I think I can do it. I have R installed along with R Studio. Will that make it easy enough for a novice like me?

EDIT: Given the suggestion of Bill Huber I plan to look at bounds on the error-in-variables regression by regression both y on x and x on y. We already know that for one version of OLS the answer is essentially the same as error-in-variables when the two error variances are assumed to be equal. If the is true for the other regression then I think that will show that the Deming regression gives an appropriate solution. Do you agree??

In order to meet the client's request I need to do the requested bootstrap analysis that was vaguely defined. Ethically I think it would be wrong to just provide the bootstrap because it doesn't really solve the client's real problem, which is to justify their assay measuring procedure. So I will give them both analyses and request at least that they tell the FDA that in addition to do the bootstrap I did inverse regression and bounded the Deming regressions which I think is more appropriate. Also I think that analysis will show that their method is equivalent to the reference and the Deming regression is therefore adequate also.

I plan to use the R program that whuber suggested in his answer to enable me to bootstrap the Deming regression. I am not very familiar with R but I think I can do it. I have R installed along with R Studio. Will that make it easy enough for a novice like me?

Also I have SAS and am more comfortable programming in SAS. So if anyone knows a way to do this in SAS I would appreciate knowing about it.