Summarization of correlated but noisy measurement data In an experiment several measurements are taken using similar but different measuring instruments. (The number of measurement tools used in a single experiment could range from 2 to 500 instruments, but most have a low number (~2 - 3) of instruments used.) Since all the instruments are measuring the same effect, it is expected that they produce similar measurements, but possibly with difference sources and levels of noise. Some of the measuring tools may unknowingly be malfunctioning and produce erroneous data altogether. This means most of the measurements are somewhat correlated (> 0.8), but some could be uncorrelated or even inversely correlated. How can one summarize the measurements of the instruments in such a way as to best represent the real value of the quantity being measured?
Possible approaches to this problem might include using:
(1) a regression model to fit the measurements and then interpolate the measurement's summarized value,
(2) the first component of a principle component analysis,
(3) or the scores from a factor analysis.
Which method is most appropriate for dealing with the task or is another approach better for doing this summarization?
 A: I understand from the context of your question that some measurements are biased (since you can get erroneous data), so the summary will be biased. As far as I am aware, the theoretical framework of linear regression, PCA and factorial analysis assumes unbiased records, so their properties might be different in your case.
For that reason, I would use robust methods, that are not overly influenced by outliers. Robust linear regression has been worked out some time ago, and is readily available in R. The easiest is the function rlm in the package MASS, which is used like the non robust function lm. After a quick Google search, it seemed to me that robust methods for PCA and factorial analysis are still in development and much more difficult to use. If you are an R user, you can have a look at the document of rrcov and robCompositions.
I am a great fan of PCA, but here I would recommend the easiest, at least for a start, which is your method (1) with robust regression.
A: without a clear response variable, i would rule out regression of any type. as stated, the problem here is the summarization of measures from a variety of instruments. to that end, both PCA and FA methods are appropriate, however, it should be acknowledged that the purposes (and subsequent inferences) differ for these two methods.
PCA seeks to reduce, or summarize, a number of measures according to composites of the observed variables (well, the coefficients resulting from the estimated components). factor analysis, on the other hand, presumes an underlying or latent distribution which is manifest through the various measurement results. both of these methods allow for "noisy" and correlated data - in fact, they are premised upon covariance analysis - but they do require some assumptions about the normality of the observations. i'm not so sure about PCA, but FA methods are reasonably robust to violations of normality; in conditions of extreme non-normality there are alternate procedures and adjustments that can be applied (finney & distefano, 2006).
simply: if your goal is to reduce the number of measurements, PCA is your friend. if you're attempting to explain an underlying phenomena through multiple measures, FA is likely your best choice.
note: noise may be used to describe random variation or non-systematic bias, which is expected by both PCA and FA. systematic bias, however, is troublesome for most measurement problems.

Finney, S. J., & DiStefano, C. (2006). Nonnormal and categorical data in structural equation models. In G.R. Hancock & R.O. Mueller (Eds.). A second course in structural equation modeling (pp. 269 - 314). Greenwich, CT: Information Age.
