5
$\begingroup$

In an experiment several measurements are taken using similar but different measuring instruments. (The number of measurement tools used in a single experiment could range from 2 to 500 instruments, but most have a low number (~2 - 3) of instruments used.) Since all the instruments are measuring the same effect, it is expected that they produce similar measurements, but possibly with difference sources and levels of noise. Some of the measuring tools may unknowingly be malfunctioning and produce erroneous data altogether. This means most of the measurements are somewhat correlated (> 0.8), but some could be uncorrelated or even inversely correlated. How can one summarize the measurements of the instruments in such a way as to best represent the real value of the quantity being measured?

Possible approaches to this problem might include using:

(1) a regression model to fit the measurements and then interpolate the measurement's summarized value, (2) the first component of a principle component analysis, (3) or the scores from a factor analysis.

Which method is most appropriate for dealing with the task or is another approach better for doing this summarization?

Improve this question
$\endgroup$
4
  • $\begingroup$ You do not provide enough information to show how a regression model might be applied: no covariates or explanatory variables are mentioned. In the same vein, what would a PCA or factor analysis show? Nothing useful, AFAIK. In the end, the question comes down to this: in each experiment you have multiple measurements of one thing, some of which may be unreliable. How to estimate the correct value? As @gui11aume points out, you need a robust method. The most robust is the median. The answer is that simple, unless you care to elaborate. $\endgroup$
    – whuber ♦
    Commented Jun 22, 2012 at 3:53
  • $\begingroup$ Do you have a single measurement per device for each experiment, or are measurements taken over time? Do you have a model of the types errors you see for each device? $\endgroup$
    – Nick
    Commented Jun 22, 2012 at 15:32
  • $\begingroup$ @Nick: As single measurement per device per sample. While measurements are not taken over time, there are two groups of samples (n = 50+) with a differing property, it is this property which is being explored. No current error model for the devices. $\endgroup$
    – Nixuz
    Commented Jun 22, 2012 at 16:21
  • $\begingroup$ @whuber: A median might be worth a try. Could you please create an 'answer' below with the contents of your comment. Thank you! $\endgroup$
    – Nixuz
    Commented Jun 22, 2012 at 16:59

2 Answers 2

Reset to default
1
$\begingroup$

I understand from the context of your question that some measurements are biased (since you can get erroneous data), so the summary will be biased. As far as I am aware, the theoretical framework of linear regression, PCA and factorial analysis assumes unbiased records, so their properties might be different in your case.

For that reason, I would use robust methods, that are not overly influenced by outliers. Robust linear regression has been worked out some time ago, and is readily available in R. The easiest is the function rlm in the package MASS, which is used like the non robust function lm. After a quick Google search, it seemed to me that robust methods for PCA and factorial analysis are still in development and much more difficult to use. If you are an R user, you can have a look at the document of rrcov and robCompositions.

I am a great fan of PCA, but here I would recommend the easiest, at least for a start, which is your method (1) with robust regression.

Improve this answer
$\endgroup$
2
  • $\begingroup$ What would your answer be if bias was not an issue? $\endgroup$
    – Nixuz
    Commented Jun 21, 2012 at 2:03
  • $\begingroup$ @Nixuz: as I said in the answer, I usually use PCA, mostly because I know how to do it. I like the fact that it treats all variables equally, which is a good start in many situations. I use linear regression only when there is a clear response variable. $\endgroup$
    – gui11aume
    Commented Jun 21, 2012 at 9:34
1
$\begingroup$

without a clear response variable, i would rule out regression of any type. as stated, the problem here is the summarization of measures from a variety of instruments. to that end, both PCA and FA methods are appropriate, however, it should be acknowledged that the purposes (and subsequent inferences) differ for these two methods.

PCA seeks to reduce, or summarize, a number of measures according to composites of the observed variables (well, the coefficients resulting from the estimated components). factor analysis, on the other hand, presumes an underlying or latent distribution which is manifest through the various measurement results. both of these methods allow for "noisy" and correlated data - in fact, they are premised upon covariance analysis - but they do require some assumptions about the normality of the observations. i'm not so sure about PCA, but FA methods are reasonably robust to violations of normality; in conditions of extreme non-normality there are alternate procedures and adjustments that can be applied (finney & distefano, 2006).

simply: if your goal is to reduce the number of measurements, PCA is your friend. if you're attempting to explain an underlying phenomena through multiple measures, FA is likely your best choice.

note: noise may be used to describe random variation or non-systematic bias, which is expected by both PCA and FA. systematic bias, however, is troublesome for most measurement problems.

Finney, S. J., & DiStefano, C. (2006). Nonnormal and categorical data in structural equation models. In G.R. Hancock & R.O. Mueller (Eds.). A second course in structural equation modeling (pp. 269 - 314). Greenwich, CT: Information Age.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.