# Adjusting PCA based on prior information

When performing a PCA analysis on a set of questions (participants have been asked to fill out a questionnaire), what measures can be taken to adjust for clearly gender biased questions? That is, how can one adjust for the fact that males and females tend to express this same trait in different ways?

For example, if one wants to find components that describe physical activity, how should one adjust for the fact that males and females tend to express this same trait through different means.

Yes, I know that one clear solution would be to redesign the questionnaire, but what other mathematical methods are there (for those cases where redesigning is not possible, due to the nature of the questions)?

• Could you perhaps say a little more about the purpose of doing PCA in this case and why that leads you to think some kind of "adjustment" for gender bias might be needed? – whuber Aug 4 '11 at 13:37
• I didn't retag but I suspect that scales would have fit here. Why don't you simply incorporate the person-specific effect into a latent variable model, like in the multiple indicator multiple cause model (MIMIC), see e.g. one of Muthén tutorial? – chl Aug 4 '11 at 19:31
• This sounds like a job for Bayesian PCA – Emre Aug 4 '11 at 20:15
• You could residualize on gender. – Jeremy Miles Apr 22 '15 at 17:47
• @Jeremy I'm not familiar with the term, can you link to an example? – Figaro Apr 23 '15 at 13:01

This question falls into the area known as 'factorial invariance'. There are several kinds of invariance, and it's not absolutely clear which you mean.

One problem is that males and females have different levels of the indicators, even when the level of the trait being studied is the same. One example is crying as a symptom of depression. Given the same level of depression, women tend to cry more often than men. If you treat that variable equally for males and females, the female scores will be biased high. Another problem that will arise is that if there is another item that women score differently on, for the same level of depression, then you are at risk of extracting a 'gender' factor. One solution to this is to regress each variable on gender, and save the residuals. This fixes the mean for each group, for each variable, to be zero. Equivalently, for each item, subtract the male mean from the female mean, and all groups will have the same mean.

In factorial invariance, we consider several levels of this.

1. Configural invariance: same factor structure holds (This is what @Md Azimul Haque is discussing)
3. Strong factorial variable intercepts are equivalent.
4. Strict factorial invariance: residual variances are equal.

The problem my answer deals with is (3), which is often the most serious The problem that @Md Azimul Haque deals with is (1), however the approach that they describe isn't very useful, as you can't tell if it is sampling variation or actual differences when using PCA. Factorial invariance is usually investigated using confirmatory factor analysis.

There's a book by Roger Millsap which goes into considerable detail on this.

(In addition, I don't think you should be using principal components analysis; this sounds like a factor analysis problem. But that's another question.)