# Principal Components for dependent variable in a regression

My question is related to PCA. I want to estimate the effect of agriculture variability on the welfare measures for financialy included people against the excluded. For this purpose I am trying to generate an index of welfare measure through PCA by having it as a function of four sets of households expenses; food consumption, non-food consumption, household expenditure and expenditure on durable goods.

1. Can PCA be used to create an index which can then be used as a dependent variable (Household welfare index through PCA in the present case)?

2. If yes, then what is the way to make interpretation of betas on the independent variables used in regression. Say for example Age of the Household head.

• I don't understand question number 2. Can you clarify? What is "age of the household head", is it an independent variable? I thought your said that agriculture variability is the independent variable? Can you explain your regression setup in more detail? Commented May 24, 2016 at 16:20
• I can clarify my regression equation. Agri. Variability is indeed my independent variable, which I will be instrumenting with a suitable rainfall variation measure. When I said "Age of household head", it is just one of the other demographic controls that I will using to control for the omitted variable bias. Now since PCA is an index with no units (dependent variable in this regression depicting household welfare) and my control variables would be with certain units (such as years or level or education), then what would be the appropriate way to interpret the coefficients of these controls? Commented May 25, 2016 at 10:00
• I don't really know whether this fits your question, but "canonical correlation" was built for explaining the pc's of a set of items (dependent) by the pc's of another set of items (independent). It is for instance implemented by a macro in SPSS (see en.wikipedia.org/wiki/Canonical_correlation) Commented Nov 7, 2016 at 6:49
• Perhaps this -tentative- introduction to regression on pc-factors/components is interesting for your question: go.helms-net.de/stat/fa/FullComponentsFA_1.htm Commented Feb 20, 2017 at 12:34

I think mathematically, what you are attempting to do may be feasible (answer to question 1). As far as interpretation of betas (question 2), PCA is already extremely challenging to interpret as is in the best of cases. Usually, most of the explanatory power is concentrated in the first Principal Component. When studying the composition of the Betas on the variables of that first Principal Component, you often observe results that are counterintuitive and cryptic. And, extending interpretation to the second and third Components is most often as baffling.

PCA is best used for two reasons: 1) streamline a large number of independent variables into three Principal Components; and 2) resolve issues of multicollinearity associated with a very large number of independent variables. However, PCA has major drawbacks. It is so opaque (opposite of transparent). It renders clear interpretation of the results from very challenging to impossible.

An alternative to PCA is simply to streamline your model. Use a clearly defined and straigthforward dependent variable. Explore tens of independent variables if you wish, but select judiciously the best 6 or 7 ones that make the most sense supported by social sciences and economic theory. Usually, this generates a simple, explanatory model that is easy to interpret and present to various audiences. You can add more variables, but watch out for model overfitting. The latter is actually a mathematical trap that PCA models can readily fall into (overfitting the data, meanwhile being very poor predictors because they fit to the noise within the data).

• Thank you very much for your answer. To be clear, though I am derving the index through PCA, however the variables I will be using for that index would not be used for my regression. The regression model will absolutely be different, and would involve the variables representing financial inclusion, agricultural variability and other controls. Also, this regression will be fixed effect with IV. Commented May 24, 2016 at 12:45
• Hence to state it more clearly, I will involve two stages: In the first stage, I will derive my index as: HWI=f(FC,NFC,HC,CG) through PCA, and in the second stage I would use this index as dependent variable in regression that would not include the variables used for creating HWI. The model for second stage would be as follows: HWI_it=α+β〖(AGV〗_it)+γ〖(FI)〗_it+δ(〖AGV〗_it*〖FI〗_it )+(H_it )θ+μ+τ+ε Commented May 24, 2016 at 12:49
• @Anuj, thanks for the clarification, I have corrected my answer accordingly. Commented May 24, 2016 at 16:02

I would suggest using Structural Equation Modeling (SEM) to address this problem. In brief, you are proposing a latent variable, welfare index, which is directly influencing 4 manifest (measured or observable) variables. If you are then asking if there is a difference between a dichotomous grouping variable (included vs. excluded), you can run a multigroup SEM analysis to test the measurement invariance at different levels of constraints. Happy to share more if this seems useful.

I have done what you described here: instead of modeling a bunch of variables, get their PCs and regress them. It may work in some cases, e.g. when you suspect that the dependent variables are driven by a few linear factors.

Otherwise, you could try SUR regression. It can look scary at first but it's very easy to implement. The idea's that you regress the dependent variables as if they were independent with their own drivers, but then make their errors correlated. It's simpler and more stable than vector autoregression and more complicated approaches such as vector error correction.