# Linearity preservation after PCA covariate transformation

thank you for this interesting discussion about linearity of PCA. enter link description here

I have a question, which I do not know if it's trivial or not, but I do need to clarify it.

I want to fit a Cox PH Model with a set of covariates, some are continous and others categorical (3 are binary - 0/1 and 2 have more than 2 categories).

One of the assumptions of CoxPH regressions is linearity of the covariates vs log-hazard ratio.

My questions are:

1- If the continous covariates are linear vs log-hazard ratio, will the PC's ,obtained by fitting a PCA, preserve the linearity vs log-hazard ratio?

2- If we include the categorical covariates in the PCA, would this altere the linearity condition of the PC's? I assume that binary covariates may preserve it (if condition 1 is TRUE) as they are linear vs the outcome (only 2 possible values) but problems could arise when including categorical covariates with more than 2 categories.

Thank you so much for your help.

• Do you have in mind using PCA for dimension reduction, i.e., do you want to use fewer PCs than original variables for your regression, or are you just using PCA for linear transformation, and you want to use the same number of PCs as original variables, preserving all information? Commented May 8 at 9:28
• If you use it for dimension reduction, it may well invalidate the model in any case, by losing information that in fact should be in the model. Commented May 8 at 9:29
• In fact, without having written it down mathematically, I'm pretty sure the answer to 1 is no for dimension reduction and yes for full dimensionality. Same for 2. Commented May 8 at 9:31
• My idea is to use PCA for dimension reduction, but my understand is that a linear combination of covariates (each principal component is a linear combination of the raw covariates) that have a linear relationship with the outcome, may preserve the linearity with the same outcome. And this is the main question I would like to discuss/confirm. Commented May 8 at 9:58

As the question asks for use of PCA with reduced dimension, the answer to 1 (and therefore also 2) is: In general no.

This isn't different between Cox regression and standard linear regression. Here is an example. Say the linear predictor in variables $$(x_1,x_2)$$ is $$a_1x_1+a_2x_2$$. Say that $$x_2=bx_1^2$$ and that the first PC is $$c_1x_1$$ with $$a_1, a_2, b, c_1$$ suitable constants. Note that if the distribution of $$x_1$$ is symmetric about zero, drawing suggests that the first PC will be either $$c_1x_1$$ or $$c_2x_2$$; in any case it should be possible to have such a situation.

Then according to the original linear model the correct predictor as function of the first PC $$p_1$$ is a squared function $$d_1p_1+d_2p_1^2$$, therefore nonlinear.

This is of course an extreme example with (1) perfect dependence between $$x_1$$ and $$x_2$$ and (2) the PC being exactly along $$x_1$$, but in general nonlinear dependence between the original predictors will induce a nonlinear relation between the correct predictor and the PCs.

On top of this, in general, dimension reduction will invalidate the model if in fact all original predictors, i.e., the full dimensional space (or a higher dimensional space than the number of PCs, or even any information in the original predictors that is not represented in the PCs), is needed to correctly predict the outcome, because in that case the relation between outcome and PCs (i.e., the predictors used after dimension reduction) will depend on information not incorporated in the PCs, in violation of the model.

If however the full dimensional outcome of the PCA is used, the correct linear predictor in the original variables can be reconstructed as linear predictor in the PCs by suitable linear transformation. (Categorical variables with $$q>2$$ outcomes can be coded by $$q$$ binary variables, so there shouldn't be particular problems with them.)

• Thank you so much for your answer Christian, it's being more clear. With these facts, one could think that it's not possible (or extremely difficult) to fit a 'valid' cox PH model with PC's because we can't preserve the linearity achieved with the raw o transformed variables that has been before computing those Principal Components. My concern is that I want to fit a cox model with less degrees of freedom and no overfitting and it's being extremely difficult without violating linearity and proportional hazards assumptions. Commented May 8 at 18:32
• @sinectica Well, I'm not saying it's generally wrong to do PCA dimension reduction. Note that model assumptions are generally not precisely fulfilled. I say that if model assumptions hold for the original variables and all variables are needed, then model assumptions will not be fulfilled for PCA with reduced dimension. But this doesn't imply that it will necessarily be a bad model. Commented May 8 at 22:18
• Henning thank you so much again for your comments, they are very valuable. A question with regards your sentence" I say that if model assumptions hold for the original variables and all variables are needed, then model assumptions will not be fulfilled for PCA with reduced dimension.". What do you mean with it? In fact I can develop a better or worse model with more or less original covariates, but I do not understand how to distinguish if all original variables are needed or not. Commented May 9 at 11:44
• @sinectica I was just stressing that I made a formal mathematical statement that relies on mathematical assumptions (addressing your actual question). Reality is different anyway. Commented May 9 at 11:53