# Mathematically Describing PCA chained with Logistic Regression

Python's scikit-learn package has a convenient pipe function that can combine machine learning techniques into one model with fitting and predicting functions. I was following this tutorial for chaining PCA and logistic regression and everything works as expected but I am having trouble describing this in terms of math notation because I am unsure what is really going on.

I know PCA algorithm is as follows:

1. Normalize/standardize data matrix: $Y=HX$
2. Calculate covariance matrix: $S=\frac{1}{n-1} Y^T Y$
3. Eigenvalue Decomposition: $S = Z \Lambda Z^{-1}$
4. Find PC Scores of original data: $T_L=YZ_L$

Where the transformation matrix is $Z_L$, and $L$ is the number of principal components used. Therefore, we can compress the original data using $T_L = Y Z_L$, where $T_L$ still has the same number of rows but only $L$ columns, thereby resulting in a reduced dataset.

How do I then combine the resulting $T_L$ matrix which is my transformed data with a multinomial ($>2$ classes) logistic regression?

I know multinomial logistic regression model is something like: Suppose the response variables has $K$ levels in the space $G=\{0,1,2,...,K\}$, representing the set of possible classes. The probability of determining a particular class is defined as,

$$Pr(G=k | X=x) = \frac{e^{\beta_{0k}+\beta_k^Tx}}{\sum_{l=1}^K e^{\beta_{0k}+\beta_k^Tx}}$$

But I do not know how to relate the two in a logical manner. (Or how the cost function should be defined.)

• What is your goal? You can use the calculated PCs (or some of them ...) as predictors, directly Commented Dec 28, 2020 at 0:25
• @kjetilbhalvorsen My goal is (was) to write in a paper the model that I am using for prediction. The code works but I don't know how to express in mathematical notation.
– guy
Commented Dec 29, 2020 at 3:13