Let's say I have X1, X2, X3, and X4. All four variables are strictly positive (no values below zero). The variables are on different scales.
I do a PCA on the four variables' correlation matrix and get the first principal component, with loadings on X1, X2, X3, and X4 of 0.8, 0.5, 0.4, and 0.7 respectively.
How come first principal component contains negative values? And is it possible to identify the value of the first principal component at which X1, X2, X3, and X4 = 0?
Regarding your first question: PCA scores are latent variables. From a more general perspective, scale and mean of latent variables are essentially unidentified. Therefore, constraints/conventions are necessary. PCA scores are typically centered at zero such that zero means "an observation that has an average score on the PCA". Negative values just mean "lower than average" component scores. This makes sense because when you conduct PCA on a covariance or correlation matrix instead of the raw data, you essentially remove any information about means from your data (because data are inherently centered in the computation of the correlation matrix). Here's a minimal example from R for that:
library(mvtnorm); library(psych)
sigma = matrix(0.7,ncol = 10, nrow = 10)
diag(sigma) = 1
X = rmvnorm(n = 200, mean = rep(100,10), sigma = sigma)
fit = princomp(X,scores = TRUE)
summary(fit)
fit$loadings
fit$scores
describe(fit$scores)
Regarding your second question: If you are interested in the value of the component at certain values of the observed variables, you just need to insert these values into the scoring function of the PCA. BUT pay attention that your data is inherently centered when conducting the PCA. The covariance matrix is X'X/(n-1) where X is your centered data matrix. That is, from the perspective of PCA, your data was actually centered. Therefore, I think more discussion is necessary about this point and I refrain from making more specific recommendations without knowing more in detail why you need this.
Edit: Just to be precise. Sometimes PCA is conducted on the uncentered matrix X'X. Then, deviations from the origin also occur as principal components. However, the default in most software packages is to center before calculating X'X and I assumed that you did use some standard software.
Principal components are the eigenvectors of the scatter matrix, and if $u$ is a PC, $-u$ could also be chosen as PC. So, even if your all PCs have all positive values, we can invert them to have negative ones. The critical equation is, (let $X_c$ be the centered data): $$X_c^TX_c=\sum\lambda_iu_iu_i^T$$ This is satisfied by both $u_i$ and $-u_i$.
Moreover, PCs show the directions of variation in the data. The first PC is the direction of the most variation. By enforcing all positive values, you're confining the data to the first orthant, but it may not be. Consider a 2D data, which is scattered around $y=-x$ line with some small noise. The first PC will be $[1/\sqrt{2},-1/\sqrt{2}]$, which has both positive and negative values.
