I want to understand the correlation between a set of $\color{red}{\textrm{inputs}}$ and a set of $\color{blue}{\textrm{outputs}}$ deriving from numerical simulations, and possibly reduce dimensions of my problem. A number of simulation samples $n$ was ran by randomly selecting inputs within given intervals using the Latin Hypercube sampling (LHS) method. Then, the respective outputs were stored, so I have the combination of inputs-outputs for each sample.
Since the inputs were selected via LHS method, the input-input correlation should not exist. The output-output correlation can exist, but I am more interested in the input-output correlation. It is also possible that the combination of multiple inputs affect the outputs in unexpected ways.
A correlation matrix (pearson,spearman) can identify correlations between single inputs and single outputs. But to include more complicated correlations, I am considering other tools, such as Principal Component Analysis (PCA). However, I am not sure if it's applicable, or how to interpret the results in such scenario. Let me give a concrete example.
In my specific case, there are 16 inputs and 3 outputs for $n=$5200. My first approach was to concatenate them and perform PCA in this $n \times 19$ matrix. What I found was that the first 3 principal components only explain ~35% of the observed variance, and to reach 95% I need 12 principal components. This seems to indicate that PCA is not of much help. I believe this is a symptom of the fact that there's so much variance in the inputs due to the sampling method. Then I looked at the loading factors, some of which are presented here:
PC1 (17%): $\color{blue}{\textrm{output1=0.6}}$, $\color{blue}{\textrm{output2=0.5}}$, $\color{red}{\textrm{input1=0.45}}$, $\color{red}{\textrm{input2=0.25}}$, $\color{red}{\textrm{...(inputs)...}}$, $\color{blue}{\textrm{output3=0.06}}$, $\color{red}{\textrm{...(inputs)...}}$
PC2 (29%): $\color{blue}{\textrm{output3=0.7}}$, $\color{red}{\textrm{input2=0.6}}$, $\color{red}{\textrm{input3=0.2}}$, $\color{red}{\textrm{...(inputs)...}}$, $\color{blue}{\textrm{output2=0.07}}$, $\color{blue}{\textrm{output1=0.03}}$, $\color{red}{\textrm{...(inputs)...}}$
PC3 (36%): $\color{red}{\textrm{input5=0.7}}$, $\color{red}{\textrm{input6=0.6}}$, $\color{red}{\textrm{...(inputs)...}}$, $\color{blue}{\textrm{output3=0.04}}$, $\color{red}{\textrm{...(inputs)...}}$, $\color{blue}{\textrm{output1=output2=0.03}}$, $\color{red}{\textrm{...(inputs)...}}$
My interpretation is that PC1 informs mostly how $\color{red}{\textrm{input1}}$ and $\color{red}{\textrm{input2}}$ correlate with $\color{blue}{\textrm{output1}}$ and $\color{blue}{\textrm{output2}}$. PC2 indicates that $\color{blue}{\textrm{output3}}$ is mostly correlated with $\color{red}{\textrm{input2}}$ and $\color{red}{\textrm{input3}}$. Then, PC3 seems to correlate only the $\color{red}{\textrm{inputs}}$ with themselves, but this is an useless information and not at all expected (due to LHS). Should it be ignored then?
Regardless, at this level I am still al 36% of explained variance. So it seems that the correlations are either very small or non-linear. Can someone point out if I am missing something about PCA, or if it is not appropriate in this case?
