Well my objective is to predict solar energy radiation at a particular location given some features like wind, temperature, humidity ...
I have a total data for 10 years where I have the measurement for the features at 100 locations(nearby) and the solar energy measurement at the target location for each day.
The original features were wind, temperature, humidity and my target is solar energy. Well for each feature wind/temperature/humidity, I had some ensembles plus geographic location. That means for humidity I had around 50 different readings. Similarly for wind I had 50 different readings and so on. All in all suppose taking the humidity feature, I had 50 reading for humidity at 100 surrounding locations and my target is the solar energy at a particular location.
Initially I applied PCA separately on wind, humidity .. features and extracted principal components that preserved variance of around 90%. However, the projected values on these separated principal components still had some correlation between them. I mean suppose I selected 10 components of wind, 10 components of humidity and so on. Then when I merged these individually PCAed components, they had some correlation between them.
So I applied a final PCA on the combined data to get final components
Now when I plot the scatter plot of the targets with the features obtained after applying PCA two times, it's very bad. I am showing the scatter plots of the features after applying PCA. Here they are. It's really bad except the first one. I had around 52 features selected after applying PCA twice and these are the ones from the top four components.
I am not sure if I am supposed to get such results. Further if I train a model with just the first feature, it's much worse than using all the features even though the other have very bad scatter plots. I am confused related to how to interpret this situation. Any suggestions guys?
Further before applying second PCA(after applying first PCA) I had the following scatter plots of the one time PCAed values with the target
As you can see the scatter plots after applying first PCA are worse than after applying PCA twice. However, using the features after the first PCA, I get much better accuracy rather than after applying PCA twice. I am not able to interpret this situation what's causing this. Any help guys?