When the data set contains many predictors (and most of the predictors are categorical variable), and the correlation matrix shows that none of those predictors have high correlation with the response variables, then what should be the next step to go for building a predictive model? This is a real problem I am facing. The maximum correlation between a predictor and the response variable is 0.001. Note that correlation only measures the linear relationship between 2 random variables. This means, even if none of the predictors seems correlated with the response variable, this does not necessarily mean that those predictors are independent of the response variable. May be there are complex non-linear relationship, which is usually the case.
I've asked a professor, who suggested that first do PCA on the set of predictors. But I think the idea of PCA is just change a different coordinate system to express the same set of predictors. Then if the set of predictors seem uncorrelated with the response variable, then why after changing the coordinate system, the same set of predictors will start to be correlated with the response variable? Also, the result of PCA is a linear combination of the original set of predictors, then if the original set of predictors are uncorrelated with the response variable, then why it's linear combination will be correlated with the response variable? I couldn't understand this point intuitively. Can someone provide an example?
Also, in my situation, most of the predictors are categorical variables, then I am not sure whether it make sense to do PCA, although you can always do PCA.