Although there is lots of information on all three subjects, I could not find information on how to solve my specific issue. Most likely that is because I am not an expert in statistics and might have overlooked something. If someone knows of a link that deals with my specific question I will be most happy. If there is no such thing, maybe you can explain what I need to do.
I have a 4000 x 10 matrix (10 columns by 4000 rows). Column 1 is the dependent variable, all other columns are explanatory variables. Columns 2:6 are all categorical variables ( and contain strings as 'on' versus 'off', and 'South' versus 'West' versus 'North' versus 'East', etc.), and columns 7:10 are all numeric AND are correlated (to various degrees) to one another. There is no a priori knowledge of any correlations among vectors 2:6 or between any vectors 2:6 versus any vectors 7:10.
I want to determine a linear regression model. I could use PCA to address the correlated variables but what about the categorical variables? I can apply lasso, but what about the correlated variables? And can I address lasso's issue with categorical variables by using group lasso?
What would be the correct procedure to obtain a best linear regression model for this dataset? I use matlab but I am not after any matlab coding but I want to understand the process steps I need to undertake.