Cross-validation and feature selection of a multivariate regression I've been trying to create a multivariate regression model to fit my training data into the prediction of a value. I've put my data into a matrix X with  m x n where m is the number of instances and n the number of features/predictors. My label vector is then m x 1. This is my code to predict the theta values, or parameters.
theta_matrix =  pinv(X'*X)*X'*y_label;

Now I want to slip the data into train and test, and by researching I've found that cross-validation in 10-fold can be a good option. If I do so, wouldn't it get me 10 sets of parameters theta? So what to choose from then?
And about feature selection, I've found that stepwise can be a good choice, but I think it does not take into account that features can be correlated. Any alternative?
 A: You can use cross-validation to understand how your model would behave on completely new or 'unseen' data. 
You should also use cross-validation to select which features to use - try sequential feature selection (sequentialfs in Matlab) or Lasso (lasso in Matlab). cvpartition command in Matlab will allow you to setup your test/train partitions for cross-validation, sequentialfs will take a partition object as an input.
Once you have an idea of how your model will behave on unseen data. You can decide what features to use and generate your 'final' model by running the same routine on all available data, which will give a single set of coefficients.
An excellent answer on a similar topic is here
A: "And about feature selection, I've found that stepwise can be a good choice, but I think it does not that into account that features can be correlated. Any alternative?"
Stepwise regression is a great way to do feature selection.  Make sure you are using a metric that penalizes for incorporating more features though.  For example stepwise regression on just $R^2$ may not be useful, but stepwise regression on AIC may be useful.
In general I remove correlated features before doing stepwise regression (although in theory stepwise regression using AIC shouldn't pull in two correlated features).  I do this by looking at pairwise correlation but also by looking at multicollinearity, in particular VIF. 
Finally, instead of doing any feature selection, you can do principal components analysis or another dimension-reduction technique to reduce your number of inputs.  When I've done in practice, I then use cross-validation to pick how many of the reduced dimensions to use.  For example I make $X$ different models that are trained on $1,2,3,4,...X$ lower dimensions and then see which model performs the best in terms of cross-validation.
