$R^2$ increases when removing predictors I have a multiple regression model with many predictors (admittedly more than I want: 21). When I remove one of the predictors (leaving me with 20) my R squared increases a bit. Should this happen? Is it a function of having so many predictors? Should I be worried about having too many predictors in my model?
 A: The adjusted $R^2$, in the linear model, is of the form:
$$
\bar{R}^2 = 1 - \frac{\hat{\sigma^2}}{\frac{SST}{n-1}}
$$
Where, $\hat{\sigma^2} = \frac{SSR}{n-k-1}$. And SST is the total sum of squares.
Now the usual $R^2$ is of the form:
$$
R^2 = 1 - \frac{\frac{SSR}{n}}{\frac{SST}{n}}
$$
Where SSR is the sum of squared residuals. The diffference between these two measures, is that the adjusted $R^2$ dependes on k (the number of regressors). Therefore the adjusted $R^2$ can go both or down when adding addtional information. This is unlike the usual $R^2$, which can never decrease when addtional information is added - even if this information is complete nonsense.
For the second part of your question, 21 (20) seems like a large number of regressors. I would worry about overfitting, also (from a reader perspective) it makes the model hard to interpret. 
A: In order to be able to answer the question about whether n or n+1 represents the 'right' number of regressors, you need to establish a way to test your fit.  The main risk in using too many regressors (features) is overfitting.   The only way to establish this is by testing the regressor against Out of Sample data. 
To achieve this, you will need to split your dataset into a training and testing subset.  "Fit" each model on the training dataset and test the fitted regressor against the test dataset.  Whichever model exhibits the best score (e.g., the means of establishing best fit, in your case Root Mean Squared Error) will be the most appropriate given the data. 
There are more formal ways to control overfitting.   For example, there is a whole body of machine learning regressors called "Ridge Regressors" and "Lasso Regressors" which explicitly control for complexity.   Utilizing them requires a bit more refinement.  
See the following for more info:  
http://scikit-learn.org/stable/modules/linear_model.html
