How can I increase the $R^2$ -value of my linear model? Should I increase it? I'm constructing a linear model from a data set with 10 variables and my current "best" model uses 4 variables. I've tested the variables and not all of them show significance, so the most that I might add to the model might be 5 variables overall. However, with 4 and 5 variables, eventhough I get very good Pr(>|t|) values for all the variables that I've added and while my $R^2$ value has been improving, it's still just "only" $0.4287$ and the scale is $[0,1]$.
Should I be happy with $< 0.5$ $R^2$ value of should I aim to improve it in order to improve my model even more? What can I do to increase the $R^2$ value now that I have already added all variables that I can add as predictors? Should I look into interaction terms? Or something else?
 A: One common approach in these cases is to plot the train and test errors as a function of the size of your dataset. This might give some insight as to how the specific bias variance tradeoff of the algorithm you're using, is working for the particular problem.
For a specific value of n, generate several subsets of size n from your dataset. For each one, calculate the train error and test error (the latter, using the hold-out data), and record the average.
Repeat this for several values of n, and plot the two curves:


*

*If the algorithm is suffering from high bias, then the test error will initially decrease, then settle at a relatively-fixed high value.

*If the algorithm is suffering from high variance, there will typically remain a large gap between the train and test error.
In your case, you mention that you're using a linear predictor, so high bias is somewhat more of a suspect. If the graphs corroborate this, a good guess would be to switch to one of many higher-variance algorithms.
