If you have multiple dimensions to your data, where it is not possible to visualize them together, how to decide if your model should be linear or polynomial?
If you did the same course as me, Andrew Ng's machine learning course on coursera, you will remember that it was suggested there to split the data into 3 parts.
- Training set
- Cross-validation set
- Test set
Briefly, you fit each possible model to the training set. However you can't easily use the training set to decide which model is best, because additional terms in a model can only lead to a model fitting the training set better, with the possibility of overfitting (this is when the regression coefficients are "tuned" to noise in the training data). Rather you choose which model to use based on the error on the cross-validation set. Finally, you test how well the chosen model predicts using the test set.
If your model is highly dimensional, then it might be best to consider other approaches.
@Pranay Just to add, one way to decide weather to fit a linear or a non-linear function is to do residual analysis. Starting from a linear function you need to find the residual values for all predicted values as :
Yactual - Yobserverd
When you plot these values you can decide weather to fit a linear or a polynomial based how the residual values are scattered from the mean as:
If the values are randomly scattered then it may be safe to fit a linear function.
On the contrary if you can find a pattern in the residuals, then fitting a linear function is not going to work:
The above are the few among the many things you can do with residuals; they can be used to analyze large number of patterns in data.
Hope this helps.