Suppose I have massive data set (think Terabytes) is available to train a learning algorithm. Which one of the following conditions must be true to obtain good performance (low error rate)
a. Using a complex model with many parameters, including high order polynomial features of the original space (e.g x^10)
b. Training with a small number of parameters
I am thinking about regularization by adding peanlty and Bias -variance tradeoff. Please suggest what is good way to go...