# Rules of thumb for preventing underfitting + overfitting?

Let's say I have a supervised machine learning problem, where I am trying to map $D$ dimensional feature vectors $(x_1,x_2,...,x_D)$ to $1$ of $C$ possible class labels. I have $N$ training examples to "learn" from, and $K$ testing examples to classify. My goal is to maximize classification accuracy on the $K$ testing examples. My model, $M$, has $P$ free parameters.

My questions are:

1. Are there any rules of thumb for how I can choose $P$ so that I neither underfit nor overfit to the data?
2. Will the answer depend on $M$ (i.e. whether I am using Neural Nets, Random Forests, etc)?
3. Also, if I adjust the hyperparameter settings of $M$, $Q$ times, does this affect the answer?
• Have you looked for the answer to this in other questions. I'm pretty sure I have seen several since I started here. I know the folks in charge like to have clear reasons why you think this is not a repeat of one of them. – EngrStudent Mar 22 '15 at 17:29
• @EngrStudent A unique aspect of the question above may be that I am asking if the "best" $P$ is dependent on how many times you change hyperparameters. I wasn't able to find a question that tackled that specific question, although if there was one I would love to read the answer! Edit: Also, are all parameters equal? I.e. does $P$ depend on the model, I haven't seen a question tackling that in specific (though I could be wrong). I have seen questions asking very generally about overfitting, but I think there are specific aspects to my question that I haven't seen here before. – kyphos Mar 22 '15 at 17:32