# What regularizer to use for small datasets?

If I have a sparse dataset with very few points, which regularization scheme should I use?

That is, I have a dataset with only 10 points. Are there regularizers that would help me in this situation?

• One thing to keep in mind is that a lot of regularization schemes correspond to priors, and with very little data you often just get your prior back – jld Feb 11 '18 at 5:25

On the other hand, how much you want to regularize is depending on your data size and the complexity of the model. Suppose we use L2 on polynomial fit for $10$ data points. If you want to use $5th$ order model, it is better to set $\lambda$ larger, comparing to you want to fit with $3rd$ order model.