Is there an intuition or any relevant reading about the relationship between dimensionality of data, number of samples, model complexity and test accuracy of classification?

E.g. for the simple cat/non-cat image classification. Can I know the amount of true and false samples needed to train an accurate (and probably complex) model without trying all the possible combinations? Moreover, what's the impact of the image resolution in the process?

Any graph down-sampling ImageNet, either in resolution or dataset size, for example?

Thank you


Yes. It seems like you're looking for sample complexity.

VC dimension is a related concept. The field that deals with these concepts is called computational learning theory. Understanding Machine Learning is an example of textbook that uses this approach to ML and contains calculations/estimation of VC dimension of the methods it covers (it's free BTW).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.