What is the difference between sieve estimation and structural risk minimization? I was wondering if you could help me out. I am quite confused about the difference between sieve estimators (Ulf Grenander) and structural risk minimization (SRM) (Vladimir Vapnik). Could anyone give more information on the difference between these procedures? Both seem to trade off the considered model complexity with the available sample size.
Thanks in advance :D
 A: A sieve estimator is a type of semi-nonparametric procedure in which we have a parameter that lives in a high-dimensional or infinite-dimensional space. Since it is difficult to maximize an objective function over an infinite-dimensional space we instead maximize over a sequence of finite-dimensional spaces, these are called sieves (hence, the name). We are essentially making a "promise" to infinitely increase these spaces so that they are dense in the space in which our parameter lives. There are many assumptions we need for this to work well and this description is meant to be heuristic for better explanations take a look at the linked references below.
SRM is essentially a regularization procedure to avoid overfitting. Think about how we add a regularization term in the SVM estimator. Basically, we are trying to find a reasonable trade-off between how well the model fits the data and how complicated the model is. Interestingly, there is a relationship here. In both cases, we are essentially looking to approximate a complicated model in a less complicated space.
Some references:
https://www.sciencedirect.com/science/article/pii/S157344120706076X
http://www.cnel.ufl.edu/courses/EEL6814/srm.pdf
