What is meant when we say we have a saturated model?
A saturated model is one in which there are as many estimated parameters as data points. By definition, this will lead to a perfect fit, but will be of little use statistically, as you have no data left to estimate variance.
For example, if you have 6 data points and fit a 5th-order polynomial to the data, you would have a saturated model (one parameter for each of the 5 powers of your independant variable plus one for the constant term).
A saturated model is a model that is overparameterized to the point that it is basically just interpolating the data. In some settings, such as image compression and reconstruction, this isn't necessarily a bad thing, but if you're trying to build a predictive model it's very problematic.
In short, saturated models lead to extremely high-variance predictors that are being pushed around by the noise more than the actual data.
As a thought experiment, imagine you've got a saturated model, and there is noise in the data, then imagine fitting the model a few hundred times, each time with a different realization of the noise, and then predicting a new point. You're likely to get radically different results each time, both for your fit and your prediction (and polynomial models are especially egregious in this regard); in other words the variance of the fit and the predictor are extremely high.
By contrast a model that is not saturated will (if constructed reasonably) give fits that are more consistent with each other even under different noise realization, and the variance of the predictor will also be reduced.
As everybody else said before, it means that you have as much parameters have you have data points. So, no goodness of fit testing. But this does not mean that "by definition", the model can perfectly fit any data point. I can tell you by personal experience of working with some saturated models that could not predict specific data points. It is quite rare, but possible.
Another important issue is that saturated does not mean useless. For instance, in mathematical models of human cognition, model parameters are associated with specific cognitive processes that have a theoretical background. If a model is saturated, you can test its adequacy by doing focused experiments with manipulations that should affect only specific parameters. If the theoretical predictions match the observed differences (or lack of) in parameter estimates, then one can say that the model is valid.
An example: Imagine for instance a model that has two sets of parameters, one for cognitive processing, and another for motor responses. Imagine now that you have an experiment with two conditions, one in which the participants ability to respond is impaired (they can only use one hand instead of two), and in the other condition there is no impairment. If the model is valid, differences in parameter estimates for both conditions should only occur for the motor response parameters.
Also, be aware that even if one model is non-saturated, it might still be non-identifiable, which means that different combinations of parameter values produce the same result, which compromises any model fit.
If you wanna find more information on these issues in general, you might wanna take look at these papers:
Bamber, D., & van Santen, J. P. H. (1985). How many parameters can a model have and still be testable? Journal of Mathematical Psychology, 29, 443-473.
Bamber, D., & van Santen, J. P. H. (2000). How to Assess a Model's Testability and Identifiability. Journal of Mathematical Psychology, 44, 20-40.
A model is saturated if and only if it has as many parameters as it has data points (observations). Or put otherwise, in non-saturated models the degrees of freedom are bigger than zero.
This basically means that this model is useless, because it does not describe the data more parsimoniously than the raw data does (and describing data parsimoniously is generally the idea behind using a model). Furthermore, saturated models can (but don't necessarily) provide a (useless) perfect fit because they just interpolate or iterate the data.
Take for example the mean as a model for some data. If you have only one data point (e.g., 5) using the mean (i.e., 5; note that the mean is a saturated model for only one data point) does not help at all. However if you already have two data points (e.g., 5 and 7) using the mean (i.e., 6) as a model provides you with a more parsimonious description than the original data.
It is also useful if you need to calculate AIC for a quasi-likelihood model. The estimate of dispersion should come from the saturated model. You would divide the LL you are fitting by the estimated dispersion from the saturated model in the AIC calculation.
In the context of SEM (or path analysis), a saturated model or a just-identified model is a model in which the number of free parameters exactly equals the number of variances and unique covariances. For example the following model is a saturated model because there are 3*4/2 data points (variances and unique covariances) and also 6 free parameters to be estimated:
In regression, I think a common use of the term "saturated model" is as follows. A saturated model has as many independent variables as there are unique levels (combinations) of the coviarates. Of course this is only possible with categorical covariates. So if you have two dummy variables X1 and X2, a regression is saturated if the independent variables you include are X1, X2, and X1*X2.
This is advantageous because the conditional expectation function of Y given X1 and X2 is necessarily linear in parameters when the model is saturated (it is linear in X1, X2, X1*X2). Importantly, this model does not generally have "as many estimated parameters as data points," nor does it generally have a "perfect fit."
Here is one source for this, there are many others: "When would we expect the CEF to be linear? Two cases. One is if the data (the outcome and covariates) are multivariate Normal. The other is if the linear regression is saturated. A saturated regression model is one in which there is a parameter for each unique combination of the covariates. In this case, the regression fits the CEF perfectly because the CEF is a linear function of the dummy categories." Prof. Blackwell's lecture notes, page 2.