I read here that there are statistical approaches to assess whether a tractable machine learning model (e.g., a linear regression model) overfits a dataset:
Simpler models that have originated in statistics often don't need test datasets. Instead, what degree the model is overfit can be calculated directly as statistical significance: a ‘p-value’. These statistical methods are powerful, well established, and form the foundation of modern science. The advantage is that the training set doesn't ever need to be split and we get a much more precise understanding of how confident we can be about a model.
That is a huge advantage: we don't need to split the training data or perform cross-validation if the model is simple enough.
Unfortunately I haven't seen such a thing in practice. I know we get a p-value for each fitted coefficient when fitting regression models like here, but I don't believe these p-values determine overfitting.
Do you have a more elaborate explanation and preferably some examples of this use case?