Looking for a Picture that shows Statistical Models "Learning" from Data I am trying to find a picture (e.g. on Google Images) that illustrates the following concept:

*

*The existence of a "true ideal" function that by definition is never fully observable


*The process of a statistical model trying to "learn" this "true ideal" function based on data points from this "true ideal" function
The closest thing I could find was the following picture:

This picture clearly shows observations from the "true ideal" (and fundamentally unobservable) function and an attempt to "learn" this "true ideal" function based on limited observations.
Does anyone else have any other recommendations for pictures that illustrate this concept?
Reference:

*

*https://www.pushpakjagtap.com/project/data/

*https://blog.dominodatalab.com/fitting-gaussian-process-models-python
 A: It's easy to make your own. I'll do it in R.
set.seed(2022)

# Simulate some values of a predictor variable
#
x <- seq(-2, 2, 0.1)

# Define the true function
#
yhat <- x + 7 

# Add some noise to the function
#
y <- yhat + rnorm(length(x), 0, 1)

# Fit a model...linear regression here
#
L <- lm(y ~ x) 

# Define the predicted values for the x inputs
#
preds <- predict(L)

# Plot observed (x, y) pairs (the data))
#
plot(x, y)

# Plot true function in red on top of the data
#
lines(x, yhat, col = 'red') 

# Plot regression line in blue
#
lines(x, preds, col = 'blue') 


If you follow this rationale, you can make your plot for as complicated of a true function as you desire and as complicated of a model as you desire. For instance, I can use a neural network model of that same true linear function.
library(nnet)

# Simulate some values of a predictor variable
#
x <- seq(-2, 2, 0.1)

# Define the true function
#
yhat <- x + 7 

# Add some noise to the function
#
y <- yhat + rnorm(length(x), 0, 1) 

# Fit a model...neural network here
#     "size = 25" means that there are 25 neurons in the hidden layer
#     "linout = T" means to use a linear output, rather than the default sigmoid
#
L <- nnet::nnet(y ~ x, size = 25, linout = T) 

# Define the predicted values for the x inputs
#
preds <- predict(L)

# Plot observed (x, y) pairs (the data))
#
plot(x, y)

# Plot true function in red on top of the data
#
lines(x, yhat, col = 'red') 

# Plot regression line in blue
#
lines(x, preds, col = 'blue') 


A: I would put emphasis on "the process of a statistical model trying to "learn" this "true ideal" function". This is clearly what gradient descent does.
Consider a simple linear regression: yet there is an analytical solution for its coefficients ($\hat \beta = (X^T X)^{-1} X^T y$), coefficients are also often found with gradient descent. So, the optimization method starts in some "start point" and then moves regression line to minimize loss function and improve fit of model to data.
See example of visualisation: Scipython blog You can see there the "process" of fitting coefficients to data, as the regression line moves closer and closer to the true function at each iteration of the gradient descent.
