# In a neural network, why can't there be more weights than the number of observations?

After having this exact same issue with caret, I arrived at this thread. However, I do not intuitively understand why this answer is correct.

Why can't there be more weights than the number of observations? Is this a bug/idiosyncrasy in this particular R package or is there a statistical reason for it?

Consider training an MNIST network in Keras. You can have stellar out-of-sample accuracy when you have more than 60,000$$^{\dagger}$$ weights, so certainly a neural network model allows for more weights than observations.
$$^{\dagger}$$There are 60,000 training images in the MNIST data set.