The universal approximation theorem says that a neural network can be used to approximate any continuous function under some regular conditions with arbitrarily small approximation error, provided we have enough number of nodes in the hidden layer.

Suppose a response y depend on a p-dimensional vector x only through s of the entries (s

However, in practice, this is not usually the case. Why neural network can not learn the sparse structure automatically? Is it because of the non-convexity? Or we don't have enough sample size to learn the specific relationship?

  • $\begingroup$ Welcome to Cross-Validated. Have you checked for other posts that might have answers to you question first? Check here stats.stackexchange.com/questions/345737/… $\endgroup$
    – Skander H.
    Jun 25 '19 at 1:52
  • $\begingroup$ Welcome to CV! It looks like part of your question was deleted. Can you double-check and make sure the whole text is there. $\endgroup$ Jun 29 '19 at 23:57