Universal approximation capability of neural networks with random weights

Question

There is a ton of literature (see, for example, a highly cited paper by Huang et al. (2006)) on neural networks with random weights (NNRWs), i.e. neural networks whose weights are random except for those in the output layer, which are determined analytically by solving a linear system.

Two recent publications seem to have differing opinions on the universal approximation capability of such networks. According to Weiping Cao et al. (2017):

NNRW is easy to implement and its universal approximation capability has been proven in theory.

According to Feilong Cao et al. (2016):

From approximation theory, it is obvious that such a way to randomly assign the input weights and biases cannot guarantee the universal approximation capability (in the sense of probability one) of the resulting random learner models.

Which of these papers has it right?

Answer 1

Not sure if you still need an answer but just in case. The paper you linked by Weiping Cao et al states the following:

NNRW have shown good generalization performance on the problems with higher noise, how to prove it in theory ... [is] not clear

The universal approximation ability of ELM and its variations have been rigorously proved in theory, while there is no theoretical proof for Schmidts method. Both Broomhead et al. [65] and Paos group [8,9] also did not give rigorous proof for full random hidden nodes cases.

Both of which are in line with Feilong Cao et al's work