There is a ton of literature (see, for example, a highly cited paper by Huang et al. (2006)) on neural networks with random weights (NNRWs), i.e. neural networks whose weights are random except for those in the output layer, which are determined analytically by solving a linear system.

Two recent publications seem to have differing opinions on the universal approximation capability of such networks. According to Weiping Cao et al. (2017):

NNRW is easy to implement and its universal approximation capability has been proven in theory.

According to Feilong Cao et al. (2016):

From approximation theory, it is obvious that such a way to randomly assign the input weights and biases cannot guarantee the universal approximation capability (in the sense of probability one) of the resulting random learner models.

Which of these papers has it right?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.