1
$\begingroup$

What is a precise definition of shallow architecture in machine learning?

$\endgroup$
1
  • 4
    $\begingroup$ You should always try to give some context to your question! $\endgroup$ Jun 9, 2014 at 20:26

4 Answers 4

4
$\begingroup$

It means an artificial neural networks with few hidden layers. To be contrasted with deep neural networks. The precise boundary is debatable.

$\endgroup$
1
$\begingroup$

Shallow and deep refer to two different (but related) ways to go about modelling a problem. Shallow architectures rely on the paper by Cybenko: Approximation by Superposition of a Sigmoidal Function, where the shows that arbitrary decision regions can be arbitrarily well approximated by MLPs.

Basically it means that you would just add more neurons to your layer, thus making your network wider, as the complexity of your problem at hand increases (in very rough terms). My point is thus, that there is no precise definition to that term. Is more about how you approach the complexity of a problem.

$\endgroup$
1
$\begingroup$

I believe usually "shallow" means only one hidden layer. For example:

1. MLP with one hidden layer: data --> hidden --> softmax (class label)
2. SVM: data --> feature (can be considered as hidden) --> class label

Anything with more than 2 hidden layers (inclusive) can be called deep.

$\endgroup$
0
$\begingroup$

according to "Deep Learning Methods and Applications By Li Deng and Dong Yu", shallow-structured architectures, typically contain at most one or two layers of non-linear feature transformations. Examples of shallow architectures are Gaussian mixture models (GMMs), linear or non-linear dynamical systems, conditional random fields (CRFs), maximum entropy (MaxEnt) models, support vector machines (SVMs), logistic regression, kernel regression, multilayer perceptrons (MLPs) with a single hidden layer including extreme learning machines (ELMs). For instance, SVMs use a shallow linear pattern separation model with one or zero feature transformation layer when the kernel trick is used or otherwise.

$\endgroup$
1
  • 2
    $\begingroup$ According to which book? $\endgroup$
    – Sycorax
    Sep 11, 2019 at 16:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.