0
$\begingroup$

I am watching a video on Thompson Sampling for Machine Learning and at [16:38] the presenter mentions that we can use various models to fit the data, (such as Gaussian Process, SVM in my opinion), but he used Neural network as it doesnt imply any structure on the model so it can go and discover what ever.

What does in it mean that neural Network doesnt imply any structure, what is structure here?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Negative log-likelihood based optimization has a big assumption attached. Unless he is learning a loss function implicitly with semi-supervised learning, then Neural Networks aren't as free as is implied. Forests in general are way more assumption free in my opinion. $\endgroup$
    – Firebug
    Dec 25, 2019 at 22:40
  • $\begingroup$ So -ve log-likelihood based optimization impose a constraint to the learning process and hence cant be considered model-free? $\endgroup$
    – GENIVI-LEARNER
    Dec 26, 2019 at 11:32
  • 1
    $\begingroup$ Yes, you must assume a distribution to perform likelihood-based estimation. Most (not all) neural networks do that. $\endgroup$
    – Firebug
    Dec 26, 2019 at 15:49
  • $\begingroup$ @Firebug oh i see so that "in general" if we are using likelihood based estimation, we are "implying" a distribution. right? $\endgroup$
    – GENIVI-LEARNER
    Dec 26, 2019 at 21:44

1 Answer 1

Reset to default
0
$\begingroup$

Structure in this context refers to a predetermined model or assumptions related to that model or the underlying sample or population distributions. For example when we use a Linear Regression model to fit the data we assume that the assumptions for linear regressions hold for our data. When we use a random forrest, assumptions about random forrests need to hold. The structure of the model also makes it tractable. A neural network on the other hand has no predetermined structure or assumptions about the underlying distribution of the data. The downside is that the resulting model is not tractable and cannot be interpreted easily.

Improve this answer
$\endgroup$
6
  • $\begingroup$ So the resulting model cannot be analytically represented like in the case of others. right? $\endgroup$
    – GENIVI-LEARNER
    Dec 25, 2019 at 21:59
  • $\begingroup$ Yes it cannot be represented analytically. The neural network would be the weighted connections between your perceptron layers, but they do not have any meaning outside the model. $\endgroup$
    – tomanizer
    Dec 25, 2019 at 22:05
  • $\begingroup$ What assumptions are there in Random Forests? They are just, if not more, liberal than the ones in Neural Networks. And Neural Networks do not lack structure and (in their most successful applications) do make structural assumptions on data as well. $\endgroup$
    – Firebug
    Dec 25, 2019 at 22:42
  • $\begingroup$ Your are correct. Random forrests was a poorly chosen example since it makes no assumptions about the structure. There are some ongoing discussions about the effect of multicolinearity on random forrests on this site and elsewhere. In any case I should have chosen a model with strict assumptions as an example. $\endgroup$
    – tomanizer
    Dec 25, 2019 at 22:50
  • 1
    $\begingroup$ @GENIVI-LEARNER no, even the objective function imposes an assumption on the distribution of the outputs. Also, neural networks are so diverse it's really hard to pinpoint what they even are. See this question $\endgroup$
    – Firebug
    Dec 26, 2019 at 15:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.