I'm studying machine learning and I feel there is a strong relationship between the concept of VC dimension and the more classical (statistical) concept of degrees of freedom.

Can anyone explain such a connection?


Yaser Abu-Mostafa --- Learning with data

Degrees of freedom are an abstraction of the effective number of parameters. The effective number is based on how many dichotomies one can get, rather than how many real-valued parameters are used. In the case of 2-dimensional perceptron, one can think of slope and intercept (plus a binary degree of freedom for which region goes to +1), or one can think of 3 parameters w_0,w_1,w_2 (though the weights can be simultaneously scaled up or down without affecting the resulting hypothesis). The degrees of freedom, however, are 3 because we have the flexibility to shatter 3 points, not because of one way or another of counting the number of parameters.

2-d perceptron

  • 1
    $\begingroup$ It looks like you're quoting a resource with this answer, could you please attribute it to its original source? $\endgroup$ – Matthew Drury Jun 14 '15 at 18:49
  • 1
    $\begingroup$ Yes, of course. I will add the source as soon as I will be in front of my pc! $\endgroup$ – stochazesthai Jun 14 '15 at 18:50
  • $\begingroup$ Source: book.caltech.edu/bookforum/showthread.php?t=4251 $\endgroup$ – Vimal Jan 22 '18 at 22:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.