I have been working on some self study "machine learning". Based on a few posts here, I wanted to make a program that "learned" via Bayes Law. I test it with some simple truth tables. It recalls the past training data well. I note that some machines are able to make inferances in new situations, to different degrees. My particular version cannot do so, which raises a question. Perhaps it doesn't learn, perhaps it only regurgitates.

My question is: In a broader, philosophical sense, does a program (any program) still qualify as "learning" if it cannot infer about things that it has not seen historically? What are the bounds on such things?

  • 1
    $\begingroup$ If you do not provide us your code we cannot comment on it - it may be just a bug in the code that makes it not work as you'd like. We don't even know what your algorithm is! The fact that your algorithm does not work on your data does not mean that in general such algorithms do not work. $\endgroup$ – Tim Mar 21 '15 at 7:43
  • $\begingroup$ That's not the question, I accept that such algorithms work. The code above works perfectly as I would like. The question is: Can something (my code for instance) be said to learn if it cannot reason beyond past experiences? What's the definition on learning? $\endgroup$ – RegressForward Mar 21 '15 at 12:40
  • $\begingroup$ Either case thank you for the question, You've allowed me to make a considerable improvement in clarity. $\endgroup$ – RegressForward Mar 21 '15 at 13:07

Just my two cents. Generally, as you most likely know, Bayesian networks represent a foundation of a large subset of machine learning approaches, methods and algorithms. For example, see this paper. I think that the fact that your particular implementation of Bayesian inference/learning fails to produce correct probablities from your specific and particular data set does not really question Bayesian inference as a machine learning tool.

P.S. A note from the "terminology police" :-): in R , bayes() is not a command - it's a function.

  • $\begingroup$ Oh- Good point on function vs command. But that's not quite my question, I'm actually very happy with the implementation. 9000 draws might not be enough to get the probabilities perfect - the approach is very rudimentary! Specifically: does it still "learn" if it it simply cannot ever infer a probability for row 3? $\endgroup$ – RegressForward Mar 19 '15 at 0:16
  • $\begingroup$ @RegressForward: I'm not sure about whether the code "learns" - I guess it depends on how you define "learning" in this context. I just find a bit odd that two probabilities "hover" around 0.5. Maybe that's OK, but maybe that reveals some issue with the code. Let's see what other people think about your question. $\endgroup$ – Aleksandr Blekh Mar 19 '15 at 0:30
  • 1
    $\begingroup$ I improved the code so it does not resample with replacement. (Which is why it was only -close- the correct value, it grabbed a few too many samples of one sort rather than the other.) $\endgroup$ – RegressForward Mar 20 '15 at 17:42
  • $\begingroup$ @RegressForward: Great! I'm glad that you've figured it out. Thank you for the update (consider updating your question with the new code or posting it as an answer). $\endgroup$ – Aleksandr Blekh Mar 20 '15 at 18:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.