I have been studying machine learning on my own, from online videos and tutorials and by referring books from the library. I am finding it very difficult to understand the concepts. I am not speaking of the various algorithms but the main basic concept that machines can learn.
I have two questions as follows:
I implemented a neural network following Hebbian learning to change weights. After each iteration, I compute the error between the desired and actual output and change weights appropriately. I can also introduce a learning rate and change its value in accordance to the actual and desired output. By doing so, I am driving my neural network to reach the desired output. But, is this what is called learning? I read a lot of times that in supervised learning, more the number of training samples, the better the system can learn. I am confused here.
I am also implementing a simple very basic polarity detector for English sentences and here, from Brown corpus, I check if the sentence is negative by consulting the negative word dictionary in opinion_lexicon and say if the sentence is negative or positive. Here, I am converting the English words to binary values to predict if the sentence is positive or negative. Say, I spend a lot time gathering a lot of sentences, and annotating them as positive or negative manually and then, feed my polarity detector my results and then run the polarity detector as well and compare the obtained results with the manual results. How will this make the system learn? Am I again, supposed to vary the weights in accordance with the actual and desired output? If I get a million twitter posts or if I somehow gather a million sentences, is this how am I supposed to train my analyzer to detect their polarity?
The question in simple terms is, I find it difficult to comprehend the concept of making a program learn English from a million sentences, since as a programmer, I have this mindset of making programs "use" numerical data for any analysis. So, even if I have a million sentences, would I not be tagging them all as positive or negative, making a binary prediction again? Any classifier, be it HMM or KNN or logistic regression, they all work on numbers assigned, using a particular "rule", to a text or pixels or anything.
What am I missing here?
Please explain with some some references which will make my understanding of machine learning simpler.