Beginner trying to improve text classification I'm a beginner in machine learning. Basically, I'm starting from the very beginning.
My problem
I have 100 text documents, with 200 pages. 
I would like to be able to detect topics in these documents (for example, detecting addresses, detecting when it talks about meditation, or absolutely anything...)
These topics are defined by users, and can be added or removed.
I'm thinking about supervised learning, since it will be done manually by users, I can't afford to mark 1M sentences as "meditation topic". 
What I currently do


*

*A user selects "I live in Brown street" and marks it as "address"

*I stem all the words

*I link these stem words to "address", and increase their score (live_address +1, brown_adress + 1, street_adress +1).

*When I parse a new document, I calculate the score of each word for each category for each sentence. If I see "I live in Oxford Street", the score will be 2(live_adress +1, street_adress +1) for this sentence. 

*If the score for this sentence is greater than X, I decide this might be an address.


Moreover, user have the ability to rate this suggestion. All the words in the sentence marked as address will have an updated score. Meaning if user downvote the sentence, "I love to live in the street" will have "love_address -1, live_address-1, street_address -1"
What I think is bad
First, I think as shown in the last example, this solution is not perfect. I don't take care of the sentence's context. The sentence "I live in Oxford Street in London" should have even a greater weight since we have Oxford, street, London in the same sentence. The words should be linked I guess.
Secondly, my database size will soon explode if I mark all the words I see for each topic, even more if I include the link between two words for each category and its weight.
My question
Is my algorithm that bad?
I think people way smarter than me have thought about this, and I'm sure there is a way to detect sentences in a text thanks to an input topic.
What is for you the best way to do this, regarding that we have 


*

*A supervised (or semi supervised) learning

*The ability to add or remove a topic to tell the machine "now you have to learn how to detect this"


Don't hesitate to assume I'm a beginner when you explain, because honestly this is totally new for me. 
I'm currently trying to understand Convolutional Neural Networks, is it worth it or will it be totally useless for my problem?
Any book, algorithm, video, API, software that could help me?
Thanks a lot.
 A: The strategy of having a score for each word, adding them up, and then checking against a threshold to decide if a topic is present or not is a common technique. When you don't care about the order of the words in the sentence it is called the bag-of-words method. There are lots of more sophisticated ways of learning scores for each word--instead of always doing +1 for each word like you did. You could look up the naive Bayes algorithm as a good place to get started.
I would recommend sticking with the bag-of-word approach (where single isolated words are treated individually) until you get more farther along. If the accuracy is not as good as you want you can add bi-grams, pairs of adjacent words, as features. For example, you would have scores for (I, live), (live, in), (in, Brown), (Brown, Street).
Convolutional neural networks can be used for the problem you are working on. It is just a different way of learning and combining scores for the words in each sentence. My advice is not to do that until you have a solid understanding of the simpler techniques.
