Based on a response to the question "Statistical classification of text", I've decided that I want to use a random forest to perform various classifications of text.
Specifically, I am getting YouTube video data from their XML feeds. Here is an example:
<entry gd:etag="W/"DUQBR347eCp7I2A9WhdXFEQ."">
<id>tag:youtube.com,2008:video:ltkMAcv2cas</id>
<published>2010-12-23T18:11:28.000Z</published>
<updated>2011-08-28T02:09:16.000Z</updated>
<category scheme="http://schemas.google.com/g/2005#kind"
term="http://gdata.youtube.com/schemas/2007#video"/>
<category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat"
term="SSF4AE"/>
<!-- More categories/keywords here -->
<title>SSF4AE Mago 2D God VS Kazunoko aka Inoue</title>
<!-- More video information here, description, etc -->
</entry>
With that, I have very specific data that I believe I can use as the basis for variables in a random forest:
Variable Value
-------- -----
Title SSF4AE Mago 2D God VS Kazunoko aka Inoue
Category SSF4AE
Looking at the video and based on the title/categories, I know that this is a video of the game Super Street Fighter 4: Arcade Edition (SSF4AE).
Looking at the alglib Decision Forest, I also read the general principals of data analysis methods, which speaks about encoding specifically for the algorithms on alglib; while I won't use it specifically, I believe it expanded my understanding of how data should be represented for a random forest.
Basically, using the above data, I'd need to transform each data point into a number of some sort.
For example, I could transform this into the following variables:
Variable Value
-------- -----
Has "SSF4AE" in Title 1
Equals "SSF4AE" 1
Now, assuming I know that this video is for SSF4AE, and that I know that SSF4AE is a common acronym used to identify the game, would using the above as variables work for a random forest for training and classification?
Of course, that's a small set, and it's a very rough set of variables; my understanding is that a random forest basically will create nodes where there is a threshold for the variable, values over this fall on one side of the tree, values under on the other side.
In the above example, there are only two options, true or false.
However, I imagine I can do something like get the term-frequency/inverse document frequency and use that (using Lucene.NET or elasticsearch), like so (numbers are not accurate)
Variable Value
-------- -----
Term Query for "SSF4AE" in Title 0.36
Term Query for "SSF4AE" in Category 1.00
And then train the random forest on that.
I assume that if the variables were done as above, then I could also place things such as boosts on the variables (in the tf/idf calculation) where I see that they are more prominent or I feel I want them to have more weight.
And finally, I can add more variables, but they have to be very specific. For example, I know that if there is a character Yun that only appears in SSF4AE, (not previous versions of Street Fighter 4), as well as in Street Fighter III. To that end, I can add two more variables:
Variable Value
-------- -----
Term Query for "SSF4AE" in Title 0.36
Term Query for "SSF4AE" in Category 1.00
One character is Yun 1
Term Query for "3S" in Title 0.00
Term Query for "3SO" in Title 0.00
("3SO" and "3S" being common acronyms for Street Fighter III: Third Strike)
However, for another video, which I know is for 3S, I the variables might look like this:
Variable Value
-------- -----
Term Query for "SSF4AE" in Title 0.00
Term Query for "SSF4AE" in Category 0.00
One character is Yun 1
Term Query for "3S" in Title 0.54
Term Query for "3SO" in Title 0.00
Where the training set would indicate that video is for 3S.
That said, are these feasible ways of going about determining the variables of a random forest?
Also, if I understand correctly, the random forest will also indicate which variables contributed the most to the classification, so I can discard them during classification later (if I want) or add different ones to get better classifications from my training data.