Hot answers tagged classification
9
Thanks for a very good question! I will try to give my intuition behind it.
In order to understand this, remember the "ingredients" of random forest classifier (there are some modifications, but this is the general pipeline):
At each step of building individual tree we find the best split of data
While building a tree we use not the whole dataset, but ...
7
Expanding on my comment; let's simplify. Suppose there is one Boolean variable A. Suppose that, if the animal is a chicken, A is true 60% of the time and, if the animal is a penguin, it is true 10% of the time.
Now given A, what is the probability it is a chicken?
No way to know. We could have
1000 chickens - 600 with A = 1 and 400 with A = 0
10 penguins ...
7
I think a first step is to check whether the reported training and test performance are in fact correct.
Is the splitting during the 5-fold cross validation done in a way that yields statistically independent cv train/test sets? E.g. if there are repeated measurements in the data, do they always end up in the same set?
95.83% accuracy in a 5-fold cv of ...
3
Support vector machines focus only on the points that are the most difficult to tell apart, whereas other classifiers pay attention to all of the points.
The intuition behind the support vector machine approach is that if a classifier is good at the most challenging comparisons (the points in B and A that are closest to each other in Figure 2), then the ...
3
Proportion classified correctly is an improper scoring rule, i.e., it is optimized by a bogus model. I would use the quadratic proper scoring rule known as the Brier score, or the concordance probability (area under ROC curve in the binary $Y$ case). Random forest works better than SVM in your case.
3
Decision Trees and Random Forests are actually extremely good classifiers. While SVM's are seen as more complex it does not actually mean they will perform better.
The paper "An Empirical Comparison of Supervised Learning Algorithms" by Rich Caruana compared 10 different binary classifiers, SVN, Neural-Networks, KNN, Logistic Regression, Naive Bayes, ...
3
To elaborate on the answers of our colleagues above: say you map purple, blue, red to $x = 1, 2, 3$. Say $x$ represents the colour of a hat, and $y$ sales. Then if we have a model with an intercept, call it $a$ and the coefficient of $x$, call it $b$, then we'd be saying:
$y = a + b x$
We only get to choose one $b$ here, which has to cater for all the ...
2
It depends what data are represented by colours.
You can use colour as a single variable with colours expressed by numbers in regression if colours represent a scale for some data. For example, purple means wind speed 10 m/s, blue means 20 m/s, and red colour means wind speed 30 m/s. So we can say something (wind in example) is two times larger if the ...
2
The proportion classified correctly is an improper scoring rule (i.e., it is optimized by a bogus model) and is a discontinuous function of the data. Therefore the methods used to estimate its statistical properties need to be more carefully chosen. For bootstrapping, the particular variant of the bootstrap you choose will matter, whereas for proper and ...
2
It sounds like a two component gaussian mixture model would be exactly what you're looking for.
I don't tend to use R, but GMMs are so commonly used I can't imagine it they're not implemented somewhere. If you really can't find them anywhere then they're not too hard to implement yourself. Wikipedia has the exact expectation maximisation update steps ...
2
There are several procedures and measures that you can caluclate to describe the so-called inter-rater agreement. One of the most famous is Cohen's kappa. You can calculate it in R with the function kappa2 from the irr package. Another measure is the intraclass-correlation. You can calculate it in R with the icc function from the irr package. Here is a good ...
2
For what I understand, once a new document arrives, SVM just applies the discriminant function and decides if the document is going to be classified or not. This would mean that the kernel function is not exploited in the test phase.
You can find a very good introduction to SVM in this tutorial by a UCL PhD student. In addition, this videolecture from the ...
1
If it is only 70%-30% there is probably no really need to balance the dataset. The class imbalance problem is caused by there not being enough patterns belonging to the minority class, not by the ratio of positive and negative patterns itself per se. Generally if you have enough data, the "class imbalance problem" doesn't arise. Note also that if you ...
1
As a general principle:
0) you keep the sufficient statistics and the current ML estimates
1) when you get new data, update the sufficient statistics
2) use the previous MLEs as the starting point, use some convenient optimization method to find the new optimum from there. You may need to experiment a bit to find which approaches make the best tradeoffs ...
1
"whether a problem is linear or not"
In a binary classification problem, if the dataset can be separated by a hyper-plane, it's a linear problem.
If the dataset is not linear separable, while you try a linear classifier to find such a hyper-plane that is not existed at all, the algorithm may seem to run forever.
One suggestion: You can sample a small ...
1
Possibilities include the use of an inappropriate kernel (e.g. a linear kernel for a non-linear problem), poor choice of kernel and regularisation hyper-parameters. Good model selection (choice of kernel and hyper-parameter tuning is the key to getting good performance from SVMs, they can only be expected to give good results when used correctly).
SVMs ...
1
It depends on the specific naive model you apply. In general, for Text Classification, you do want to consider repetition of terms, so the answer is yes.
The other point is that you are considering the probability based on the document event space. You can also do it based on the term space:
p(viagra | spam) = 5 times spam term in class spam / 50 terms in ...
1
This is called Dataset Shift setting. This pdf [1] should help you understand several of the underlying issues involved.
For the moment however, you can use least squares importance fitting to obtain importance estimates for your training data using your test set (you don't need the test set labels, just the feature vectors) [2]. Once you gain the ...
1
have you read up on land degradation theory? i think the issue is that you are trying use RF to classify degradation into a static map, when degradation is something that occurs over time. in degradation theory, you would have a vegetation state at time t0...tn, but the static map cannot be used to evaluate a vegetative state in a time series. how a state ...
1
@StephanKolassa It looks like your data could fit a Rasch model. Those individuals with large amounts of missing data will wind up with larger standard errors. As long as you have a sufficient sample size, you should be able to get stable estimates for persons, items, and groups. I do not know how to do this in R, but could do this in Facets ...
Only top voted, non community-wiki answers of a minimum length are eligible
