New answers tagged classification
1
vote
Is it normal for simple logistic regression to significantly outperform any other statistical ML algorithm?
My informal answer is that maximum likelihood estimation, the method behind logistic regression, finds the set of parameters that fit the data the best given some assumptions. If your dataset ...
- 3,070
0
votes
Is it normal for simple logistic regression to significantly outperform any other statistical ML algorithm?
I haven't done enough projects where I've compared different models to say whether Logistic Regression usually outperforms other ML algorithms so unfortunately I can't answer that part of the question....
- 1
0
votes
Tensor Classification Models
Basically, your data consists of multivariate timeseries.
Since you have a time-varying dimension, you are looking at a few models that can do that.
1D Convolutional Neural Networks: convolutions are ...
- 15.7k
1
vote
Tensor Classification Models
You can flatten the tensor and run the usual machine learning methods on the vector: random forest, kNN, SVM, logistic regression, multilayer perceptron neural networks, etc.
This has been done for $2$...
- 30.8k
0
votes
High Performance Classification or Similarity Algorithim for Mixed Data Types?
There are lots of possibilities.
One would be to obtain for each feature a "score" and then create as a total rank a weighted sum of those scores $s_i$, with weights $w_i$ chosen by a domain ...
- 2,843
7
votes
Accepted
If the AUC score is 100 percent can F1 value be 99.94 percent?
$AUC$ measures the separability of the probability outputs of your model. If the positive group's lowest probability of being positive is less than the negatives group's highest probability of being ...
- 30.8k
1
vote
Churn model- how to handle new users without enough historic data?
There is a technical difference between 0 and NA, and new users should have a NA rather than a 0 for their number of visits in the preceeding weeks. I think some implementations of trees are able to ...
- 25.8k
3
votes
Accepted
Clustering while knowing the ground truth: Why would someone choose this approach?
You would want to cluster instead of classifying when the real-world problems don't share the same categories as the evaluation set you use.
For instance, let's say you know the true clusters of a ...
- 6,702
0
votes
Accepted
Comparing impact of training data size - what testing data size?
There are two important things to consider here.
First, unless you have a very high signal-to-noise ratio, your sample size is too small for reliable use of split-sample validation. See Frank Harrell'...
- 61.9k
2
votes
Comparing impact of training data size - what testing data size?
Yes - if you want to be sure the differences in accuracy are due only to the increase in the size of the training set, then use the same test data for each training size.
If you want more confidence ...
- 422
5
votes
Accepted
Why exactly does a classifier need the same prevalence in the train and test sets?
"Why exactly does a classifier need the same prevalence in the train and test sets?"
Perhaps my answer to a related question on the DS SE might help
Doesn't over(/under)sampling an ...
- 48.4k
3
votes
Why exactly does a classifier need the same prevalence in the train and test sets?
I would proceed with caution. The only time I would rebalance would be if I knew the characteristics of the original population, so that when I did rebalanced it would reflect the proportion of the ...
- 901
6
votes
class weighted classification
Against @gunes, I defend that you can use whatever metric you want. Yes accuracy may give you unexpected results in a imbalanced problem, but the choice of metric is based on the needs of your problem....
- 5,147
5
votes
class weighted classification
If you want to include accuracy as a performance metric, balanced accuracy is a better choice than accuracy because of the imbalance in class distributions here. I would also recommend reading these ...
- 51.1k
1
vote
Accepted
Will a dataset with multiple labels perform better than with binary labels?
If the data is clustered in the way you describe in your comment, then for the two-class problem you are likely to have a non-linear decision boundary. As linear models can't model non-linear decision ...
- 422
2
votes
Why is a random forest regressor better than a random forest classifier when predicting a category?
From the comments:
- How did you calculate accuracy for regression?
- By using the method .score(X_test, y_test)
RandomForestRegressor and RandomForestClassifier return completely different metrics ...
Tim♦
- 113k
1
vote
AUROC too high in image classification
It may be that your minority class(es) have poor performance in the hard classification (obtained by choosing the class with the largest predicted probability), but that their predicted probabilities ...
- 2,836
0
votes
Does it make sense to apply Bayesian formula on top on a classification problem output?
Sure you can. Bayes theorem is a general probability theory theorem, it can be applied to any probabilities (or probability densities). Bayes theorem is useful because it lets us "reverse" ...
Tim♦
- 113k
2
votes
how to deal with data leakage in historical data
I just came up with idea that:
It’s not realistic to use future data to predict the past. That’s not a real use case.
There could be some temporal related changes for players, and sanity wise, using ...
3
votes
Why is cross entropy loss better than MSE for multi-class classification?
Crossentropy loss is equivalent to maximum likelihood estimation in a multinomial logistic regression. Consequently, we get all of the wonderful features of maximum likelihood estimation.
This topic ...
- 30.8k
0
votes
How to compute a 'pair confusion matrix'?
Per sklearn documentation I understand it this way.
Let say there are 2 label clusterings: true and pred
...
- 161
2
votes
Features are Relevant for Regression but not necessarily for Classification - what to make of this?
Loosely speaking, I would interpret it to mean that a subset of features are most important for determining the direction (gain/loss), and then the other features come up in determining the magnitude.
...
- 30.8k
2
votes
Gaussian Process for Classification
I am uncertain that this presented rationale holds strongly.
I think it is mostly because that "assuming Gaussian noise (for the latent Gaussian process) and a step-function likelihood is exactly ...
- 35k
1
vote
How to plot the random and best model in Lift and gain charts?
I have never seen these kinds of plots before, but here's the idea.
The "random" line is representing the results you would get if you did completely random guessing - so this is the "...
- 1,082
0
votes
Accepted
How to get the True Negative Rate from this code?
The True Negative Ratio is the fraction of the correctly classified negative samples from all negative samples.
You can compute it using numpy:
...
- 224
1
vote
How to tell if my features improve model performance?
This is what an out-of-sample test set reveals. In fact, machine learning tends not to care much about in-sample (“training set”) performance, since you can play connect-the-dots and memorize the data,...
- 30.8k
1
vote
Accepted
Expected Prediction Error for 0-1 Loss Function
Figured this out by writing the sum explicitly: The expected conditional loss given by selecting a class $g$ is given as $\sum P(G_i \neq g|X=x)$, which is effectively equivalent to $1-P(g|X=x)$.
- 11
0
votes
What is a “center loss”?
In short, it tries to increase the inter-class distance of the embeddings using the softmax function and decrease the intra-class distance for embeddings of each class using the center loss.
To make ...
- 5,141
1
vote
How to use time-series observations on multi-class classification problem?
One of the challenges of time series classification is that most ML algorithms assume the feature variables are independent of each other. That's often not the case in a time series, as a value of the ...
- 422
0
votes
Low classification accuracy
I see that you've edited to state that you're tuning n_estimators and max_depth. For random forest, the most important parameter ...
- 78.1k
11
votes
How to make regression results to be integers？
What you’re doing is an ordinal regression task, which TensorFlow seems to support, and I recommend looking into this approach. At the same time, remember Box’s famous quote.
All models are wrong, ...
- 30.8k
0
votes
Ridge classification: Interpreting prediction
Are use sure that when changing the labels you didn't reverse them? Because both results you got are the same, just the second one got the labels in reverse, $1 - 0.65 = 0.35$.
Both encodings are ...
Tim♦
- 113k
0
votes
Building a multilabel classifier for text
I think the latter approach is likely to give better results, because the model doesn't have to disambiguate the segments' words' contribution to the multiple labels. However, that comes at the cost ...
- 2,836
-1
votes
Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data?
Machine Learning and Statistical inference deal with different type of problems and are not comparable in this point of view.
Statistical inference is used in problems that are inherently statistic, ...
4
votes
Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data?
Machine learning (often) needs a lot of data because it doesn't start with a well defined model and uses (additional) data to define or improve the model. As a consequence there are often a lot of ...
- 46.2k
18
votes
Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data?
Machine learning does not require large amounts of data, it is just that the current bandwagon is for models that work on big data (mainly deep neural networks, which have been around since the 1990s, ...
- 48.4k
3
votes
Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data?
A typical machine learning model contains thousands to millions of parameters, while statistical modelling is typically limited to a handful parameters.
As a rule of thumb, the minimum an amount of ...
- 31
31
votes
Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data?
All/other things being equal (when?) machine learning models require similar quantities of data as statistical models. In general statistical models tend to have more assumptions than machine learning ...
- 6,145
19
votes
Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data?
Well, you could do inference with a small amount of data. We just have concepts like statistical power to tell us when our results would be reliable and when they would not be.
In general, lots of ...
- 25.8k
1
vote
After training a binary classifier, are TPR & FPR independent of a test set?
Nope. Not really sure you can conclude anything useful here. If you think about your model as assigning probabilities of observations being true then I think the issue is more clear.
Example
Let's say ...
- 1,688
0
votes
How low does the cross entropy loss need to be for me to be confident in my model?
The following chart demonstrates distribution of cross-entropy for random guessing based on the label distribution. It is clear that in the middle of the chart the cross-entropy reaches its maximum (....
- 181
Top 50 recent answers are included