New answers tagged

1 vote

Is it normal for simple logistic regression to significantly outperform any other statistical ML algorithm?

My informal answer is that maximum likelihood estimation, the method behind logistic regression, finds the set of parameters that fit the data the best given some assumptions. If your dataset ...
user avatar
  • 3,070
0 votes

Is it normal for simple logistic regression to significantly outperform any other statistical ML algorithm?

I haven't done enough projects where I've compared different models to say whether Logistic Regression usually outperforms other ML algorithms so unfortunately I can't answer that part of the question....
user avatar
0 votes

Tensor Classification Models

Basically, your data consists of multivariate timeseries. Since you have a time-varying dimension, you are looking at a few models that can do that. 1D Convolutional Neural Networks: convolutions are ...
user avatar
  • 15.7k
1 vote

Tensor Classification Models

You can flatten the tensor and run the usual machine learning methods on the vector: random forest, kNN, SVM, logistic regression, multilayer perceptron neural networks, etc. This has been done for $2$...
user avatar
  • 30.8k
0 votes

High Performance Classification or Similarity Algorithim for Mixed Data Types?

There are lots of possibilities. One would be to obtain for each feature a "score" and then create as a total rank a weighted sum of those scores $s_i$, with weights $w_i$ chosen by a domain ...
user avatar
  • 2,843
7 votes
Accepted

If the AUC score is 100 percent can F1 value be 99.94 percent?

$AUC$ measures the separability of the probability outputs of your model. If the positive group's lowest probability of being positive is less than the negatives group's highest probability of being ...
user avatar
  • 30.8k
1 vote

Churn model- how to handle new users without enough historic data?

There is a technical difference between 0 and NA, and new users should have a NA rather than a 0 for their number of visits in the preceeding weeks. I think some implementations of trees are able to ...
user avatar
3 votes
Accepted

Clustering while knowing the ground truth: Why would someone choose this approach?

You would want to cluster instead of classifying when the real-world problems don't share the same categories as the evaluation set you use. For instance, let's say you know the true clusters of a ...
user avatar
0 votes
Accepted

Comparing impact of training data size - what testing data size?

There are two important things to consider here. First, unless you have a very high signal-to-noise ratio, your sample size is too small for reliable use of split-sample validation. See Frank Harrell'...
user avatar
  • 61.9k
2 votes

Comparing impact of training data size - what testing data size?

Yes - if you want to be sure the differences in accuracy are due only to the increase in the size of the training set, then use the same test data for each training size. If you want more confidence ...
user avatar
  • 422
5 votes
Accepted

Why exactly does a classifier need the same prevalence in the train and test sets?

"Why exactly does a classifier need the same prevalence in the train and test sets?" Perhaps my answer to a related question on the DS SE might help Doesn't over(/under)sampling an ...
user avatar
3 votes

Why exactly does a classifier need the same prevalence in the train and test sets?

I would proceed with caution. The only time I would rebalance would be if I knew the characteristics of the original population, so that when I did rebalanced it would reflect the proportion of the ...
user avatar
6 votes

class weighted classification

Against @gunes, I defend that you can use whatever metric you want. Yes accuracy may give you unexpected results in a imbalanced problem, but the choice of metric is based on the needs of your problem....
user avatar
5 votes

class weighted classification

If you want to include accuracy as a performance metric, balanced accuracy is a better choice than accuracy because of the imbalance in class distributions here. I would also recommend reading these ...
user avatar
  • 51.1k
1 vote
Accepted

Will a dataset with multiple labels perform better than with binary labels?

If the data is clustered in the way you describe in your comment, then for the two-class problem you are likely to have a non-linear decision boundary. As linear models can't model non-linear decision ...
user avatar
  • 422
2 votes

Why is a random forest regressor better than a random forest classifier when predicting a category?

From the comments: - How did you calculate accuracy for regression? - By using the method .score(X_test, y_test) RandomForestRegressor and RandomForestClassifier return completely different metrics ...
user avatar
  • 113k
1 vote

AUROC too high in image classification

It may be that your minority class(es) have poor performance in the hard classification (obtained by choosing the class with the largest predicted probability), but that their predicted probabilities ...
user avatar
  • 2,836
0 votes

Does it make sense to apply Bayesian formula on top on a classification problem output?

Sure you can. Bayes theorem is a general probability theory theorem, it can be applied to any probabilities (or probability densities). Bayes theorem is useful because it lets us "reverse" ...
user avatar
  • 113k
2 votes

how to deal with data leakage in historical data

I just came up with idea that: It’s not realistic to use future data to predict the past. That’s not a real use case. There could be some temporal related changes for players, and sanity wise, using ...
user avatar
3 votes

Why is cross entropy loss better than MSE for multi-class classification?

Crossentropy loss is equivalent to maximum likelihood estimation in a multinomial logistic regression. Consequently, we get all of the wonderful features of maximum likelihood estimation. This topic ...
user avatar
  • 30.8k
0 votes

How to compute a 'pair confusion matrix'?

Per sklearn documentation I understand it this way. Let say there are 2 label clusterings: true and pred ...
user avatar
  • 161
2 votes

Features are Relevant for Regression but not necessarily for Classification - what to make of this?

Loosely speaking, I would interpret it to mean that a subset of features are most important for determining the direction (gain/loss), and then the other features come up in determining the magnitude. ...
user avatar
  • 30.8k
2 votes

Gaussian Process for Classification

I am uncertain that this presented rationale holds strongly. I think it is mostly because that "assuming Gaussian noise (for the latent Gaussian process) and a step-function likelihood is exactly ...
user avatar
1 vote

How to plot the random and best model in Lift and gain charts?

I have never seen these kinds of plots before, but here's the idea. The "random" line is representing the results you would get if you did completely random guessing - so this is the "...
user avatar
0 votes
Accepted

How to get the True Negative Rate from this code?

The True Negative Ratio is the fraction of the correctly classified negative samples from all negative samples. You can compute it using numpy: ...
user avatar
  • 224
1 vote

How to tell if my features improve model performance?

This is what an out-of-sample test set reveals. In fact, machine learning tends not to care much about in-sample (“training set”) performance, since you can play connect-the-dots and memorize the data,...
user avatar
  • 30.8k
1 vote
Accepted

Expected Prediction Error for 0-1 Loss Function

Figured this out by writing the sum explicitly: The expected conditional loss given by selecting a class $g$ is given as $\sum P(G_i \neq g|X=x)$, which is effectively equivalent to $1-P(g|X=x)$.
user avatar
0 votes

What is a “center loss”?

In short, it tries to increase the inter-class distance of the embeddings using the softmax function and decrease the intra-class distance for embeddings of each class using the center loss. To make ...
user avatar
  • 5,141
1 vote

How to use time-series observations on multi-class classification problem?

One of the challenges of time series classification is that most ML algorithms assume the feature variables are independent of each other. That's often not the case in a time series, as a value of the ...
user avatar
  • 422
0 votes

Low classification accuracy

I see that you've edited to state that you're tuning n_estimators and max_depth. For random forest, the most important parameter ...
user avatar
  • 78.1k
11 votes

How to make regression results to be integers?

What you’re doing is an ordinal regression task, which TensorFlow seems to support, and I recommend looking into this approach. At the same time, remember Box’s famous quote. All models are wrong, ...
user avatar
  • 30.8k
0 votes

Ridge classification: Interpreting prediction

Are use sure that when changing the labels you didn't reverse them? Because both results you got are the same, just the second one got the labels in reverse, $1 - 0.65 = 0.35$. Both encodings are ...
user avatar
  • 113k
0 votes

Building a multilabel classifier for text

I think the latter approach is likely to give better results, because the model doesn't have to disambiguate the segments' words' contribution to the multiple labels. However, that comes at the cost ...
user avatar
  • 2,836
-1 votes

Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data?

Machine Learning and Statistical inference deal with different type of problems and are not comparable in this point of view. Statistical inference is used in problems that are inherently statistic, ...
user avatar
4 votes

Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data?

Machine learning (often) needs a lot of data because it doesn't start with a well defined model and uses (additional) data to define or improve the model. As a consequence there are often a lot of ...
user avatar
18 votes

Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data?

Machine learning does not require large amounts of data, it is just that the current bandwagon is for models that work on big data (mainly deep neural networks, which have been around since the 1990s, ...
user avatar
3 votes

Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data?

A typical machine learning model contains thousands to millions of parameters, while statistical modelling is typically limited to a handful parameters. As a rule of thumb, the minimum an amount of ...
user avatar
31 votes

Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data?

All/other things being equal (when?) machine learning models require similar quantities of data as statistical models. In general statistical models tend to have more assumptions than machine learning ...
user avatar
  • 6,145
19 votes

Why does Machine Learning need a lot of data while one can do statistical inference with a small set of data?

Well, you could do inference with a small amount of data. We just have concepts like statistical power to tell us when our results would be reliable and when they would not be. In general, lots of ...
user avatar
1 vote

After training a binary classifier, are TPR & FPR independent of a test set?

Nope. Not really sure you can conclude anything useful here. If you think about your model as assigning probabilities of observations being true then I think the issue is more clear. Example Let's say ...
user avatar
  • 1,688
0 votes

How low does the cross entropy loss need to be for me to be confident in my model?

The following chart demonstrates distribution of cross-entropy for random guessing based on the label distribution. It is clear that in the middle of the chart the cross-entropy reaches its maximum (....
user avatar
  • 181

Top 50 recent answers are included