Questions tagged [classification]

Statistical classification is the problem of identifying the sub-population to which new observations belong, where the identity of the sub-population is unknown, on the basis of a training set of data containing observations whose sub-population is known. Therefore these classifications will show a variable behavior which can be studied by statistics.

Filter by
Sorted by
Tagged with
0 votes
0 answers
3 views

How to deal with unknown classes with a convolution neural network classifier?

I'm quite new into the DL and ML field. I'm training a CNN able to classify 3 different classes, however I would like in the testing phase to make the CNN able to not misclassify images that do not ...
user avatar
  • 1
0 votes
0 answers
10 views

Post analysis using raw data or SHAP values in Machine learning

Let's say I have SHAP value returned in dataframe for input variables like below ...
user avatar
  • 1,758
3 votes
1 answer
57 views

Usefulness of KS tests and other similar distribution comparing tests

I am working on a machine learning binary classification problem. I have an outcome variable status called as loan paid and <...
user avatar
  • 1,758
0 votes
0 answers
8 views

Why PR score is down when balanced accuracy is good?

I just read this discussion here and here. I have a dataset of 977 records where class proportion is 77:23. My balanced accuracy is 75.5, ...
user avatar
  • 1,758
0 votes
0 answers
7 views

What methods can I used to assign cases to groups when some variables are dependant on others?

I have a sample size of 42 cases, with about 5 variables for each case. Most of these variables are measurements, and I want to assign the cases into 2 groups (condition present, and condition absent) ...
user avatar
  • 1
1 vote
0 answers
12 views

Optimize classification rule in multinomial logistic regression

We know that in the case of logistic regression, a classification threshold p=0.5 is generally not an optimal choice when seeking to optimise sensitivity and sensitivity. This is generally due to the ...
user avatar
  • 138
0 votes
0 answers
8 views

Why variable representation plays a role in prediction?

I am working on binary classification using a random forest, where the data have 977 records and 6 columns. The class ratio is 77:23. I have two derived input variables. One variable is called ...
user avatar
  • 1,758
0 votes
0 answers
8 views

Precision vs Recall Tradeoff plots 2 separated lines

I'm trying to build a binary classifier with high recall and slightly better precision so as to avoid a lot of False Positives. So far the best scores I have got from all different types of model ...
user avatar
1 vote
2 answers
67 views

Is it normal for simple logistic regression to significantly outperform any other statistical ML algorithm?

I'm working on a simple classification project with an imbalanced (minority-to-majority-ratio ~ 0.2) dataset that has ~4000 rows and ~200 features. I noticed that, for my dataset, a simple logistic ...
user avatar
  • 11
0 votes
0 answers
15 views

How to conduct t-test for comparing the accuracy of two binary classifiers? [closed]

I am using two binary classifiers that predict the accuracy of samples over a dataset. I need to check if the difference in the mean accuracy between the two models is statistically significant. ...
user avatar
1 vote
0 answers
10 views

How do I perform a train-validation split on data with class imbalance such that the class imbalance ratio is preserved?

My data has class imbalance-- that is, some classes have significantly fewer training samples than the others. I want to perform a train-validation split in such as way that the class ratios are ...
user avatar
1 vote
0 answers
13 views

Risk score uncertainty quantification

I am working on various risk score estimation problems. I assume individual subjects are associated with a true risk $$ r_i = f(x_i) + \varepsilon_i$$ where $x_i$ is some available information about ...
user avatar
  • 954
1 vote
1 answer
10 views

High Performance Classification or Similarity Algorithim for Mixed Data Types?

I have a database holding 10-ish features that describe different breeds of dogs. They are mostly categorical features, but some provide ranges for values. Here's a demo representation of the database,...
user avatar
3 votes
1 answer
155 views

If the AUC score is 100 percent can F1 value be 99.94 percent?

If the AUC score is 100 percent can the F1 value be 99.94 percent? I would expect 100 percent, too.
user avatar
  • 189
0 votes
1 answer
19 views

Churn model- how to handle new users without enough historic data?

I'm making a churn model. My observation window (historic data) length is 3 weeks. There are some users that are not been registered to the app that I'm analyzing for three weeks, and as a result, I ...
user avatar
  • 67
0 votes
0 answers
12 views

Handle User Behavior Change when creating a ML classifer

I'm creating a churn model. My first thought was that the bigger the training set, it would be better. However, 2020 was a crazy year because the COVID 19. For example, a user who was sick and ...
user avatar
  • 67
0 votes
0 answers
20 views

Can I convert a classification problem with an ordinal dependent variable in a regression problem?

It looks interesting to me to know about the variables related to the students performance, so I started to look into the following dataset: https://archive.ics.uci.edu/ml/datasets/Higher+Education+...
user avatar
0 votes
0 answers
11 views

Is there a dataset of images with varying sizes? [closed]

I'm working on a project dealing with image classification where I deal with images with varying sizes. I would like to validate my approach in other datasets. I would like to find other datasets with ...
user avatar
  • 378
2 votes
1 answer
59 views

Clustering while knowing the ground truth: Why would someone choose this approach?

If the ground truth of the class/cluster/segment that our observations belong to, is known in advance, why would someone choose to perform clustering instead of classification? In fact, doesn't the ...
user avatar
4 votes
2 answers
110 views

Why exactly does a classifier need the same prevalence in the train and test sets?

Here are some commonly seen statements about the importance of prevalence in the train and test sets when developing a classifier: "Another reason not to rebalance datasets is that models should ...
user avatar
  • 902
0 votes
0 answers
13 views

Possible ways to feed two variables containing text data into an ML model in an NLP problem

In a natural language processing (NLP) problem, we have a couple of variables, say A and B. A denotes a phrase (1-2 words) and B denotes another phrase (>3 words). There is one target variable that ...
user avatar
0 votes
2 answers
29 views

Comparing impact of training data size - what testing data size?

I am training a classifier using BERT and want to check how the accuracy changes with increasing training data size. Up until now, I have 1k annotated training samples and tested the accuracy for ...
user avatar
  • 3
2 votes
2 answers
302 views

class weighted classification

I am working on my multi-class classification project and I have a question: I have three classes in proportion: 50%, 47% and 3%. I decided to use class_weight="balanced" parameter in random ...
user avatar
  • 31
0 votes
0 answers
9 views

GPT-2: Why should text classification work on the last output embedding?

When GPT-2 is fine-tuned for text classification (positive vs. negative), the head of the model is a linear layer that takes the LAST output embedding and outputs 2 class logits. I still can't grasp ...
user avatar
1 vote
1 answer
36 views

Why is a random forest regressor better than a random forest classifier when predicting a category?

I am building a model that recommends the optimal golf club based on data I have gathered. Since the model prediction should be a category, ie. a golf club, I would assume I would have to use a ...
user avatar
0 votes
0 answers
7 views

Fuzzy membership - Kernel SVM in R

I'm trying to perform a binary classification task using SVM with radial basis kernel in R and I want to assign fuzzy memberships to the datapoints. Already existing function in R packages such as ...
user avatar
  • 1
0 votes
0 answers
8 views

Windowing timeseries classification data

INTRODUCTION: So basically, I have a dataset with 6 columns and around 10k rows. The output column is a label corresponding to every row, with the labels being 0 and 1. The dataset is timeseries based....
user avatar
0 votes
0 answers
16 views

What to do with 99% F1 score in binary classification?

I've been handed a binary classification model to look after. The model uses the F1 score for comparison purposes. The challenge is that the F1 score against the test dataset is very high, like 99%, ...
user avatar
  • 111
1 vote
1 answer
29 views

Does it make sense to apply Bayesian formula on top on a classification problem output?

In classification tasks we normally get a set of numbers that represent a probability distribution - they sum to 1. For further discussion, suppose we only have two classes: ...
user avatar
  • 185
0 votes
0 answers
26 views

Are machine learning algorithms flexible enough to learn changing feature importances?

I have a prediction problem where each row/entity contains data over a range of time and features can change in importance over time even for a single entity. I am wondering if machine learning models ...
user avatar
0 votes
1 answer
24 views

how to deal with data leakage in historical data

I have a dataset containing matches from 2000 TO 2018 and I am asked to predict match outcomes for the year 2017 to avoid data leakage I am going to just train my model from 2000 to 2016. in the ...
user avatar
2 votes
1 answer
25 views

Why is cross entropy loss better than MSE for multi-class classification? [duplicate]

I know there's a lot of material on this, but I'm still struggling to find a scenario where cross-entropy loss is better than MSE loss for a multi-class classification problem. For example, if we have ...
user avatar
  • 41
1 vote
0 answers
22 views

Biased logistic regression in pytorch

My model has decently high AUC=90%, but is biased, underestimating the probability $y=1$. This is systematic across some of the input features as well. How can I nudge the bias term, or otherwise ...
user avatar
3 votes
1 answer
26 views

Features are Relevant for Regression but not necessarily for Classification - what to make of this?

I have used the R Boruta package to check for feature relevance in predicting log returns of financial time series, the targets being the log returns themselves (for regression) and the sign of log ...
user avatar
0 votes
0 answers
5 views

Meaning of the sample variance computed from a k-fold CV

Let's consider a k-fold cross-validation to estimate the generalization error of a model. I would like to clarify the relationship between the following quantities: the variance of the CV estimator ...
user avatar
  • 869
0 votes
0 answers
37 views

Bagging with SVM and Neural Networks in R with caret

I am fairly new to the bagging technique and Caret's bagControl() as well as bag() and am currently trying to build an ensemble ...
user avatar
  • 1
3 votes
1 answer
30 views

Gaussian Process for Classification

I was reading about gaussian process for classification and there is something that is unclear. My understanding is the inverse probit is used instead of the sigmoid and this is because it allows for ...
user avatar
2 votes
1 answer
22 views

Will a dataset with multiple labels perform better than with binary labels?

Suppose I have a dataset comprised of garbages. Will a model perform better if I only label the dataset with biodegradable or non-biodegradable? Or will it be better if I label them with plastics, ...
user avatar
  • 287
1 vote
0 answers
17 views

Metrics for imbalanced multi-class classification [duplicate]

I am looking for informations about metrics for classification with 3 unbalanced classes. I have following numbers of samples in every class: 1 As you can see two classes are quite balanced and one is ...
user avatar
  • 31
0 votes
0 answers
14 views

AUC first drops before starting to grow

I have a very imbalanced classification problem, 99% vs 1%, and I am using logistic regression in pytorch. I often see the initial weights, randomly initialized, achieve AUC of 60-70% before any ...
user avatar
1 vote
1 answer
30 views

How to tell if my features improve model performance?

Setup Task: binary classification Models: logistic regression, SVM, ELM, neural networks - anything that can do classification Dataset: 10 basic features + 6 my own features Question How do I see ...
user avatar
  • 148
0 votes
1 answer
32 views

How to get the True Negative Rate from this code?

I want to calculate the TNR. I am in a larger code project and we have this one binary classifier. The major problem is that I don't find the information about the variables in the code. What is the <...
user avatar
  • 189
0 votes
1 answer
30 views

Expected Prediction Error for 0-1 Loss Function

In ESL on pages 20 and 21, we have a derivation of expected prediction error of a classification rule $\hat{G}(X)$: $$ EPE(\hat{G}) = E_X\sum_{k=1}^{K}L[\mathcal{G}_k, \hat{G}(X)]P(\mathcal{G}_k|X) $$ ...
user avatar
0 votes
0 answers
15 views

What kernel to use for image classification from pre-trained CNN feature extractor

Suppose I have a pre-trained CNN feature extractor and I connect those to a soft margin SVM, what is the recommended kernel to use to replace $x_n^Tx$ in SVM? My dataset comprises of pictures of ...
user avatar
  • 287
1 vote
0 answers
24 views

What are some algorithms that are immune to class imbalance, and what makes them so?

This question is closely similar, however, the answer only speaks about Logistic Regression being one example. I am interested in knowing if there are more algorithms that are not affected (at least ...
user avatar
0 votes
0 answers
10 views

EDA with Data Modeling

After I read R for DataScience and ggplot2: elegant graphics for data analysis, I am learning how use modeling techniques to improve my EDA. I applied this on two notebooks (https://www.kaggle.com/...
user avatar
1 vote
0 answers
18 views

Meaningfully compare target vs observed TPR & FPR

Suppose I have a binary classifier $f$ which acts on an input $x$. Given a threshold $t$, the predicted binary output is defined as: $$ \widehat{y} = \begin{cases} 1, & f(x) \geq t \\ 0, &...
user avatar
  • 224
0 votes
0 answers
32 views

Why is Binary data better than discrete numerical data for Perceptron?

I've heard that when using classifier-type machine learning algorithms (in my case perceptron) it's better to have all fields be binary than to have a mixture of binary and numerical fields. If ...
user avatar
  • 1
0 votes
1 answer
61 views

Low classification accuracy

I want to do a multi class classification with 6 classes. Whole dataset has 12750 and 56 features samples, so every class has 2125 samples. Before prediction I reduces amount of outliers by ...
user avatar
  • 31
0 votes
0 answers
14 views

Hypothesis test for classification model

I have a model that outputs 0 or 1 for interest/not-interest in a job. I'm doing an A/B/C test comparing two models (treatment groups) and none (control group). ANOVA for hypothesis testing and t-test ...
user avatar

1
2 3 4 5
128