Questions tagged [classification]
Statistical classification is the problem of identifying the sub-population to which new observations belong, where the identity of the sub-population is unknown, on the basis of a training set of data containing observations whose sub-population is known. Therefore these classifications will show a variable behavior which can be studied by statistics.
6,814
questions
0
votes
0
answers
6
views
Search recall optimization - what appropriate loss function to use?
I am studying machine learning and wanted to work on a project of my own so that I have better chances after graduating college. I'm studying the application of ML to improve searches using a toy ...
- 161
0
votes
0
answers
10
views
Linear Discriminant Analysis with unlabeled data
In section 4.4.5 "Logistic regression or LDA?" of Elements of Statistical Learning by Friedman, Tibshirani and Hastie, it is claimed the following:
From the mixture formulation [that is, ...
- 316
0
votes
0
answers
32
views
Is it normal to have a sharp increase in validation error when using 10% of the data instead of something like 20-30%?
Scenario:
I'm training a relatively simple neural network to classify pairs of tabular datapoints (~150k), lets say drugs and diseases, whether they are related (positive) or not (negative). As I only ...
- 75
0
votes
0
answers
13
views
Training problem in CNN model for image classification
I am training Cifar-10 dataset of 32 x 32 sized coloured images for image classification.
Here is the link/source to the dataset and its description: https://www.cs.toronto.edu/~kriz/cifar.html
The ...
2
votes
2
answers
29
views
Does a relationship between missing values indicate MNAR?
I have a dataset where several features are missing values only if another feature is also missing a value. Does this indicate that the missing data is missing not at random (MNAR)? Additionally, how ...
- 55
0
votes
0
answers
10
views
If I double all feature values in sample with class 1, will class 1 have a weight of 2?
In a classification problem, I want to know how to manually manipulate the data to weight the classes without using the features provided by the machine learning library. For example, if I double all ...
- 141
0
votes
0
answers
7
views
Binary Classification - Separating data by categorical data and creating regression for each group
My and friend and I are complete beginners to statistical learning and we were playing around with a data set on loan approval using logistic regression. Each data point contains numeric variables ...
2
votes
3
answers
89
views
cutoff and auc and changing cutoff
can you tell me if this is ok?
While the AUC (i.e. AUC of 0.6) we got is acceptable since it's bigger than 0.5, we may need to re-evaluate at our cutoff selections again. Because we can select cutoffs ...
- 21
0
votes
1
answer
62
views
Do uncalibrated "probability" predictions satisfy Kolmogorov's axioms?
Let's say we have some binary variable of interest and fit a model to predict the probability of the two classes, say a logistic regression or a "classification" neural network. This model ...
- 55.1k
0
votes
1
answer
26
views
Abusing statistics- assessing the power of classification connection by flipping X and Y in a linear model
I am supposed to check the significance of the connection between an arbitary Y variable (nominal\ordinal\binary I cannot know in advance) and an arbitary numeric X variable.
This is a classification ...
- 101
0
votes
0
answers
22
views
Turning heatmap into clusters - Classification
Assume that you having a heatmap that looks like this. The goal is to classify all the "dot" inside the image. How can that be done?
The assumptions of the image:
The image has always black ...
- 349
0
votes
0
answers
13
views
Incremental Users On Platform Method
I have the following data (aggregated, user level data is not available).
date | platform | total_watch_time
platform = [mobile, mobile + desktop, mobile + desktop + tv]
I want to determine the ...
- 11
2
votes
1
answer
40
views
In case of no correlation, can a model make predictions above the expected values?
For simplicity's sake, let's suppose a binary classification problem, with a perfect 50% of probability for each of the classes, and a SkLearn's SVC model.
Let's ...
0
votes
0
answers
15
views
Supervised classifier for nested interval data and ordinal classes
I'm having trouble formalizing the following classification problem:
Let $x_i$ denote univariate (scalar), continuous, real data points
Let $y_i \in \mathbb{N}$ be their corresponding labels
Classes ...
1
vote
1
answer
57
views
Probability in a classification problem
In a classification problem (let's say of two categories, cat and dog) with a softmax output, does the probability have any physical meaning other than assigning the category to the input based on the ...
- 111
1
vote
1
answer
54
views
Distribution of accuracy from randomly guessing
Let's consider a true classification problem, that is, one where the predictor makes categorical predictions (not probabilities).
It makes sense to assess the accuracy of such a predictor. However, ...
- 55.1k
1
vote
0
answers
80
views
Is perfect isotonic probability calibration realistic?
I work with a labelled tabular dataset of about 1 million observations, with the target being binary. The dataset is heavily imbalanced - about 0.5% positive class.
I have trained a gradient boosting ...
- 11
0
votes
1
answer
35
views
Can we build a Naive Bayes model in the presence of multicollinearity?
There exists multi-collinearity in the model. As a result(may be) the accuracy of Naive Bayes model is less than logistic regression model(model is built with the help of PCA). So, is there any method ...
0
votes
0
answers
19
views
Grid search does not choose optimal results when using a different number of features based on RFE
I am using Grid search for an SVM-RFE predicting a binary outcome with 33 features.
...
- 23
2
votes
1
answer
46
views
Can a ROC curve be partly above and partly below the diagonal? [duplicate]
A ROC curve has particular meaning about how the sensitivity and specificity change as the classification threshold changes for two groups of data. The x- and y-axes both range from zero to one, and ...
- 55.1k
1
vote
1
answer
22
views
Predicted class probability in threshold moving
I am training a model for the task of Binary classification using H2O.ai. The final output to the user is the probability of class_1.
Recently, I found that by ...
14
votes
3
answers
241
views
Area under the ROC curve when there is imbalance: is there a problem, and if not, why does this rumor exist?
Cross Validated has a rather thorough debunking of class imbalance being an inherent problem that must be fixed in order to do quality predictive modeling of categorical outcomes [1, 2]. However, ...
- 55.1k
1
vote
1
answer
15
views
What can cause one label in a multi-label classification task to have the highest score most of the times?
I have a multi-label classification task in which I have 7 categories per each sample. I train a multi-label classification model (i.e., XLMRobertaForSequenceClassification). When I evaluate the model ...
- 13
0
votes
0
answers
23
views
Why use Kernel trick if soft margin SVM works for non-linearly separable data? [duplicate]
Most articles and textbooks say that soft margin SVM is used as the data is messy/not linearly separable. We introduce slack variables to make the data linearly separable. Kernels are used when the ...
- 1,245
4
votes
3
answers
115
views
Machine learning classification: best way to know if my variables are unable to distinguish between two classes
I am working with an imbalanced dataset containing 42 variables and around 136,000 observations, in order to perform a binary classification (96% of the observations belong to one class). I tried ...
- 253
0
votes
0
answers
26
views
When is the case that a classifier's accuracy is better when no data augmentation has been applied?
When is the case that a classifier's accuracy is better when no data augmentation has been applied (Compared to the dataset with the augmentations applied)? I'd like to know especially how the data is ...
- 3
1
vote
0
answers
27
views
Understanding a calibration plot for lightGBM binary classifier
I wanted to assess the performance of my lightGBM classifier using a calibration plot. If I understood correctly, a calibration plot visualizes the alignment between the predicted probabilities by the ...
- 459
1
vote
1
answer
19
views
Classification Threshold Optimization after GridSearchCV
In my machine learning problem I am using a CNN to classify images.
Since my dataset is imbalanced I want to perform classification probability threshold tuning so I can find the optimal balance ...
- 11
0
votes
1
answer
50
views
Performing a classification if having categorial labels and a distance matrix
I encountered a multi-class classification problem and I wonder which model would work the best in my scenario. I have around 50,000 vectors (each of size 200) with corresponding categorical labels ...
2
votes
1
answer
32
views
Best way of splitting panel data for machine learning
I am trying to train a machine learning model to predict the probability that a given credit card customer defaults within the six-month window after the observed date. In this context, default means ...
- 23
2
votes
0
answers
20
views
What algorithm/approach is best for a multiclass classifier where there are a significant amount of misclassifications in the source data?
I am using document embeddings (100 dimensions) to train a classifier on text data, however, I am getting poor results which I am attributing to the fact that there are a large proportion of ...
- 31
0
votes
0
answers
21
views
Strength of LSTM for text prediction
I am trying to understand the working of LSTM by considering an application. Given below is the application of LSTM for spam prediction a well known dataset.
...
- 123
1
vote
0
answers
25
views
Why I get spikes during training with vanilla gradient descent? [closed]
I developed my own NN toolbox, and it seems it works fine.
But I am not sure why I get these spikes in my loss during training:
I a training for a classification task of 2 inputs and 2 classes, ...
- 119
2
votes
2
answers
33
views
Should we (under- or over-) sample when training a ML model, if we care about edge cases?
I know this question has been somehow reiterated in multiple ways, but I have not yet found an answer that would explicitly apply to my case. I wish to train a classification model to predict who is ...
0
votes
0
answers
5
views
Which method should be used to determine the class ID of multiple SVM models?
I'm using Support Vector Machine(SVM) with image classification.
Each SVM model results a linear model
$$y = wx + b$$
Where $w$ and $b$ is the SVM parameters.
If I have multiple SVM models, I will get ...
- 349
1
vote
1
answer
45
views
What exactly is the problem with overconfident predictions?
Say I have a neural network that classifies images by training to minimise cross-entropy loss with one-hot encoded training labels. It is often seen that such neural networks are 'overconfident', with ...
- 111
0
votes
0
answers
32
views
How to prove EM algo convergence?
Original Problem
I'm looking at problem-set in cs229 about EM algorithim.
As my understanding $\ell_{\text{semi-sup}}(\theta)$ is
$$
\ell_{\text{semi-sup}}(\theta) = \sum^m_{i=1} \log \sum_{z^{(i)}} ...
- 135
0
votes
1
answer
66
views
Measuring perplexity over a limited domain in an LLM
Are there papers/a literature on measuring perplexity in using a Large Language Model such as ChatGPT/Flan over a limited domain?
I want to prompt an LLM to do movie recommendations/next job ...
- 103
0
votes
0
answers
7
views
Compare LASSO regressions as missing data increases
I'm comparing three LASSO-regression models for classifying two patient types. Each model has increasingly complex variables, which are less likely to be available. Consequently, the last model has ...
- 39
1
vote
1
answer
68
views
Model evaluation metrics for comparing predicted probability accuracy across different datasets?
I'm working on an online model scoring framework, my goal is to be able to understand if my model's predictive performance is degrading week-over-week. I have a classification model (trained on binary ...
- 21
1
vote
0
answers
24
views
Is there actually a right and wrong way to deal with major imbalance in logistic regression (or other models, really)? [duplicate]
I have seen a lot of different advice on how to deal with imbalance, and I get that it can be case-specific. But I learned in school that SMOTE oversampling or undersampling were basically the ways ...
- 11
2
votes
1
answer
23
views
R glmertree: right variables for components? limit patitioning layers? adjust plot? result interpretion?
As a very beginner user, I have the following questions during my testing:
1. What are the suitable variables in the dataset for each of the components on the right side of the below tree formula?
<...
- 33
4
votes
1
answer
115
views
How robust is multinomial logistic regression to having a multi-label problem shoehorned into it?
Consider a situation where there can be membership in group $A$, group $B$, both groups, or neither group. If we wanted to predict group membership probabilities from some covariate information, this ...
- 55.1k
4
votes
0
answers
141
views
Trying to understand the theory behind my similar / better results than XGBoost using a calibrated linear model (GAM)
I just opened a discussion on reddit asking about why/how the calibrated linear models I've been training have been getting similar / better results than XGBoost in my experiments. I was told to cross ...
- 41
6
votes
2
answers
568
views
How do you refer to data that's not part of train/test/validation?
The purpose of creating a machine learning model is to deploy in live situation, e.g., classification in the wild.
The model is trained on the training set, tuned on the validation set, and evaluated ...
- 1,373
0
votes
0
answers
41
views
How should I deal with Nan values if they occupy a great portion of the data set?
I am trying to make a classifier that predicts whether a patient is diabetic or not. I am working on this data set. While doing EDA, I found that in the column of ...
- 101
0
votes
1
answer
31
views
Creating Cumulative Uplift Curve in Time Series CV with Rolling Windows
I'm training a binary classifier using time series cross-validation and a rolling window approach, resulting in $k$ test sets. I want to construct a cumulative uplift curve to evaluate the model.
I've ...
- 105
0
votes
0
answers
25
views
Does linear separability with gamma margin guarantee convergence of perceptron algorithm?
I am studying perceptron for the first time. I came across the assumption from online resources that if the data is linearly separable with gamma margin then the perceptron algorithm will converge. Is ...
- 1
0
votes
1
answer
58
views
Determine 'w' and 'b' in hard margin SVM
I have been asked the following question related to SVM (Hard Margin) in the exam, and I failed to answer it. Can anyone help me find the solution?
Consider the dataset M:
\begin{align*}
& \left(\...
0
votes
0
answers
20
views
LMest for cluster membership over time
I have a dataset of 18 continuous variables measured over 3 time points for 90 patients. I hypothesise that there are clusters of patients with similar characteristics and that cluster membership may ...
- 11