Questions tagged [supervised-learning]
Supervised learning is the machine learning task of inferring a function from labeled training data. The training data consist of a set of training examples. In supervised learning, each example is a pair consisting of an input object (typically a vector) and a desired output value (also called the supervisory signal). A supervised learning algorithm analyzes the training data and produces an inferred function, which can be used for mapping new examples.
673
questions
0
votes
0
answers
9
views
Machine Learning College predictor [closed]
I have a training data set containing College names,student rank, branch, college cutoff. Which prediction model should I use to predict the list of colleges a student will get admission in according ...
0
votes
0
answers
19
views
Optimizing Customer Request Classification: Handling Multilabel Notes and Varied Description Lengths
I'm working on a project to classify customer requests based on a dataset that includes request descriptions, labels and sentiments. The goal is to predict the label (and sentiments) of customer ...
- 1
1
vote
0
answers
65
views
Bayesian Optimization: number of iterations as function of search space dimensionality?
I am performing Bayesian Optimization to select a hyperparameter configuration for my supervised learning model. I understand that with each additional hyperparameter that I choose to optimize, the ...
- 1,006
0
votes
0
answers
43
views
Does the order of iteration affect the answer returned by FIND-S?
This paragraph is from the book Machine Learning by Tom M.Mitchell (Page 26):
Initialize $h$ to the most specific hypothesis in $H$
For each positive training instance $x$
$\;\;\;\;\;\;$.For each ...
- 101
0
votes
0
answers
15
views
Supervised classifier for nested interval data and ordinal classes
I'm having trouble formalizing the following classification problem:
Let $x_i$ denote univariate (scalar), continuous, real data points
Let $y_i \in \mathbb{N}$ be their corresponding labels
Classes ...
1
vote
0
answers
53
views
Training Regression Models to Predict Continuous Probability Values in [0, 1] [closed]
I'm working on a machine learning project where my target variable represents continuous probability values that must fall within $[0, 1]$. While I understand regression models are suitable for this ...
-1
votes
2
answers
77
views
Example where the initial random state of a logistic regression matters?
I am looking into how random initialisation of a model would impact final results after tuning. This is a well known problem for deep learning (NN or gbdt), notably with random initialisation and ...
- 1,544
0
votes
0
answers
19
views
Removing seasonality and trend for forecasting with tree based models
I am working on a problem where I'm using tree-based models (RFs, GBTs) for forecasting. I've read that I have to de-trend the data if I'm using a tree-based model, however, I am reading conflicting ...
- 11
0
votes
0
answers
28
views
Violation of i.i.d assumption of supervised learning models with time series data
I am trying to develop a model that predicts the number of cable and joint faults in a distribution grid on a daily basis. These faults seem to increase during very hot and arid summer days (Heatwaves)...
- 33
0
votes
0
answers
73
views
Why do we need to subtract by the mean of each predictor variable in LSE's inference?
In the inference when we want to model a linear regression using the least squares estimating method, we need to subtract each predictor variable by its mean in the formulas of the estimators we have. ...
- 1
1
vote
0
answers
14
views
A ML classifier for predicting the hourly direction for a group of stocks where training stocks don't match out of sample stocks [closed]
I have historical data for 100 stocks (call them A). I would like to train a model jointly on all stocks which will learn from the cross sectional historical activity, to predict another set of stocks ...
- 121
0
votes
0
answers
30
views
Variance Inflation Factor for 2 categorical variables is coming to be greater than 5, can I drop one of them?
I am practicing Linear Regression on the Airbnb dataset. The VIF for 2 dummy variables 'room_type_Entire home/apt' and 'room_type_Private room' is coming as '9.546159' and '9.116464'. These dummy ...
- 1
0
votes
0
answers
65
views
How are the generalization error and the test set error related? Are they?
Let us say that we have a set of input data $x \in X$ with labels $y \in Y$. Given a suitable loss function $R(f(x), y)$, we can define the generalization error of a learnt function $f_{n}$, call it $...
- 9
2
votes
4
answers
434
views
Is the concept "statistical model" irrelevant in (supervised) machine learning?
In supervised machine learning, we are given a set of data
$\{x_i\}_{i=1}^N$ where each data is associated with a label $\{y_i\}_{i = 1}^N$.
We would like to create/train functions $f$ to do several ...
1
vote
0
answers
25
views
Splitting strategy for performing hyperparameter tuning, algorithm comparison and model validation in one experiment
Let's say that for a supervised machine learning experiment I am using a fixed learning algorithm (e.g. Random Forest), and I want to achieve the following:
Choose optimal hyper-parameters for the ...
- 11
0
votes
0
answers
17
views
How to classify each datapoint in a timeseries using deep learning
I am new to time series but for my project I wonder if there is something along those lines:
My aim is to classify each datapoint of a time-dependent time series. In practical terms, I have arrays of ...
- 1
1
vote
1
answer
83
views
Example of Failure of Hoeffding's Inequality for Empirical Risk Minimization
I am studying Introduction to Statistical Learning Theory by Bousquet, Boucheron and Lugosi. On pages 183 through 185 it considers the applicability of Hoeffding's Inequality to Empirical Risk ...
- 123
0
votes
1
answer
59
views
Too good results with linear regression on a non-linear dataset due to training on seen data?
I plotted some time series data that looks non-linear as can be seen below.
![Text] [Looks pretty non-linear, but I decided to implement a linear regression model for learnings sake.
...
- 1
2
votes
1
answer
102
views
Loss function for estimating the conditional variance by fitting $y_i^2$
I'm trying to detect anomolies in a dataset $i \in \{1,2,...,N\}$ where a random variable $y_i$ is expected to be drawn from a normal distribution with mean $\mu_i=0$ and variance $\sigma_i^2 (X_i)$ ...
- 187
1
vote
0
answers
29
views
Supervised Self-Organizing Maps: Overfitting or something else?
I have built several classification/identification models to identify cat behaviour using accelerometer data. I am comparing two modelling techniques, Random Forest (RF) & Supervised Self-...
8
votes
1
answer
539
views
XGBoost: universal approximator?
There are various "universal approximation theorems" for neural networks, perhaps the most famous of which is the 1989 variant by George Cybenko. Setting aside technical conditions, the ...
- 58k
0
votes
0
answers
35
views
The concept of overfitting and underfitting
From my understanding, the model has high bias but low variance. This indicates underfitting. I assume that the model can flawlessly match the training data since the error rate of misclassification ...
- 41
1
vote
1
answer
72
views
High Cross Validation but low test accuracy on LibSVM
I am solving the problem of detecting swallowing and non-swallowing events from the audio. I labelled the data using Praat software by marking the swallowing and nonswallowing events. I trained the ...
- 23
1
vote
0
answers
15
views
Is this a learning to rank problem? [duplicate]
I'm stuck on a problem I'm trying to "translate" into ML terms so as to dig deeper into the literature.
Setup
I have n samples for which I generate k $\in (0, 40]$ different low-dimensional ...
-2
votes
1
answer
60
views
What does the error in artificial neural network stand for, is the same with mean square error (MSE) [closed]
How do I calculate mean square error (MSE) from the error obtained from ANN output
- 1
3
votes
1
answer
281
views
ROC AUC has $0.5$ as random performance. Does PR AUC have a similar notion?
In considering ROC AUC, there is a sense in which $0.5$ is the performance of a random model. Conveniently, this is true, no matter the data or the prior probability of class membership; the ROC AUC ...
- 58k
1
vote
1
answer
66
views
Predicting patients' probabilities from different classification models
Suppose I am a doctor trying to predict the probability of a patient getting a heart attack. The way I would approach this, in a clinical setting, is by using a logistic regression. For exmaple: P(HA) ...
- 21
1
vote
0
answers
25
views
Learning based on correlations?
I am currently interested in a specific supervised learning sub-problem. We have access to Data X and targets y as in traditional statistical / supervised learning. However both X and y are very noisy....
- 1,544
11
votes
8
answers
7k
views
My machine learning model has precision of 30%. Can this model be useful?
I've encountered an interesting discussion at work on interpretation of precision (confusion matrix) within a machine learning model. The interpretation of precision is where there is a difference of ...
- 121
1
vote
1
answer
118
views
Neural Networks - Can I Use Any Activation for the Output Layer?
I'm new to neural networks, and in almost everything I'm reading, the activation function recommended on the output layer follows a specific pattern:
If the network does binary classification (1 ...
- 111
0
votes
0
answers
35
views
Combine TF-IDF with Supervised Learning for Semantic Similarity
I use TF-IDF to compute text similarity scores. It correctly identifies words, that are unique to a document in comparison to the whole corpus. In my case project names and codes are a strong ...
- 143
0
votes
0
answers
16
views
In supervised learning, how to deal with noisy labels if I care little about recall and a lot about precision
I am trying to solve a binary classification problem with supervised learning. Tabular dataset. I have many labels, however I know they are very noisy. So noisy that it is not realistic to get a good ...
- 346
2
votes
2
answers
116
views
Given enough time, can a fully connected layer approximate a causal convolution layer?
In the paper WaveNet: A Generative Model for Raw Audio, the authors try to capture spatial data as follows:
They do this by limiting the scope of the hidden layers to particular sections of the input,...
- 630
3
votes
1
answer
51
views
KNN does not take into account the actual magnitude of the distance
Suppose I have 3 two dimensional points $(0, 0), (10000000, 0), (-1, 0)$ and $(0, 0)$ has label 1.0. If we were to use 1-NN to predict the label for $(10000000, 0), (-1, 0)$, then the answer for both ...
- 155
1
vote
0
answers
27
views
Distribute feature importance to the components of the features in a PCA regression?
I read some interesting speculation over on the Data Science Stack. The setup is that there are multiple correlated features in a regression problem, and the goal is to determine feature importance.
...
- 58k
10
votes
4
answers
869
views
Why do language models like InstructGPT and LLM utilize reinforcement learning instead of supervised learning to learn based on user-ranked examples?
Why do language models like InstructGPT and LLM utilize reinforcement learning instead of supervised learning to learn based on user-ranked examples?
Language models like InstructGPT and ChatGPT are ...
- 1,200
1
vote
1
answer
121
views
If a class overlay is too high for one to another, is it possible to do classification?
This is a figure of PCA Map of several class data that i am trying to classify:
By looking at it, I wanted to be sure if this is not really applicable to be classified? (since it has a high overlap ...
- 159
1
vote
1
answer
165
views
Evaluating Feature Importance for a Super Learner Ensemble Meta-Model
I have been reading up on super learner ensemble methods that utilize multiple models and model configurations to make model predictions as good or better than any individual base model previously ...
0
votes
1
answer
53
views
Can supervised classification models be used as hypothesis testing to compare alternative groping of labels?
Since I didn't find any resource online, I'm asking here.
In this paper of Pereira et al.,(2007; https://doi.org/10.1600/036364407780360201), they use cross validated Canonical Discriminant analysis ...
1
vote
1
answer
53
views
Is my regression network overfitting or is my simulated MRI training data unrepresentative of real MRI val/test data?
I have been working on a regression task for the past one year and I am stuck. My project is to simulate voxel-wise MRI data using a physics-based function and train a neural network using that data ...
0
votes
0
answers
21
views
Are web-scale deep learning models considered to be supervised or unsupervised?
Consider recent large-scale deep learning models, such as transformers for NLP applications, or diffusion models for text-to-image generation. These models are trained on huge, readily-available ...
2
votes
0
answers
42
views
Example datasets where PCA could improve or decrease performance of SVM? [duplicate]
Going through the top answers in How can top principal components retain the predictive power on a dependent variable (or even lead to better predictions)?, I understand that doing PCA and keeping the ...
- 1,109
4
votes
0
answers
206
views
Choice of model surrogate for bayesian optimization
I am running BayesSearchCV to optimize the hyperparameters of my machine learning model. This particular procedure allows the user to choose the surrogate model. The options are Gaussian Process, ...
- 1,006
2
votes
1
answer
82
views
Decreasing importance of trees in XGBoost
On the XGBoost site, it is stated: "XGBoost mostly combines a huge number of regression trees with a small learning rate. In this situation, trees added early are significant and trees added late ...
- 1,006
1
vote
0
answers
10
views
Metric for target value homogeneity across feature distributions?
I want to find a metric that can quantify target value homogeneity across feature distributions.
Without any background knowledge, it is hard to describe exactly what I want.
Therefore, I provide an ...
- 303
1
vote
1
answer
129
views
area under precision recall curve = 0 while AUCROC=1
I have the following data:
predictions = [8;8;8;5;4;3;2;1];
true_target = [1;1;1;0;0;0;0;0];
When I compute the area under the precision recall curve using Matlab (...
- 211
3
votes
2
answers
92
views
Does $R^2$ increase with the number of variables no matter what model we are adjusting?
I understand this is true for OLS, but i am not sure if this is true for every other model, like ridge or a NN.
1
vote
1
answer
45
views
Better understanding classification with unbalanced test data from a mathematical perspective
Suppose I want to get a model, such as a neural network, to correctly classify pictures of cats and dogs and I know that the test set contains around $1\%$ of cats and $99\%$ of dogs.
My intuition is ...
- 177
2
votes
1
answer
92
views
Should I mimic real life distribution in my training set for good learning?
I have a database of data that is labelled good. I can make new data myself and I'll label it bad (I know how to transform a <...
1
vote
0
answers
42
views
Transfer Learning "from scratch"
I've recently started to work in machine learning and this is my first post here. Excuse me in advance for duplicates and/or slang mistakes. My question is about transfer learning (although in this ...
- 13