Skip to main content

Questions tagged [supervised-learning]

Supervised learning is the machine learning task of inferring a function from labeled training data. The training data consist of a set of training examples. In supervised learning, each example is a pair consisting of an input object (typically a vector) and a desired output value (also called the supervisory signal). A supervised learning algorithm analyzes the training data and produces an inferred function, which can be used for mapping new examples.

Filter by
Sorted by
Tagged with
1 vote
0 answers
33 views

Is it possible to compare the output probabilities of two machine learning models? [closed]

Let's suppose I have two classification machine learning models: $\text{Model}_1$ and $\text{Model}_2$. Each of them are not necessarily the same algorithm, and have not been trained necessarily with ...
Poisson Parade's user avatar
12 votes
2 answers
420 views

(THEORY) Do Tree models output probabilities?

I have a question purely theoretical about decision trees outputs for classification. I have heard a lot of people say "the output of tree models are not probabilities", and having studied ...
Felipe Araya Olea's user avatar
3 votes
0 answers
51 views

How to Automatically Identify Column Headers in New Datasets Using Machine Learning?

I have a dataset with vehicle data that includes headers (example): I receive new datasets with similar vehicle data but no column headers: I need to create an algorithm to recognise and assign ...
user782750's user avatar
3 votes
1 answer
38 views

Taking into account a non-symmetric loss function in a classification problem

Consider a binary classification method that estimates the class probability and where the observation weights can be specified (e.g. Logistic Regression). To accommodate the difference losses from TP ...
James's user avatar
  • 2,742
0 votes
0 answers
32 views

Evaluation of Contrastive learning model

I understand in contrastive learning, we have triplet of anchor, positive and negative samples which are defined in advance for supervised contrastive learning, my question is, if I want to evaluate, ...
aelbasir's user avatar
0 votes
0 answers
20 views

Which algorithm for meta model in SuperLearner?

Which algorithm do you use as the meta-model for a SuperLearner? I always see regression commonly, but I'm not getting good predictions with it. I tried all the other algorithms in my SuperLearner and ...
wisamb's user avatar
  • 161
4 votes
2 answers
64 views

How to approach dataset splitting for building time-series models?

Suppose I have 100 observations of time series data $x_1,...,x_{100}$, and that I want to split the data into a train set, a validation set, and a test set. I know that the train set must have smaller ...
David's user avatar
  • 1,276
2 votes
1 answer
53 views

On using the loss as a metric?

The context is model evaluation in supervised learning. I am coming from a numerical optimisation background. For me it is quite natural to use the loss of the model (what we optimise during training) ...
Lucas Morin's user avatar
  • 1,665
3 votes
0 answers
52 views

Isn't $f(\mathbf{x}; \mathbf{\theta}) = b + \mathbf{w}^T \mathbf{x} = b + w_1 x_1 + w_2 x_2 + \dots + w_D x_D$ the linear regression model?

Chapter 1.2.1.5 Uncertainty of Probabilistic Machine Learning: An Introduction by Kevin P. Murphy says the following: We can capture our uncertainty using the following conditional probability ...
The Pointer's user avatar
  • 2,096
0 votes
0 answers
53 views

Simple RNN for predicting the next character [duplicate]

I implemented a simple RNN from scratch (using only the numpy library )for predicting the next characters, and I trained it on a simple text=“hello world”. It works ...
user avatar
1 vote
0 answers
27 views

supervised learning aiming precision on a certain interval of y

I want to build a model that $X$ explains $y$. More specifically, I am mainly interested in a certain interval of $y \in (a, b)$, such that whenever the model predicts $\hat{y} \in (a,b)$, I want the ...
leeway00's user avatar
4 votes
1 answer
71 views

Overfitting GBM by simultaneously adding trees and lowering learning rate?

I understand that you can overfit a Gradient Boosting Machine (GBM) by using too many trees (unlike random forest), and also that you can overfit a GBM by using too high of a learning rate. My ...
David's user avatar
  • 1,276
2 votes
1 answer
47 views

When trying to predict if an event will happen in the next $n$ time-steps, is it a bad idea to label backwards in time?

Apologies if the title makes no sense. At work, I came across something that I don’t think is a good idea and I was hoping someone could help me convince my colleagues of that - or convince me that I’...
Maya's user avatar
  • 31
11 votes
7 answers
3k views

Why do we use Linear Models when tree based models often work better than linear models?

In Supervised Machine Learning, and specifically on Kaggle, it is usually seen that tree models often outperform linear models. And even in the tree-based models, it is usually XGBoost that ...
letdatado's user avatar
  • 367
0 votes
0 answers
38 views

Data leakage in time series forecasting framed as a supervised learning problem

Suppose that I have a simple univariate time series. My goal is to use the value of 3 consecutive days to predict the value of the fourth day. I built my dataset by applying a rolling window that ...
Ray's user avatar
  • 11
0 votes
0 answers
24 views

Optimizing Customer Request Classification: Handling Multilabel Notes and Varied Description Lengths

I'm working on a project to classify customer requests based on a dataset that includes request descriptions, labels and sentiments. The goal is to predict the label (and sentiments) of customer ...
deps's user avatar
  • 1
4 votes
0 answers
211 views

Bayesian Optimization: number of iterations as function of search space dimensionality?

I am performing Bayesian Optimization to select a hyperparameter configuration for my supervised learning model. I understand that with each additional hyperparameter that I choose to optimize, the ...
David's user avatar
  • 1,276
0 votes
0 answers
43 views

Does the order of iteration affect the answer returned by FIND-S?

This paragraph is from the book Machine Learning by Tom M.Mitchell (Page 26): Initialize $h$ to the most specific hypothesis in $H$ For each positive training instance $x$ $\;\;\;\;\;\;$.For each ...
Emad's user avatar
  • 101
0 votes
0 answers
16 views

Supervised classifier for nested interval data and ordinal classes

I'm having trouble formalizing the following classification problem: Let $x_i$ denote univariate (scalar), continuous, real data points Let $y_i \in \mathbb{N}$ be their corresponding labels Classes ...
themodelguy's user avatar
1 vote
0 answers
83 views

Training Regression Models to Predict Continuous Probability Values in [0, 1] [closed]

I'm working on a machine learning project where my target variable represents continuous probability values that must fall within $[0, 1]$. While I understand regression models are suitable for this ...
Anastasiya-Romanova 秀's user avatar
-1 votes
2 answers
130 views

Example where the initial random state of a logistic regression matters?

I am looking into how random initialisation of a model would impact final results after tuning. This is a well known problem for deep learning (NN or gbdt), notably with random initialisation and ...
Lucas Morin's user avatar
  • 1,665
0 votes
0 answers
22 views

Removing seasonality and trend for forecasting with tree based models

I am working on a problem where I'm using tree-based models (RFs, GBTs) for forecasting. I've read that I have to de-trend the data if I'm using a tree-based model, however, I am reading conflicting ...
harrynak's user avatar
0 votes
0 answers
44 views

Violation of i.i.d assumption of supervised learning models with time series data

I am trying to develop a model that predicts the number of cable and joint faults in a distribution grid on a daily basis. These faults seem to increase during very hot and arid summer days (Heatwaves)...
Sam Malek's user avatar
0 votes
0 answers
171 views

Why do we need to subtract by the mean of each predictor variable in LSE's inference?

In the inference when we want to model a linear regression using the least squares estimating method, we need to subtract each predictor variable by its mean in the formulas of the estimators we have. ...
Blue999's user avatar
1 vote
0 answers
15 views

A ML classifier for predicting the hourly direction for a group of stocks where training stocks don't match out of sample stocks [closed]

I have historical data for 100 stocks (call them A). I would like to train a model jointly on all stocks which will learn from the cross sectional historical activity, to predict another set of stocks ...
GlaceCelery's user avatar
0 votes
0 answers
46 views

Variance Inflation Factor for 2 categorical variables is coming to be greater than 5, can I drop one of them?

I am practicing Linear Regression on the Airbnb dataset. The VIF for 2 dummy variables 'room_type_Entire home/apt' and 'room_type_Private room' is coming as '9.546159' and '9.116464'. These dummy ...
Shri's user avatar
  • 23
4 votes
4 answers
537 views

Is the concept "statistical model" irrelevant in (supervised) machine learning?

In supervised machine learning, we are given a set of data $\{x_i\}_{i=1}^N$ where each data is associated with a label $\{y_i\}_{i = 1}^N$. We would like to create/train functions $f$ to do several ...
Curaçao Hajek's user avatar
1 vote
0 answers
39 views

Splitting strategy for performing hyperparameter tuning, algorithm comparison and model validation in one experiment

Let's say that for a supervised machine learning experiment I am using a fixed learning algorithm (e.g. Random Forest), and I want to achieve the following: Choose optimal hyper-parameters for the ...
saveturn's user avatar
1 vote
1 answer
133 views

Example of Failure of Hoeffding's Inequality for Empirical Risk Minimization

I am studying Introduction to Statistical Learning Theory by Bousquet, Boucheron and Lugosi. On pages 183 through 185 it considers the applicability of Hoeffding's Inequality to Empirical Risk ...
Extrava's user avatar
  • 123
0 votes
1 answer
77 views

Too good results with linear regression on a non-linear dataset due to training on seen data?

I plotted some time series data that looks non-linear as can be seen below. ![Text] [Looks pretty non-linear, but I decided to implement a linear regression model for learnings sake. ...
Urxtixt's user avatar
2 votes
1 answer
131 views

Loss function for estimating the conditional variance by fitting $y_i^2$

I'm trying to detect anomolies in a dataset $i \in \{1,2,...,N\}$ where a random variable $y_i$ is expected to be drawn from a normal distribution with mean $\mu_i=0$ and variance $\sigma_i^2 (X_i)$ ...
JoseOrtiz3's user avatar
1 vote
0 answers
45 views

Supervised Self-Organizing Maps: Overfitting or something else?

I have built several classification/identification models to identify cat behaviour using accelerometer data. I am comparing two modelling techniques, Random Forest (RF) & Supervised Self-...
Michelle Smit's user avatar
9 votes
1 answer
703 views

XGBoost: universal approximator?

There are various "universal approximation theorems" for neural networks, perhaps the most famous of which is the 1989 variant by George Cybenko. Setting aside technical conditions, the ...
Dave's user avatar
  • 65k
0 votes
0 answers
37 views

The concept of overfitting and underfitting

From my understanding, the model has high bias but low variance. This indicates underfitting. I assume that the model can flawlessly match the training data since the error rate of misclassification ...
Amiira's user avatar
  • 51
1 vote
1 answer
95 views

High Cross Validation but low test accuracy on LibSVM

I am solving the problem of detecting swallowing and non-swallowing events from the audio. I labelled the data using Praat software by marking the swallowing and nonswallowing events. I trained the ...
Yalçın Cenik's user avatar
1 vote
0 answers
18 views

Is this a learning to rank problem? [duplicate]

I'm stuck on a problem I'm trying to "translate" into ML terms so as to dig deeper into the literature. Setup I have n samples for which I generate k $\in (0, 40]$ different low-dimensional ...
this.isnt.nathan's user avatar
-2 votes
1 answer
131 views

What does the error in artificial neural network stand for, is the same with mean square error (MSE) [closed]

How do I calculate mean square error (MSE) from the error obtained from ANN output
Chris's user avatar
  • 1
3 votes
1 answer
565 views

ROC AUC has $0.5$ as random performance. Does PR AUC have a similar notion?

In considering ROC AUC, there is a sense in which $0.5$ is the performance of a random model. Conveniently, this is true, no matter the data or the prior probability of class membership; the ROC AUC ...
Dave's user avatar
  • 65k
1 vote
1 answer
90 views

Predicting patients' probabilities from different classification models

Suppose I am a doctor trying to predict the probability of a patient getting a heart attack. The way I would approach this, in a clinical setting, is by using a logistic regression. For exmaple: P(HA) ...
gknowme's user avatar
  • 21
1 vote
0 answers
27 views

Learning based on correlations?

I am currently interested in a specific supervised learning sub-problem. We have access to Data X and targets y as in traditional statistical / supervised learning. However both X and y are very noisy....
Lucas Morin's user avatar
  • 1,665
11 votes
8 answers
8k views

My machine learning model has precision of 30%. Can this model be useful?

I've encountered an interesting discussion at work on interpretation of precision (confusion matrix) within a machine learning model. The interpretation of precision is where there is a difference of ...
wmmwmm's user avatar
  • 121
1 vote
1 answer
217 views

Neural Networks - Can I Use Any Activation for the Output Layer?

I'm new to neural networks, and in almost everything I'm reading, the activation function recommended on the output layer follows a specific pattern: If the network does binary classification (1 ...
Krusty the Clown's user avatar
3 votes
2 answers
171 views

Given enough time, can a fully connected layer approximate a causal convolution layer?

In the paper WaveNet: A Generative Model for Raw Audio, the authors try to capture spatial data as follows: They do this by limiting the scope of the hidden layers to particular sections of the input,...
AlanSTACK's user avatar
  • 640
4 votes
1 answer
56 views

KNN does not take into account the actual magnitude of the distance

Suppose I have 3 two dimensional points $(0, 0), (10000000, 0), (-1, 0)$ and $(0, 0)$ has label 1.0. If we were to use 1-NN to predict the label for $(10000000, 0), (-1, 0)$, then the answer for both ...
koch's user avatar
  • 195
1 vote
0 answers
29 views

Distribute feature importance to the components of the features in a PCA regression?

I read some interesting speculation over on the Data Science Stack. The setup is that there are multiple correlated features in a regression problem, and the goal is to determine feature importance. ...
Dave's user avatar
  • 65k
14 votes
4 answers
1k views

Why do language models like InstructGPT and LLM utilize reinforcement learning instead of supervised learning to learn based on user-ranked examples?

Why do language models like InstructGPT and LLM utilize reinforcement learning instead of supervised learning to learn based on user-ranked examples? Language models like InstructGPT and ChatGPT are ...
resnet's user avatar
  • 1,270
1 vote
1 answer
164 views

If a class overlay is too high for one to another, is it possible to do classification?

This is a figure of PCA Map of several class data that i am trying to classify: By looking at it, I wanted to be sure if this is not really applicable to be classified? (since it has a high overlap ...
Jovan's user avatar
  • 159
1 vote
1 answer
283 views

Evaluating Feature Importance for a Super Learner Ensemble Meta-Model

I have been reading up on super learner ensemble methods that utilize multiple models and model configurations to make model predictions as good or better than any individual base model previously ...
Jake Niederer's user avatar
0 votes
1 answer
57 views

Can supervised classification models be used as hypothesis testing to compare alternative groping of labels?

Since I didn't find any resource online, I'm asking here. In this paper of Pereira et al.,(2007; https://doi.org/10.1600/036364407780360201), they use cross validated Canonical Discriminant analysis ...
Adrianorex's user avatar
1 vote
1 answer
62 views

Is my regression network overfitting or is my simulated MRI training data unrepresentative of real MRI val/test data?

I have been working on a regression task for the past one year and I am stuck. My project is to simulate voxel-wise MRI data using a physics-based function and train a neural network using that data ...
Krithika Balaji's user avatar

1
2 3 4 5
14