Questions tagged [supervised-learning]

Supervised learning is the machine learning task of inferring a function from labeled training data. The training data consist of a set of training examples. In supervised learning, each example is a pair consisting of an input object (typically a vector) and a desired output value (also called the supervisory signal). A supervised learning algorithm analyzes the training data and produces an inferred function, which can be used for mapping new examples.

Filter by
Sorted by
Tagged with
1 vote
0 answers
17 views

Interpreting learning curves

There is really few examples online regarding interpreting learning curves and they are all of different type.It is quite confusing to me honestly.May I just ask: How should we interpret them?What ...
user avatar
1 vote
0 answers
64 views

Chicken and egg problem in machine learning [closed]

Recently, I went through an ICLR paper SELF-LABELLING VIA SIMULTANEOUS CLUSTERING AND REPRESENTATION LEARNING. In the paper, authors discussed simultaneously labeling the images and training a network ...
user avatar
0 votes
1 answer
18 views

"SMOTE makes the assumption that the instance between a positive class instance and its nearest neighbors is also positive"

I am trying to get my head around this assertion by Liu, Y. et al (2011 pp. 7) about SMOTE oversampling technique that: because SMOTE makes the assumption that the instance between a positive class ...
user avatar
  • 231
0 votes
0 answers
11 views

Can a basis expansion guarantee no worse performance than original features?

Consider the typical learning problem where given inputs $x_i \in \mathbb{R}^p$ and targets $y_i \in \mathbb{R}$ for $i = 1, \dots, n$ we would like to learn some function $f$ such that $L(f(x_i), y_i)...
user avatar
  • 436
0 votes
0 answers
14 views

How do I evaluate if my data represent the target variable before training a machine learning algorithm?

I have a dataset of points cloud where each point in the point cloud has a variable. I am trying to relate the local geometry features to that point variable by using FPFH, This means I am generating ...
user avatar
0 votes
1 answer
33 views

In which category falls a mix of unsupervised and supvervised learning?

Here is the context of my problem: I want to classify between to classes. However, I have at disposal only non labeled data to do the training (the test set possess all labels for evaluation purposes)....
user avatar
  • 9
3 votes
0 answers
52 views

Looking for the Holy Grail of nonparametric regression

Unfortunately, to state precisely the question, I need some formal preliminaries. Let $d \in \mathbb{N}$. For each $d^* \in \{1,\dots,d\}$, define $\mathcal{M}_{d^*}$ be the set of probability ...
user avatar
  • 153
2 votes
1 answer
25 views

Terminology of "Regression forest", "Random forest", "Decision tree" and "Regresion tree"

I am confused about the terminology of "regression forest", "random forest regression", "random forest", "decision tree" and "regression tree". As far ...
user avatar
  • 109
1 vote
1 answer
26 views

I have set of features to relate to two different values. When I made a regressor for only one it worked well but if i use two it does not?

I have a set of 33x1 features (x) and they can be related to different two values in (y) and I have 1203985 observations. Using np.shape() you can see the dimensions of x and y. x= (1203985, 33) y=(...
user avatar
1 vote
1 answer
13 views

Problem formulation classification task

I would like to know if it is correct for a classification task in a supervised learning to say the model we are looking for is a function from RxR to a discrete space $$ f:\mathbb{R}\times\mathbb{R} \...
user avatar
1 vote
1 answer
36 views

What part of a dataset do I apply a traditional, statistical analysis to linear regression?

Note: I've edited my question as recommended below by @EdM. Suppose I have a supervised learning problem on a sizeable tidy dataset with real values—-e.g., the dataset has 100,000 rows or observations....
user avatar
0 votes
1 answer
52 views

Can all neural network layers be used as either a supervised or an unsupervised model?

I am trying to understand neural networks and by reading different articles I always find conflicting information. I wanted to understand which neural networks can be used as supervised/unsupervised. ...
user avatar
  • 116
3 votes
3 answers
124 views

Challenge an ICML Paper: For a given set of probability predictions and a log loss value, is the set of true labels giving such a loss unique?

Aggarwal's 2021 ICML paper "Label Inference Attacks from Log-loss Scores", seems to argue that the answer to the question in the title is "YES". The paper claims that, given ...
user avatar
  • 31k
1 vote
0 answers
25 views

What is the difference between these two types of training?

Suppose that I want to detect if a picture contains a particular logo, for instance the following one. Since template matching would be slow and fail those scaled or resized ones, I decided to train ...
user avatar
  • 5,151
0 votes
0 answers
10 views

Unconventional pretext task in computer vision - can I somehow justify it?

I was working on a industrial object detection neural network project. Since we had multiple images of the same object in different (but fixed) positions and light conditions, our dataset was very ...
user avatar
0 votes
0 answers
14 views

When is the existence and/or unicity of the Empirical Risk Minimizer guaranteed?

In Supervised Machine Learning, it is common to learn a target function by minimizing a (regularized) Empirical Risk Objective, i.e., for a dataset of $n$ samples $(X_i,y_i)$, the learned function $\...
user avatar
4 votes
2 answers
257 views

Forecast Time Series like data

I have time series like data, 500 data points of (x,y) pairs. Where x = time in seconds and y = signals. Each of this candidates have an additional label (which ...
user avatar
0 votes
0 answers
10 views

Embedding extraction -> Classifier VS Embedding learning+ Classification on-the-fly?

I have two questions: How should we compare in general which of the following perform better? I have a graph and would like to perform a graph classification task. Is it better to extract graph ...
user avatar
2 votes
2 answers
88 views

Is machine learning all about hyperparameter tuning?

I understand the view that ML is a big optimization problem where we are trying to minimize the loss function and achieve the most optimal solution given the input. To achieve that we are feeding a ...
user avatar
  • 141
1 vote
1 answer
37 views

Binary classification supervisor definition help

I need help with defining the supervisor for a ML model. Background: I’m predicting if a customer will respond positively to a marketing campaign. The response is a binary variable that I am given at ...
user avatar
0 votes
2 answers
36 views

Can there be such things like supervised learning in bayesian approach?

Whenever I encounter articles on supervised learning examples are things like regression, classification, object detection, which are obviously ones following frequentist approach. I've recently ...
user avatar
  • 111
1 vote
0 answers
18 views

How to interpret the supervised contrastive loss

I am currently trying to wrap my head around the supervised contrastive loss introduced in the following article : https://arxiv.org/pdf/2004.11362.pdf The loss formula in question is : $$Loss = \sum_{...
user avatar
  • 11
0 votes
1 answer
73 views

Which LSTM output should be used for predictions?

Using this question as background: https://stackoverflow.com/questions/71023822/lstm-multi-variate-multi-feature-in-pytorch I was wondering how one processes the output of a pytorch LSTM I was using ...
user avatar
  • 3
1 vote
0 answers
27 views

What is the best approach: Labeled training data and unlabeled test data [closed]

I'm new into the data science world and I am working on improving my knowledge so here is my problem: I want to build a binary classifier with the following constraints: I have 2 files training.csv ...
user avatar
0 votes
0 answers
17 views

Question about proof in DAGGER algorithm paper

I'm new to imitation learning and trying to read a paper of DAGGER algorithm (https://arxiv.org/pdf/1011.0686.pdf). When reading the paper, I got stuck at proof of Theorem 2.2. This is a beginner's ...
user avatar
0 votes
0 answers
6 views

Sample Selection within motion planning data

The target of this study is to attempt to learn behavior of an unknown algorithm from raw data. The environment in use is a 2D motion planning environment. We assume the algorithm behaves similarly to ...
user avatar
  • 101
1 vote
1 answer
27 views

Bootstrap validation with a categorical outcome: should I sample each outcome separately?

I am doing something like what rms::validate does: bootstrap data frame rows in a supervised learning problem, fit a model to each bootstrap sample, apply that ...
user avatar
  • 31k
0 votes
0 answers
19 views

Introduce a new variable to GLM

I have this one interview question regarding GLM model and would love some insights into method/product sense/common sense input. -Consider this car insurance pricing model: y (car price) = B1 * ...
user avatar
1 vote
0 answers
24 views

Can a Supervised Routine be Compared Against an Unsupervised?

Just a question out of curiosity. Suppose that I had generated: (1) an unsupervised decision tree using 'interpretable clustering,' and (2) a second supervised decision tree (whether CART, or a ...
user avatar
  • 174
0 votes
0 answers
72 views

How to calculate the bias b in support vector machine when the dual coefficient alpha is obtained?

For my example, I have two data points x = {(54001.988, 19999), (30021.983, 15000} and their labels are y = {1, -1}. I calculated the dual coefficient(Lagrange multipliers) alpha = {10000, 10000}. The ...
user avatar
1 vote
1 answer
29 views

Limit of Momentum Update Equation

I am self-studying on optimization algorithm on https://d2l.ai/chapter_optimization/momentum.html and couldn't get my head around some derivation: Instead of the standard gradient descent update ...
user avatar
0 votes
0 answers
22 views

How to train a model to maximize the difference of correlations?

I have two labeled datasets, $A$ and $B$: $(X_A, y_A)$ and $(X_B, y_B)$. $X_A, X_A \in \mathbb{R}^{m \times n}$ and $y_A, y_B \in \mathbb{R}^n$. $m$ is the number of features, $n$ is the number of ...
user avatar
  • 21
0 votes
0 answers
50 views

What is a good method for applying grid search on ensemble models?

I built an experiment where i am studying the performance of ensemble models for a classification task. Basically, i'm comparing Random Forest with Adaboost. However, Adaboost is built with a mix of ...
user avatar
  • 133
3 votes
0 answers
55 views

Is there a notion of maximum likelihood estimation for random forest?

If we have a regression function $R(X)$, whether it is linear or nonlinear, if we make a Gaussian assumption about the error term, optimizing square loss is equivalent to maximum likelihood estimation....
user avatar
  • 31k
0 votes
0 answers
14 views

Assumption of Gaussian distribution for features

It is true that a method like linear regression requires the residuals to be normally distributed. However, on some forums you see people suggest that some supervised learning algorithms perform ...
user avatar
0 votes
0 answers
11 views

Why is the multiple instance learning equivalent of a problem giving much less accuracy?

I am currently working with a dataset of brain images. I have all of them labelled (0: healthy, 1: not healthy) so that I can train a fully supervised model on them. Furthermore, I have them grouped ...
user avatar
0 votes
0 answers
16 views

How Should Learning Rate, Warm Up Learning Rate, Weight Decay, and Other Training Parameters Be Scaled With Batch Size?

Currently, due to memory limitations, I am scaling my batch size by a factor of k, which probably means I need to scale other factors of my algorithm too. For learning rate, I have heard that I should ...
user avatar
2 votes
1 answer
43 views

Bayesian machine learning for supervised learning

I began to read about Bayesian machine learning. I have an expertise in using algorithms such as gradient boosting or random forests for supervised learning problems. In order to understand the ...
user avatar
  • 883
1 vote
0 answers
43 views

Constructive learning models that can run (predict) fairly well on a Raspberry pi?

I'm looking for a model that is based on constructive learning and will do it's predictions live (via video camera) on the Raspberry Pi 4 Model B, either trained on a PC or pre-trained, I found out ...
user avatar
  • 111
0 votes
1 answer
74 views

Binary Classification with a third 'uncertain' class label

Consider the task of classifying an image into two classes: Image shows a cat; Image shows no cat. A data set is provided for training/testing a binary classifier. However, three labels are provided ...
user avatar
  • 1
2 votes
2 answers
64 views

What are the pros and cons of using a supervised learning approach for building a recommender system compared to traditional approaches?

Pretty much all articles I read about recommender systems use Collaborative Filtering, Content Based Filtering and Hybrid approaches. Virtually no one mentioned about supervised learning approaches. ...
user avatar
  • 643
0 votes
0 answers
33 views

Estimate assignment probabilities of a multi-class classification problem

I a dataset of $N$ observations $x_1, \ldots, x_N$. We know that to each observation $x_i\in\mathbb{R}^k$ one of $m$ possible class labels has been assigned to it $y_i\in\{1, \ldots, m\}$. For each ...
user avatar
1 vote
1 answer
21 views

Suggestions on how to translate output from a forecasting model as a risk index?

This question may seem a bit odd but here we go. I have a supervised-learning pipeline that I am using to forecast a continuous variable. The model displays reasonably good evaluation metrics across ...
user avatar
0 votes
0 answers
11 views

How to use K-fold classifier for comparison of different models

I am learning machine learning and went through a term K-fold cross validation. I also took notes from this site to enhance my understanding. As per the tutorial if it is 3 fold cross validation and ...
user avatar
  • 121
0 votes
0 answers
28 views

Interpreting R2 Over Cross-Validated Folds

I am running a supervised regression with cross validation, and wish to use $R^2$ as my performance metric. I am using Leave-P-Out cross validation, with P=2, which gives me approximately 4500 folds, ...
user avatar
  • 213
0 votes
0 answers
16 views

Supervised NN not learning [duplicate]

I wanted to create a supervised NN that recognises numbers on images from scratch, pretty ambitious project for someone like me. But when I test it with a simple XOr gate, it completely fails. It just ...
user avatar
0 votes
0 answers
22 views

Why Does using a OneVsRest Model for multiclass problem result in overall low accuracy but high accuracy for each individual class?

I have a multiclass problem i.e. 4 labels 0,1,2,3. I used a OneVsRest model wrapped around an xgboost model. What happens therefore is that i train a model 4x for each class. e.g.: ...
user avatar
  • 299
0 votes
0 answers
27 views

Should I scale time-series features for supervised learning classification?

I couldn't find an answer to this in the archives so posting this here. I am currently building out a supervised-learning / classification pipeline for time series forecasting (e.g. predicting the ...
user avatar
0 votes
0 answers
32 views

What is the difference between weak supervision and distant supervision? Is it just me or is there no clear-cut definition?

I've been trying to pin down what would make each category stand out. My understanding is: Weak supervision seems to be a broader term for distant supervision. The papers that I've read seem to ...
user avatar
  • 2,351
0 votes
1 answer
49 views

Learning HMM parameters by counting?

In 8.4.3 of the book Speech and Language Processing: An introduction to natural language processing, the two matrices transition probabilities and emission probabilities can be learned by counting as ...
user avatar
  • 5,151

1
2 3 4 5
13