Skip to main content

Questions tagged [multitask-learning]

Multi-task learning (MTL) is an approach to machine learning that learns a problem together with other related problems at the same time, using a shared representation.

Filter by
Sorted by
Tagged with
1 vote
0 answers
20 views

Multi-task learning-Loss function

0 I am training a convolutional autoencoder with two velocity fields (2D array) as inputs and outputs. These fields represent wind velocities in both the x and y directions within a square domain. My ...
Sarah's user avatar
  • 11
0 votes
0 answers
8 views

Correlation for a Multitask learning regression model

I have calculated the correlation for a MTL model like this as given below, and I am not sure whether my approach is correct or not. This is how I calculated it: 0) Correlation is calculated for ...
User's user avatar
  • 1
0 votes
0 answers
13 views

How to calculate correlation for a Multitask learning model?

I have calculated the correlation for a MTL model like this as given below, and I am not sure whether my approach is correct or not. This is how I calculated it: 0) Correlation is calculated for ...
User's user avatar
  • 1
0 votes
0 answers
14 views

How to develop shared bottom tower serving different tasks

I have two model classes both pyramid architecture. Let's say first task is predicting user will buy something with architecture [feature_embedding_128, dense_1048, dense_512, dense_128, dense_1] ...
aghd's user avatar
  • 314
0 votes
0 answers
44 views

How to penalize disagreement between two classification loses?

I am working with a multi-head, multi-loss neural network. Each of the two heads is associated with a multi-class classification loss. The losses are combined additively. Assume loss 1 is trained to ...
Gertrude Porter's user avatar
2 votes
1 answer
44 views

How do additional loss terms impact the parameter count in a Deep Neural Network?

I've been working on training DNNs for various tasks and recently started incorporating additional loss terms into my models to improve convergence and performance. I'm curious to know if I need to ...
Chutlhu's user avatar
  • 153
0 votes
0 answers
3 views

Quantify Independence/Jointness of Tasks in Multitask Model

Suppose I have two identical tasks that are over two related but not identical domains- for example, next word prediction in Hamlet in both English and in Spanish. I want to train a model on a mix of ...
Garrodactyl's user avatar
1 vote
1 answer
91 views

Tuning lambda in glmnet mgaussian multitask learning model for optimal support recovery

I was using a multivariate gaussian (mgaussian) glmnet model to solve the multitask learning problem below (deconvolution of a ...
Tom Wenseleers's user avatar
0 votes
1 answer
140 views

How to compute the derivative of the total loss wrt external trainable parameters?

I was just curious how external trainable parameters are updated. The challenge is to compute the derivative, the rest is handled by the optimiser. I assumed a simple DNN as follows: $$\hat{y} =\sigma(...
Prakhar Sharma's user avatar
1 vote
0 answers
49 views

Capping labels negatively impacts business metrics

I have this deep neural network model with an integer label to predict. The label is heavily skewed so we cap the labels at some value (let's say 90 %ile). Now when we build and run the model, it ...
aghd's user avatar
  • 314
3 votes
1 answer
308 views

Multiple-domain adaptation vs multi-task learning

I am confused with the definitions of domain adaptation and multi-task learning. I have K datasets, each with the same feature and label space and thus the same learning problem, but with different ...
Tim D's user avatar
  • 31
0 votes
0 answers
488 views

Objective function of multi-task learning

I have two tasks in hand to train using multi-task learning. Task 1 - sequence classification task (x1,x2,...,xn -> Y) Task 2 - sequence to sequence prediction (X1,x2,....,xn -> y1,y2,...,ym) Two ...
Asif Khan's user avatar
  • 121
0 votes
0 answers
365 views

How to know how correlated tasks are in Multitask learning?

In Multitask Learning (MTL,) one factor that can have an effect on the performance of an MTL architecture is how correlated or uncorrelated tasks are. As far as I understand, the more correlated tasks ...
Pedram's user avatar
  • 143
1 vote
0 answers
432 views

When does multi-task learning make more sense than multi-label classification?

As part of writing a book on machine learning, I am creating an extreme multi-label stack overflow question tagger for thousands of tags with varying numbers of training examples and I’ve approached ...
rjurney's user avatar
  • 151
3 votes
2 answers
850 views

Parallel multi-task learning vs. continual learning

Assuming we want to learn k tasks jointly, and the data for all tasks are available. We may either train a model with parallel multi-task learning (eg. each batch is a mixture of samples from the k ...
thinkbear's user avatar
  • 321
5 votes
2 answers
6k views

Multi-task learning: weight selection for combining loss functions

I am training a system that combines two sub-systems: one for classification and another for reconstruction. Can anyone suggestion what are the common practice for weight selection for combining two ...
talk2speech's user avatar
1 vote
0 answers
112 views

Covariance Matrix of HIERARCHICAL MULTITASK GAUSSIAN PROCESS

I'm currently trying to develop a Gaussian Process to predict different levels of different individuals over time. So it is a Time Regression Problem in which we have multiple tasks, but also ...
Tbertin's user avatar
  • 399
2 votes
0 answers
684 views

How can I maximise binary cross entropy loss?

I have a multi-task learning model with two binary classification tasks. One part of the model creates a shared feature representation that is fed into two subnets in parallel. The loss function for ...
JM1982's user avatar
  • 21
3 votes
1 answer
885 views

Classification followed by regression?

I have the following problem: I have a dataset for which my observations have a bunch of features and a continuous response (regression problem). However, some of my observations (about a fourth of ...
Tom's user avatar
  • 1,373
7 votes
1 answer
3k views

Multi-task XGBoost

Is there a way to adapt the XGBoost algorithm to the multi-task case? Say there are related output variables and for some samples, some of those outcomes are missing. Is there a way to train XGBoost ...
user5054's user avatar
  • 1,569
11 votes
1 answer
20k views

How to define multiple losses in machine learning?

I'm using TensorFlow for training CNN for classification. In machine learning, there are several different definitions for loss function. In general, we may select one specific loss (e.g., binary ...
mining's user avatar
  • 1,019
2 votes
0 answers
166 views

How to optimize the hyperparameters of a Deep Gaussian Process?

I am trying to understand an article from NIPS 2017 where Alaa and van der Schaar create a Deep Multitask Gaussian Processes (DMGP) with competing risks. I don't grab what are they trying to optimize. ...
Revolucion for Monica's user avatar
2 votes
1 answer
50 views

Robust machine learning for slightly different class proportions in multiple data sets

Say we have n similar data sets, with the same variables, and outcome labels x and y. In these data sets, domains slightly differ as suggested by the proportion of the minority class x (ranging from 1%...
sluijs's user avatar
  • 269
1 vote
0 answers
889 views

How to add task wise early stopping in keras?

I created a multi task network with keras. Because different tasks have different level of difficulties, for some tasks overfitting occurs earlier than others so they should be stopped. I know keras ...
Panda's user avatar
  • 255
0 votes
2 answers
125 views

Multitask learning Gaussian Processes

A small question In the book of Rasmussen Page 115 last paragraph. When we have multiple databases you setup a gaussian for each database and the optimisation is said can be done by adding the ...
Wilmer Ariza's user avatar
17 votes
3 answers
12k views

Difference between multitask learning and transfer learning

I am reading Caruana (1997) Multitask learning (pdf). In the definition of multi task learning, the author states that; Usually, we do not care how well extra tasks are learned; their sole purpose ...
Rafael's user avatar
  • 1,385
0 votes
2 answers
307 views

How to apply the SVD based approach to sample from a matrix-variate normal with given correlation matrices and standard deviations?

A sample from a multivariate normal distribution $X$ can be constructed to have a covariance $C$ even for positive semi-definite covariances according to this technique involving an SVD. Furthermore, $...
baf84b4c's user avatar
  • 163
6 votes
2 answers
2k views

Sampling from matrix-variate normal distribution with singular covariances? [duplicate]

The matrix-variate normal distribution can be sampled indirectly by utilizing the Cholesky decomposition of two positive definite covariance matrices. However, if one or both of the covariance ...
baf84b4c's user avatar
  • 163
3 votes
1 answer
297 views

Training N classifiers for N labels vs one classifier with N labels

I have a classification problem which is multi-label with N labels. I would like to know which method would be the better choice? Training N classifiers (1 for each label) or a single classifier which ...
jvc's user avatar
  • 133
1 vote
0 answers
344 views

What are 'tasks' in multi-task/transfer learning, linking to co-regionalisation

I am currently working on implementing a multi-task model for Gaussian process prediction, but am having some trouble interpreting the materials online (conference talks, papers, presentations/slides ...
mediantis's user avatar
1 vote
0 answers
90 views

Modeling worker performance parameters for optimum allocation of tasks to workers

Lets say we have an English to French translation task in a company and there are 100s of workers who are proficient in doing this task but each worker has its own unique attributes which allow them ...
stats101's user avatar
  • 337
13 votes
6 answers
9k views

What is the difference between Multitask and Multiclass learning

Consider a image labeling problem, where I need to assign one or more labels to an image. The possible labels are human, moving ,...
A.D's user avatar
  • 2,534
1 vote
2 answers
99 views

In the context of ANNs, is there multi-task learning iff the network has more than 1 output?

This is a terminology question: in the context of artificial neural networks, does multi-task learning occur iff the network has more than 1 output?
Franck Dernoncourt's user avatar