Skip to main content
Cross Validated

Return to Question

Notice removed Draw attention by user1111929
Bounty Ended with jpmuc's answer chosen by user1111929
Tweeted twitter.com/StackStats/status/1096922296785293312
deleted 1 character in body
Source Link
user1111929
user1111929
  • 220
  • 1
  • 8

I have a training set mapping some Likert-scale variables (integers between 1 and 7, rescaled to real numbers between 0 and 1) to predict a continuous variable between 0 and 1. The data set is reasonable large ($10^4$-$10^5$ rows) but very noisy (105-20% has bogus labeling).

When applying regression algorithms to this dataset, the regression algorithm predicts scores only in a subinterval of [0,1], e.g. scores below 0.1 and above 0.9 are not reached for any input value of the columns. This is expected given the bogus labels; but also inacceptable, as the prediction for non-bogus rows is severely impacted. Moreover, since 0.0 and 1.0 are the most common target values in the training set (it is a 'How much ... are you?' type prediction) it is not appreciated by the users that 0.0 and/or 1.0 would be unreachable. (Simply stretching the result to cover [0,1] greatly increases the error, so is probably suboptimal.)

What method could I use to detect and remove these mislabeled/bogus rows, or at least a significant amount of them? Since they are a minority, I would think something with majority voting should be the answer, but it's not clear how to implement such an approach, since $7^{n_{cols}} > n_{rows}$.

I have a training set mapping some Likert-scale variables (integers between 1 and 7, rescaled to real numbers between 0 and 1) to predict a continuous variable between 0 and 1. The data set is reasonable large ($10^4$-$10^5$ rows) but very noisy (10-20% has bogus labeling).

When applying regression algorithms to this dataset, the regression algorithm predicts scores only in a subinterval of [0,1], e.g. scores below 0.1 and above 0.9 are not reached for any input value of the columns. This is expected given the bogus labels; but also inacceptable, as the prediction for non-bogus rows is severely impacted. Moreover, since 0.0 and 1.0 are the most common target values in the training set (it is a 'How much ... are you?' type prediction) it is not appreciated by the users that 0.0 and/or 1.0 would be unreachable. (Simply stretching the result to cover [0,1] greatly increases the error, so is probably suboptimal.)

What method could I use to detect and remove these mislabeled/bogus rows, or at least a significant amount of them? Since they are a minority, I would think something with majority voting should be the answer, but it's not clear how to implement such an approach, since $7^{n_{cols}} > n_{rows}$.

I have a training set mapping some Likert-scale variables (integers between 1 and 7, rescaled to real numbers between 0 and 1) to predict a continuous variable between 0 and 1. The data set is reasonable large ($10^4$-$10^5$ rows) but very noisy (5-20% has bogus labeling).

When applying regression algorithms to this dataset, the regression algorithm predicts scores only in a subinterval of [0,1], e.g. scores below 0.1 and above 0.9 are not reached for any input value of the columns. This is expected given the bogus labels; but also inacceptable, as the prediction for non-bogus rows is severely impacted. Moreover, since 0.0 and 1.0 are the most common target values in the training set (it is a 'How much ... are you?' type prediction) it is not appreciated by the users that 0.0 and/or 1.0 would be unreachable. (Simply stretching the result to cover [0,1] greatly increases the error, so is probably suboptimal.)

What method could I use to detect and remove these mislabeled/bogus rows, or at least a significant amount of them? Since they are a minority, I would think something with majority voting should be the answer, but it's not clear how to implement such an approach, since $7^{n_{cols}} > n_{rows}$.

Notice added Draw attention by user1111929
Bounty Started worth 50 reputation by user1111929
Source Link
user1111929
user1111929
  • 220
  • 1
  • 8

Regression algorithm on [0,1] with lots of mislabeled data

I have a training set mapping some Likert-scale variables (integers between 1 and 7, rescaled to real numbers between 0 and 1) to predict a continuous variable between 0 and 1. The data set is reasonable large ($10^4$-$10^5$ rows) but very noisy (10-20% has bogus labeling).

When applying regression algorithms to this dataset, the regression algorithm predicts scores only in a subinterval of [0,1], e.g. scores below 0.1 and above 0.9 are not reached for any input value of the columns. This is expected given the bogus labels; but also inacceptable, as the prediction for non-bogus rows is severely impacted. Moreover, since 0.0 and 1.0 are the most common target values in the training set (it is a 'How much ... are you?' type prediction) it is not appreciated by the users that 0.0 and/or 1.0 would be unreachable. (Simply stretching the result to cover [0,1] greatly increases the error, so is probably suboptimal.)

What method could I use to detect and remove these mislabeled/bogus rows, or at least a significant amount of them? Since they are a minority, I would think something with majority voting should be the answer, but it's not clear how to implement such an approach, since $7^{n_{cols}} > n_{rows}$.