Return to Question

For the sake of the question, assume I'm presented with a simply classification task (for simplicity, let's assume binary classification). We are given a feature matrix $X$ and a target vector $y$ of the same length, $0, 1$ - valued as stated. My question would be how to handle errors in my target data when training some classification model (e.g. xgboost). I expect either one of two types of errors:

1. Manual entry errors, i.e. purely random errors stemming from data entry
2. Systemic errors, example: conistent underreporting, i.e. for every $1$ in $y$ I can be sure that it was entered correctly and reflects the actual value, but occasionally an actual $1$ is entered as $0$.

I'll assume those errors appear in low frequency, but not in vanishingly small numbers (think 1%-5% errors in $y$); otherwise I would assume either ignoring the errors or abondoning the project altogether would be advised. To keep things simple, let's also assume roughly balanced classes (i.e. neither class appears less often than, say 20% in the error-free actual $y_{actual}$.).

Are there any alterations to the usual procedure and modelling that should be made here? Would the first kind of error and the second kind lead to different adjustments?

My first thought was that I probably want less variance than I would usually accept (i.e. use methods to prevent overfitting judiciously), hoping my model doesn't learn 'wrong' information.

For the sake of the question, assume I'm presented with a simple classification task (for simplicity, let's assume binary classification). We are given a feature matrix $X$ and a target vector $y$ of the same length, $0, 1$ - valued as stated. My question would be how to handle errors in my target data when training some classification model (e.g. xgboost). I expect either one of two types of errors:

1. Manual entry errors, i.e. purely random errors stemming from data entry.
2. Systemic errors, example: conistent underreporting, i.e. for every $1$ in $y$ I can be sure that it was entered correctly and reflects the actual value, but occasionally an actual $1$ is entered as $0$.

I'll assume those errors appear in low frequency, but not in vanishingly small numbers (think 1%-5% errors in $y$); otherwise I would assume either ignoring the errors or abondoning the project altogether would be advised. To keep things simple, let's also assume roughly balanced classes (i.e. neither class appears less often than, say 20% in the error-free actual $y_{actual}$.).

Are there any alterations to the usual procedure and modelling that should be made here? Would the first kind of error and the second kind lead to different adjustments?

My first thought was that I probably want less variance than I would usually accept (i.e. use methods to prevent overfitting judiciously), hoping my model doesn't learn 'wrong' information.

For the sake of the question, assume I'm presented with a simply classification task (for simplicity, let's assume binary classification). We are given a feature matrix $X$ and a target vector $y$ of the same length, $0, 1$ - valued as stated. My question would be how to handle errors in my target data when training some classification model (e.g. xgboost). I expect either one of two types of errors:

1. Manual entry errors, i.e. purely random errors stemming from data entry
2. Systemic errors, example: conistent underreporting, i.e. for every $1$ in $y$ I can be sure that it was entered correctly and reflects the actual value, but occasionally an actual $1$ is entered as $0$.

I'll assume those errors appear in low frequency, but not in vanishingly small numbers (think 1%-5% errors in $y$); otherwise I would assume either ignoring the errors or abondoning the project altogether would be advised.

Are there any alterations to the usual procedure and modelling that should be made here? Would the first kind of error and the second kind lead to different adjustments?

My first thought was that I probably want less variance than I would usually accept (i.e. use methods to prevent overfitting judiciously), hoping my model doesn't learn 'wrong' information.

For the sake of the question, assume I'm presented with a simply classification task (for simplicity, let's assume binary classification). We are given a feature matrix $X$ and a target vector $y$ of the same length, $0, 1$ - valued as stated. My question would be how to handle errors in my target data when training some classification model (e.g. xgboost). I expect either one of two types of errors:

1. Manual entry errors, i.e. purely random errors stemming from data entry
2. Systemic errors, example: conistent underreporting, i.e. for every $1$ in $y$ I can be sure that it was entered correctly and reflects the actual value, but occasionally an actual $1$ is entered as $0$.

I'll assume those errors appear in low frequency, but not in vanishingly small numbers (think 1%-5% errors in $y$); otherwise I would assume either ignoring the errors or abondoning the project altogether would be advised. To keep things simple, let's also assume roughly balanced classes (i.e. neither class appears less often than, say 20% in the error-free actual $y_{actual}$.).

Are there any alterations to the usual procedure and modelling that should be made here? Would the first kind of error and the second kind lead to different adjustments?

My first thought was that I probably want less variance than I would usually accept (i.e. use methods to prevent overfitting judiciously), hoping my model doesn't learn 'wrong' information.