1
$\begingroup$

I'm doing regression prediction over a dataset which has a lot of samples with same features but different target values. E.g.

f1 |f2 |f3 |f4 |target
1.0|3.0|4.1|5.2|120.2
1.0|3.0|4.1|5.2|95.1

of course some outlier analysis could be done, but seems that target values are actually affected by a measurement error. also taking the mean of targets would be an option.

My questions:

  • can prediction methods handle situations like this, by keeping in the training dataset all the above examples?
  • which are the proper output scoring metrics to deal with cases?

Thanks!

$\endgroup$
1
  • 1
    $\begingroup$ It’s fine. The usual two-sample t-test is an example of this. $\endgroup$
    – Dave
    Nov 13, 2020 at 12:26

1 Answer 1

2
$\begingroup$

First, using your terminology, there must be at least some observations with different features, i.e. the matrix of features should be of full rank. You can search for "multicolinearity" problems that your dataset might cause.

Second, repeating features are characteristic of longitudinal data, or panel data, where multiple measurements are taken over the same individuals. An appropriate modeling strategy in those cases is to add random effects to account for the dependence in the data: look up "linear mixed models". Such models are perfectly equipped to deal with this type of data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.