2
$\begingroup$

This is a multiclass classification task for a balanced dataset. I am using the Random Forest Classifier for classification, and I have calculated the evaluation metrics:

Training set metrics:

Accuracy: 1.0

Precision: 1.0

Recall: 1.0

F1-score: 1.0

Testing set metrics:

Accuracy: 0.9995151906916613

Precision: 0.99955912143285

Recall: 0.9992709151615126

F1-score: 0.9994144032680475

Cross-validation scores:

[0.99982833 0.99991416 1. 0.99974249 0.99957082 1.

  1.     0.99974247 1.         1.        ]
    

I obtained these results after filtering out the noise from the data, as shown in the image.

enter image description here

It is important to note that I perform pre-processing before splitting the data into training and testing sets, but I am unsure if this yields realistic results.

I am new to machine learning, and I'm wondering why the results are so good!

$\endgroup$
9
  • $\begingroup$ It's nearly impossible to say given the numbers are so close to 1.00 and each other. Why do you think you overfitted? $\endgroup$
    – gunes
    Commented Jun 27, 2023 at 19:01
  • $\begingroup$ thanks for reply, I am new in machine learning, and you are right there is no overfitting. I am studying this subject in university for the first time and I face some difficulties, So are these good results for both training set and testing set.. Thank you so much in advanced. $\endgroup$ Commented Jun 27, 2023 at 19:42
  • $\begingroup$ Do you have pseudoreplication? en.m.wikipedia.org/wiki/Pseudoreplication or another form of data leakage? $\endgroup$
    – Ggjj11
    Commented Jun 27, 2023 at 20:02
  • 4
    $\begingroup$ Obviously we have no way to tell you that your results and what you have done was are correct. However the posted numbers don't indicate in any way that anything went wrong. If I were you I'd be curious why the results are so good, but I can imagine situations in which this is the case, particularly certain artificially generated toy problems used for teaching/demonstrating. $\endgroup$ Commented Jun 27, 2023 at 20:08
  • $\begingroup$ Thanks for your response. I'm also curious about why the results are so good! It's noteworthy that I obtained these results after filtering out the noise from the data. $\endgroup$ Commented Jun 27, 2023 at 20:18

1 Answer 1

4
$\begingroup$

This high an accuracy suggests one of several things:

  1. It could be a trivially easy prediction task. It being trivially easy does not exclude the possibility that a model for it could be useful.
  2. It could be a prediction task, where any approach will achieve a high accuracy (and perhaps accuracy is not so meaningful). Examples include cases where predicting the majority class would perform amazingly well, already. E.g. for predicting whether you will die tomorrow, the constant prediction "No" will achieve >>99% accuracy across the population.
  3. There is target leakage. Performing pre-processing before train-test-splitting is certainly potential suspect and I've seen real cases where it has (misleadingly) increased metrics like accuracy from <80% to ~99%.
  4. Your cross-validation may not be set-up in an appropriate way for the problem. E.g. with time series data (which you seem to say you have), you may want to use the future as your test set based on training from the past, or predict for a completely new time series (based on training on several others - e.g. if you have time series of measurements from different people, you may want to predict things for a new person). This really depends on how the model would get used in practice and the (cross-)validation-splitting should match what would truly be available for the model. Otherwise you may end up answering (often) pointless questions like "Can I predict something for tomorrow, if I magically didn't just know the past up to now, but also what happens in the future after tomorrow?".
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.