0
$\begingroup$

I am following up on a paper that demonstrates using deep learning (CNN) for classification. Specifically, their approach transformed the spatial data into fixed-length segments appropriate for CNN requirements. In their work, the channel's dimension consists of 3-kinematics of min_speed, avg_speed, max_speed.

Based on the outcome of their work, the model achieves 80+ on the accuracy, and well on other metrics. Out of curiosity, I took the task to replicate their work and was successful.

However, on a further check, I decided to take a look at the distribution of these kinematics computed per class before feeding to the CNN, in a boxplot. It turns out that a lot of data points fall as outliers.

So I begin to doubt.

  1. Are we feeding the network with instances full of outliers despite this interesting result?
  2. How does feeding a deep learning algorithm with data points full of outliers affect it´s performance?
  3. Should we expect the network to learn representation from what seems to be outliers in the first place?

The figure below shows a boxplot of the kinematics in the channel dimension of instances. enter image description here

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ The other way to read the result is that the DNN is robust to values far away from the central tendency. What's the problem? $\endgroup$
    – Sycorax ♦
    Jun 2, 2020 at 13:32
  • $\begingroup$ I was digging in to understand why this pretty good model can't generalised on a similar dataset collected in a different location, then I was tempted to explore the distribution of the kinematics to understand if there's a variation which leads model unable to generalised. So I come to understand this. $\endgroup$
    – arilwan
    Jun 2, 2020 at 14:17
  • 1
    $\begingroup$ It's possible to imagine how the process you describe could have a different data-generating process between the two image sources. A common example of photographer bias in image recognition is that cats and dogs will tend to have their eyes horizontally aligned & about 20% of the way down the image because these images are visually pleasing. If Google Search uses this fact to prioritize certain cat/dog photos, but Bing doesn't, then you'll have two different distributions of images. A DNN can pick up this fact if the model learns to detect eyes at a certain position of the image. $\endgroup$
    – Sycorax ♦
    Jun 2, 2020 at 16:13
  • 1
    $\begingroup$ And as a result, a DNN trained on Google Search images may not perform well when tested on Bing results. $\endgroup$
    – Sycorax ♦
    Jun 2, 2020 at 16:14
  • 1
    $\begingroup$ The image example has moved us far afield from your question about 3-kinematics, though. It seems like you could compare the distributions of the two different data sources to make a conclusion about whether or not they are similar enough. $\endgroup$
    – Sycorax ♦
    Jun 2, 2020 at 16:27

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.