0
$\begingroup$

I am following up on a paper that demonstrates using deep learning (CNN) for classification. Specifically, their approach transformed the spatial data into fixed-length segments appropriate for CNN requirements. In their work, the channel's dimension consists of 3-kinematics of min_speed, avg_speed, max_speed.

Based on the outcome of their work, the model achieves 80+ on the accuracy, and well on other metrics. Out of curiosity, I took the task to replicate their work and was successful.

However, on a further check, I decided to take a look at the distribution of these kinematics computed per class before feeding to the CNN, in a boxplot. It turns out that a lot of data points fall as outliers.

So I begin to doubt.

  1. Are we feeding the network with instances full of outliers despite this interesting result?
  2. How does feeding a deep learning algorithm with data points full of outliers affect it´s performance?
  3. Should we expect the network to learn representation from what seems to be outliers in the first place?

The figure below shows a boxplot of the kinematics in the channel dimension of instances. enter image description here

$\endgroup$
7
  • 1
    $\begingroup$ The other way to read the result is that the DNN is robust to values far away from the central tendency. What's the problem? $\endgroup$
    – Sycorax
    Jun 2, 2020 at 13:32
  • $\begingroup$ I was digging in to understand why this pretty good model can't generalised on a similar dataset collected in a different location, then I was tempted to explore the distribution of the kinematics to understand if there's a variation which leads model unable to generalised. So I come to understand this. $\endgroup$
    – arilwan
    Jun 2, 2020 at 14:17
  • 1
    $\begingroup$ It's possible to imagine how the process you describe could have a different data-generating process between the two image sources. A common example of photographer bias in image recognition is that cats and dogs will tend to have their eyes horizontally aligned & about 20% of the way down the image because these images are visually pleasing. If Google Search uses this fact to prioritize certain cat/dog photos, but Bing doesn't, then you'll have two different distributions of images. A DNN can pick up this fact if the model learns to detect eyes at a certain position of the image. $\endgroup$
    – Sycorax
    Jun 2, 2020 at 16:13
  • 1
    $\begingroup$ And as a result, a DNN trained on Google Search images may not perform well when tested on Bing results. $\endgroup$
    – Sycorax
    Jun 2, 2020 at 16:14
  • 1
    $\begingroup$ The image example has moved us far afield from your question about 3-kinematics, though. It seems like you could compare the distributions of the two different data sources to make a conclusion about whether or not they are similar enough. $\endgroup$
    – Sycorax
    Jun 2, 2020 at 16:27

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.