0
$\begingroup$

The objects or points in this scenario are videos. Each video is represented by a set of features (suppose 10). Now if we have two videos that we need to calculate euclidean distance between them, it is pretty clear for me how to do it. But if the features for the video were generated on multiple levels instead of just one, i.e instead of generating the features for the whole duration of the video, we do it in several parts, like we divide the video into 4 parts (quarters) according to its duration and generate the features for those 4 parts/segments.

ps: The dividing of a video according to its duration is just an example for the sake of clarity of this question, but in reality it is divided according to some other criteria that could result for one video to be segmented into 3 parts and another one into 4 parts.

Now the question, if the video was represented in the above explained format, what is the best way to calculate the distance (euclidean etc.) between two videos given that not all videos have same number of parts/segments ?

My ideas included:

  • If the two videos have unbalanced number of segments, only compare the common segments. (This would imply losing a lot of data)
  • If video1 has n segments and video2 has m segments, the comparison would be: for every i in n compare it to every j in m. (This maybe computationally complex)

I'd appreciate all help, like referral to an article, paper, section in some book, or simply your informed answer.

Improve this question
$\endgroup$
2
  • $\begingroup$ Why extract features? How about representing each frame of each video as a matrix of pixels and then each video as a tensor? You can then use a tensor norm to compare them. $\endgroup$
    – Sam Weisenthal
    Commented Jun 23, 2017 at 15:34
  • $\begingroup$ The aim is to identify and categorize videos based on their content, be it audio, video, metadata, transcripts... $\endgroup$
    – Hasan Jawad
    Commented Jun 24, 2017 at 13:21

1 Answer 1

Reset to default
1
$\begingroup$

Well there is a pretty standard procedure, Just search for Bag of words. The two videos are mapped into basically a histograms of features regardless of video duration. Your question is like asking how two documents can be compared given that they contain different words.

First you build a dictionary, and then you map your documents onto your dictionary and each document is represented by a histogram of which and how many times each word appears in the document. In your case features of videos can be lots of stuff, search bibliography for video features, you shall find dense trajectories or ISA features and many many more...

Improve this answer
$\endgroup$
3
  • $\begingroup$ I have my own bibliography of features for videos, and I have the histogram (vector) for each video. The method of comparison doesn't matter now, the issue is that the videos are not just one segment. Each video is split into different number of segments and each segment has its own histogram too. Its like a problem of how to take these different number of segments of two separate videos and use them in a comparison method ? $\endgroup$
    – Hasan Jawad
    Commented Jun 24, 2017 at 13:19
  • $\begingroup$ You can combine your feature vectors for example averaging histograms to end up with one histogram for each video which I thing is not a good choice or you can use the modified Hausdorff distance, treating histograms as sets. However there might be other options to explore of which I am unaware... $\endgroup$
    – vgavriilidis
    Commented Jun 26, 2017 at 11:46
  • $\begingroup$ Hausdorff distance suggestion is quiet an interesting solution. I'm working on an implementation to see what results I could come up with. Thanks for keeping up with me. $\endgroup$
    – Hasan Jawad
    Commented Jun 27, 2017 at 13:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.