2
$\begingroup$

So, I have experience in machine learning for NLP and a little in neural networks for NLP, but never so far done anything in computer vision in this area so bear with me if what I am asking is a little naive.

I have many images of playing cards. These images can have one or several cards in the same image. What I want to do is train a CNN (maybe another algorithm?) to recognise the images but I have no clue where to start. I mean, if I have one card, the CNN should give me the card's name (or an id or whatever), and if I have several cards in the image, the CNN should give me the list of names of all the cards in the image.

To train such a model, what should I have as a dataset? Images of one card only with its name? Images of multiple cards with all the names of those cards? A mix of both?

Thank you for your help!

$\endgroup$
1
$\begingroup$

Easiest approach is to train a CNN to qualify the 52 cards. And use a simple OpenCV application to acquire the image(s) and do some simple findContour operations and mask those contours one at a time and supply them to the NCC. It's very tricky though if those have no space between them. Because then there won't be individual contours to be detected. You'll then have to get creative by looking for edges and find the pip and index of each card and mask those.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

This problem sounds like a great candidate for the encoder/decoder type of models. You can encode the image with a CNN and have the decoder output the card names via a RNN (LSTM/GRU). You will stop when the decoder outputs a 'stop' symbol. The RNN will end up producing a list of card names.

This is similar to the automatic image captioning which combines a CNN encoder with an LSTM language decoder.

You can also add an attention mechanism on your decoder to select part of the CNN output to decode. This should give better results than a straight forward encode/decode.

Finally, you can take the CTC approach used in speech transcription where spectrograms are lined up with letters. Your spectrograms will be slices of the final layer in your CNN and your letter would be the card names.

If you have boundary data of the cards the problem becomes easier as you don't have to learn a reader mechanism. Adding this data is expensive though and in my opinion not in the spirit of end-to-end deep-learning.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.