I have a question. There's a library (uses this paper) which suggests in its cross lingual part that if the XLM-R is trained in english dataset, it can be directly applied to datasets in other languages, and zero shot cross lingual transfer can be conducted.
So, my question is, if I trained XLM-R for english summarisation task, will it be able to transfer that knowledge and generate summaries in other languages using zero shot cross lingual transfer? I already have code written and tested for small dataset, but it requires a good amount of computational power for the whole dataset, so that's the reason for asking this question.
Edit: A little update for anyone who was interested in this question. I tried to train the XLM-R on a very little portion of food review dataset (5500 examples), it still seems to output some summaries (1-2 words on test dataset). This was for english language. I switched the language to german (I think it's lexically similar to english. I might be wrong as neither of the languages are my first language, anyways), but didn't train the model on the german dataset, and the output was horrible. It's giving the same words as the summaries. I even switched the language to japanese for the test dataset alone, still no results were found (as expected).