How to read equations in DL/ML research papers I'm a junior in Computer Science. I have been learning about DL/ML on my own for almost a year and a half. A couple of months ago, I started to read research papers, and I have difficulty reading them and understanding them. English is a second language for me, and although I can read undergraduate level books, reading graduate-level books and research papers seems hard for me. I'm planning to do graduate studies in AI, and understanding graduate-level books and research papers are crucial.
For example today I was reading this research paper: https://arxiv.org/abs/1703.07737 and I stumbled upon this equation:

I have difficulty understanding what it is saying. I have experience with calculus, basics of linear algebra, and basics of matrix calculus. I looked online and I found this implementation: https://omoindrot.github.io/triplet-loss I noticed that during the definition of batch_all_triplet_loss function, he used very fancy matrix vectorization.
It would never cross my mind to do this kind of vectorization, not with my current level of understanding. How the author of this blog, was able to understand that equation and to write it with this level of vectorization? Are there books/resources that help to transition from an undergraduate level to graduate level?
 A: None of the concepts used in this equation are actually advanced beyond 1st year math or physics undergrad concepts, with the possible exception of the notation $[x]_+$, which just means $max(x,0)$. It's just that the formula is a very complicated one with very obtuse notation.
Being able to read this type of notation is something people just get used to with practice. A good book for this, which also doubles as the standard reference volume for ML theory, is Elements of Statistical Learning Theory, and its undergraduate companion volume by the same authors, Introduction to Statistical Learning.
If you have a solid CS undergrad background and want to get into graduate level A.I. and M.L., I recommend that you take an intermediate probability and stats course (or go through an equivalent text book on your own, if your curriculum isn't that flexible), beyond just the standard introductory material that most CS students learn. Also make sure you get the basics of integral transforms like the Fourier Transform and the Laplace Transform, and the basics of Information Theory, like Shannon Entropy and Kullback-Liebler Divergence, etc...in my experience, a lot of AI and ML grad students are from other STEM backgrounds (EE, Physics, etc...) and tend to be already familiar with these concepts from their undergrad curriculum, so prof.'s go over them quickly and the students coming from pure CS backgrounds tend to struggle as a result.
A: Since math is rarely the main focus of deep learning papers (excepting theory papers of course), usually the notation is not that polished, and small typos are not uncommon. I find that the key is to ignore all mathematical notation at first, and try to figure out the motivation / high level ideas from the authors.
In the case of this paper, based on a 2 minute skim, I figured out that they're starting from something called a "triplet loss" (which I am familiar with), then add on "hard negative mining" (another popular idea), and then tweaks it a bit more to turn it into equation 6.
Since I am already familiar with triplet loss and hard negative mining, I can make sense of equation 4 and 5 without trying to reference how every single subscript or superscript is defined. Then it's not much of a leap to see how the authors arrive at equation 6.
It's very easy to get lost by trying to parse every bit of notation sequentially. I think it's much more efficient to

*

*be familiar with the past literature and the common ideas in the literature (in this case, triplet loss + hard negative mining),

*figure out what the authors are trying to change / improve on at a conceptual level,

*Using this knowledge, you can usually decipher what the math says without reading the fine notation.


To be honest it would never cross my mind to do this kind of
vectorization, not with my current level of understanding. How the
author of this blog, was able to understand that equation and to write
it with this level of vectorization? Also how the author of the paper
was able to write this equation? I have difficulty interpreting basic
operations in matrix formula notation.

I think this is a separate problem -- It's very important to have an good understanding of basic linear algebra if you want to understand a lot of the notation. On the other hand if you're just referring to the code -- I think the broadcasting and manipulations of rank-n "tensors" common to many ML frameworks is tricky for everyone the first time they see it -- just keep reading and implementing and it should come naturally.
