No, you should select SNPs that are correlated with $X$ (strictly speaking, those that are correlated with $X$ because they are in linkage disequilibrium with variants that affect $X$: correlation is not enough), and that are correlated with $Y$ only because they are correlated with $X$ (the so-called exclusion restriction)
It's easier to think about if we first think about one SNP at a time and pretend the SNP genotype is binary rather than binomial, and pretend that the SNP is actually the functional variant rather than just near it.
Let $\lambda_1$ be the mean of $X$ for people with the variant allele and $\lambda_0$ be the mean of $X$ for people with the reference allele. The difference $\lambda_1-\lambda_0$ is the effect of the SNP on $X$. It's the effect, because (by assumption) the only way $X$ is correlated with genotype is through the effect of genotype on $X$.
Let $\mu_1$ be the mean of $Y$ for people with the variant allele and $\mu_0$ be the mean of $Y$ for people with the reference allele. The difference $\mu_1-\mu_0$ is the effect of the SNP on $Y$. It's the effect, because (by assumption) the only way $Y$ is correlated with genotype is through the effect of genotype on $X$ and the effect of $X$ on $Y$. The assumption that the genotype has no effect on $Y$ except through $X$ is a strong one and is critical: violations of it matter.
Under these assumptions, if the effect of $X$ on $Y$ is linear, we can estimate it by $(\mu_1-\mu_0)/(\lambda_1-\lambda_0)$.
Now, going back to the oversimplifications. It doesn't matter if the SNP we used is just a marker rather than the functional variant, because the impact of this on the numerator and denominator cancel.
It doesn't matter that genotype has three levels (for bi-allelic diploid SNPs) as long as the effect of SNP on $X$ is linear (an additive genetic model), and these only needs to be approximately true
It does potentially matter that the effect of $X$ on $Y$ is linear, and this is typically not true, but the genetic effects being studied by Mendelian randomisation are usually weak enough that linearity is a reasonable approximation (and the test for non-zero effect still makes sense).
It does seriously matter that the correlation of genotype with $X$ is causal. For example, there is a well-studied polymorphism in the alcohol dehydrogenase 1B gene that is correlated with using chopsticks to eat (that is: it's more common in East Asia than in other parts of the world), but the correlation is not causal and a Mendelian randomisation analysis would be invalid.
And, again, the assumption that all the effect of genotype on $Y$ goes through $X$ is important. For example, a polymorphism affecting bitter taste perception might decrease $X$ defined as broccoli consumption and increase $Y$ defined as weight, but the effect might be due to, say, increased consumption of sweet foods, and a Mendelian randomisation analysis would be bogus.
