What is an appropriate amount of mathematical background for reading The Elements of Statistical Learning? Of course, more is always better, but what are the key things you'd recommend a reader know before diving in, in order to get a worthwhile experience out of it?
In terms of Mathematics one does not need a heavy background for a first comprehensive read of ESL. What a first year CS/Engineering undergraduate would know is adequate. For example, in terms of Linear Algebra the levels of G. Strang's Introduction to Linear Algebra and in terms of Multivariate Calculus J. Stewart's Calculus will get you through most of ESL without any issues. Refreshing your basic conditional probabilities will not go to waste either.
Some topics like the ones based on reproducing Hilbert kernel spaces, wavelets and importance sampling will not be fully accessible and one would probably need to read about them separately in order to fully grasp them. Nevertheless the book tries to explain them in a relative straightforward manner so the reader can obtain enough working knowledge to get the gist of the text.
Check the thread Can you recommend a book to read before Elements of Statistical Learning? too. It is not really concerned with Mathematical preliminaries but rather with "conceptual ones" mostly. As a somewhat smoother and more mathematically self-contained alternative to ESL you might want to consider K. Murphy's Machine Learning: a Probabilistic Perspective.