Proof of statistical assumptions behind the methods or tests When reading statistics literature, they provide the assumptions of when specific tests/methodologies can be used. Take factor analysis as an example; you may find this inside a paper: "Generally speaking, cases with missing values are deleted to prevent overestimation (Tabachnick & Fidell, 2007). "
I wonder if the assumptions behind statistical methods/tests are all the fruits of experiments in research, so we just rote memorise the rules and use them in industrial applications? In an industrial setting, I witness people do it in this way, but I'm not confident what if they are someday being asked how to prove the assumptions are necessary, will they be able to explain that? As I'm not aware of how exactly all these assumptions and rules can be explained easily if there are any people who challenge these assumptions at all.
But I am a bit shaky in applying something I can't explain very clearly. I find this question dangling around my mind whenever I need to use any statistical techniques: Do we need to know how to prove things from the ground up before we apply methods in statistics? If not, when would it be helpful if I knew how to prove some assumptions behind the method (if they can be proved at all)? How do I balance practicality and proofs in the statistical world? I find those assumptions challenging to remember and not intuitive at all! How to make it more intuitive so I can remember not to miss a single assumption next time I apply them?
Is there any body of knowledge that lays the foundation for all the assumptions for all statistical tests/methods?
TIA.
 A: Understanding derivations (and how the assumptions are used in those derivations) is definitely helpful when using statistics. It can also make it a lot easier when you are trying to figure out how much an assumption matters (e.g. making it easier to understand how to simulate where algebraic derivation of the impact under other assumptions may be difficult).
Indeed a warning - you often see (in books, papers, etc) claims about things that are required/assumed that are not actually the case. If you know how to derive something you will know what assumptions you make in the derivation. It will save you from a great deal of bad advice (without it how do you tell a good book from a bad one? I assure you that popularity of a book in some application area is very often not a good indicator)
A common example of something that is often wrong in non-mathematical sources is normality of (marginal distributions of) DVs or even IVs in regression. Many books have their readers avidly checking things that have nothing to do with what the assumptions were and then taking drastic (and often highly counterproductive) actions to 'correct' those non-problems, causing issues  where there had been none, or sometimes leading people into abandoning perfectly reasonable analysis choices.

I wonder if the assumptions behind statistical methods/tests are all the fruits of experiments in research,

Assumptions of some procedure do not necessarily arise from empirical/experimental considerations but mathematical choices under which the properties of some procedure are derived (though situational knowledge will typically inform the choices that are made).
e.g. take the common "independent and identically distributed" assumption; that's an extremely convenient assumption, but you don't observe these things as outcomes (you don't really observe things being indepdendent, or not being heterogenous), it's something you might sometimes be able to argue should be a reasonable approximation for some situation, or in another situation, not.
Consideration of the suitability of those choices for using the procedure in some subsequent situation may (/should) be informed by prior empirical/expert knowledge/theoretical understanding of the situation of course.
A: A lot of older applied statistics and econometrics textbooks start with a wide range of assumptions, without always making them clear. Readers then learn a wide range of tools based on (possibly strong) assumptions, and only much later come back to re-examine the assumptions.
A new school of agnostic statistics practitioners are flipping this in a really exciting way. In their work, some of the it very very new, they offer introduction that starts with no assumptions. They present first principles and then slowly adds assumptions in a way that makes crystal clear what assumptions we are making, why we're making them, and how we might think about assessing whether they are reasonable or not.
Some names in this line of agnostic regression are David Friedman, Winston Lin, and P Aronow.
Refer to any of David A Freedman's books.
For the best recent work, see Aronow and Miller's recent textbook, Foundations of Agnostic Statistics: https://www.amazon.com/Foundations-Agnostic-Statistics-Peter-Aronow-dp-1316631141/dp/1316631141/ref=mt_other?_encoding=UTF8&me=&qid=
