"Examining the data" before analysis: I still don't know what I'm looking for I am a recent graduate of a biostatistics program working as an analyst for health-related studies. We were taught to always "examine the data" before conducting an analysis by generating summary statistics to determine if the analysis you plan to run is appropriate for the data you have, has bias, could be done in a more valid way, etc. I still don't understand... What exactly am I looking for that can improve my analysis methods?
I understand that each method of analysis has a set of assumptions, and these assumptions must be met for the method to produce valid results. But it seems like there's SO much more to be learned from descriptive statistics tables than I can see. I can ensure that the data meets these assumptions, and still someone far more senior than me will say something like "Ah yes, but there isn't enough variability between these two categories of this variable and so the estimates for these categories won't be reliable."
Say, for example, I plan on running a Cox proportional hazards model. I have generated a summary table which shows the frequencies and means of my covariates for event=0 and event=1 of my outcome. This type of descriptive statistics table is a staple in pretty much every peer-reviewed observational study ever. What is the point of this table? What information can I glean from this table that will affect how I will run my Cox model?
I have tried searching online and in books for what feels like days for even one resource that can help me understand what to look for in the data that can help me improve my analysis methodology, but the only resources I could find were not for people actually doing the analysis (not technical enough).
I feel as if I'm going insane. What have I missed?!
 A: It is useful to think of data analysis as being split into two parts, called exploratory analysis and confirmatory analysis.  These two aspects of analysis complement each other, and they are both important (though the former does not always get enough credit), but you have to be very careful when you combine them.  As a general rule, it is bad to use the same data for both, and it is bad to give yourself too much flexibility at the modelling stage.  Here is some general advice.

Exploratory data analysis: In exploratory data analysis we don't yet have a specific research question (i.e., not specific enough to test yet) and we want to look at the data to figure out what kind of patterns are there that would indicate useful hypotheses to test (using different data).  In this phase we rely primarily on graphs of the data.  If we fit any formal models, these are used merely to obtain other useful metrics to graph (e.g., residuals from a particular kind of analysis), and not to undertake formal testing of hypotheses.  Descriptive statistics are less helpful than graphs, and so it is not surprising that when you look at the former, you don't see anything useful.  Instead you should try making some scatter-plots, correlation plots, density plots, violin plots, time-series plots, etc., based on how your data is structured.  This should give you a better sense of what is going on in the data, and allow you to see patterns indicating research hypotheses to test (with different data).
For the particular case you mention, you have a summary table of frequencies.  This is effectively a matrix of count values, and you can represent this graphically in a matrix plot, showing the size of each frequency (e.g., as shades of a colour, or as circles of different sizes, etc.).  You can also make a similar plot showing relative frequencies, scaled by the row or column totals.  This should allow you to visualise the extent to which the absolute/relative frequencies change in shape as you move across the rows or columns.  Since pictures are more rapidly interpreted than tables of numbers, it is likely that this will capture your data more clearly than a table of frequency values.
Confirmatory data analysis: The other part of data analysis is confirmatory data analysis, where we already have hypotheses that are specific enough to test, and we know (within a reasonably narrow scope) how to test them without further inspection of the data.  At this point we model the data and use the model to test our hypotheses.  We still look at diagnostic plots to make sure our model fits the data, and we tinker with our model if it doesn't fit, but at this stage, we know the hypotheses we are testing and we know the model form we want to use, within a narrow amount of "wiggle room".
Danger!  Danger! Now, as anyone in applied statistics can tell you, there is enormous danger in testing hypotheses using the same data that you looked at in the exploratory phase to formulate your hypotheses.  If you undertake exploratory analysis, see patterns, formulate hypotheses based on these patterns, and then test those hypotheses using the same data, you are biasing your analysis enormously in favour of positive results in whatever tests you conduct (i.e., you are going to get lots of false positives).  This kind of post hoc analysis leads to such invalid statistical practices as data dredging, p-hacking, etc., but it also sometimes manifests in more subtle ways.  In fact, the same phenomenon occurs to a lesser degree whenever we have a wide discretion in our modelling at the confirmatory analysis stage.  So any statistician reading your question is going to wince a little bit when you say that you have been taught to examine the data before your analysis, without any further qualification.
Be careful to split exploratory and confirmatory analysis: Aside from overt data dredging, there are many subtle and unintentional ways that you can bias your analysis if you have too much flexibility at the confirmatory analysis stage.  The only way to learn about this is to read widely about it, and there are many excellent discussions on this topic, particularly by the excellent statistician Andrew Gelman.  Gelman and Loken (2013) discuss the "garden of forking paths" that confronts researchers who have excessively wide scope to vary their models in the confirmatory stage, and Gelman and Guerts (2017) discuss the application of these issues to the replicability crisis in social science research.  There is also a useful discussion of researcher degrees-of-freedom by Jeff Leeks here.  These are good resources to get you starting in understanding this issue.
