Skip to main content
Post Made Community Wiki by whuber
Post Closed as "Opinion-based" by whuber
Source Link
Hefe
  • 111
  • 3

How does one develop an intuitive understanding of statistical methods and the ability to "know" which test to use, why, and how to interpret results

This is not a mathematical or coding question. Apologies if this is not the forum for this and please let me know if I need to ask this somewhere else.

I have a masters degree in data science and I have been working as a database analyst for a year. I work in medical sciences research within academia on a team that uses hospitalization data to conduct research on certain medical conditions, treatments, hospital practices, and outcomes.

My main role within the team is more the data curation side of projects. I'm usually given an overview of the research question and a list of criteria that define the cohort for that study, and I use those specifications to build our cohort and perform data cleaning and any transformations to the data required to correctly structure our data.

Often I then calculate basic summary statistics for our finalized output dataset to get a broad understanding of the cohort. These are typically just simple counts of numbers of patients who fit certain sets of conditions we'll be interested in analyzing later.

Occasionally I'll run some chi-squares or t-tests on some of the variables, or I'll be in charge of performing data visualization tasks, but any further statistical analyses tend to be passed on to someone else or if I'm to do it, someone with either a PhD or an MD or both on the team will just tell me what to do after validating my data and process.

My question is that, after all this training, all these years, all this experience, all the many many times I have been in classes that explained these things, I still feel like if I'm lucky enough to grasp how just one test works (let's say linear regression), why we use it, what it tells us, and how to interpret it, I never retain that knowledge and worse very little of it sticks in the first place.

I'd like to develop this basic comprehension and eventually intuition for knowing which tests are used in which situations and why, the relationship between the data structure and choice of stats methodology in relation to the hypothesis, the ability to understand the intricacies of the parameters you set for the model and why, how to interpret the results, and all the foundational concepts like variance and covariance and standard error and residuals and the different types of effects and the mathematics that explain all of this and on and on.

It feels like it's just a different language that certain people can grasp and that I'm not one of them. Or that I'm just too lazy to put in the time and effort to become proficient. Or what I'm hoping is the case which is that I just haven't found the right strategy to learn this stuff and that the traditional classroom and lecture format is just not effective (for me) for learning this stuff.

I know it's probably like anything else, I just need to practice and etc. I would agree with that but I also would love to hear your thoughts and tips and personal experience with the process of developing your stats mastery. Of course there is the factor of innate ability, so outside of being a stats genius from birth, what do you think helps people get better at it? I feel like this is the next step in developing my career in this field.