What to learn after Casella & Berger? I am a pure math grad student with little background in applied mathematics. Since last fall I have been taking classes on Casella & Berger's book, and I have finished hundreds (230+) of pages of exercise problems in the book. Right now I am at Chapter 10.  
However, since I have not majored in statistics or planned to be a statistician, I do not think I will be able to invest time regularly to continue learning data analysis. My experience so far is telling me that, to be a statistician, one needs to bear with a lot of tedious computation involving various distributions (Weibull, Cauchy, $t$, $F$...). I found while the fundamental ideas are simple, the implementation (for example the LRT in hypothesis testing) can still be difficult due to technicalities.
Is my understanding correct? Is there a way I can learn probability & statistics that not only covers more advanced material, but can also help in case I need data analysis in real life? Will I need to spend $\ge$20 hrs per week on it like I used to?
While I believe there is no royal road in learning mathematics, I often cannot help wondering – most of the time we do not know what the distribution is for real life data, so what is the purpose for us to focus exclusively on various families of distributions? If the sample size is small and the central limit theorem does not apply, how can we properly analyze the data besides the sample average and variance if the distribution is unknown?
My semester will end in a month, and I do not want my knowledge to evaporate after I start to focus on my PhD research. So I decided to ask. I am learning R, and I have some programming background, but my level is about the same as a code monkey.
 A: My advice, coming from the opposite perspective (Stats PhD student) is to work through a regression textbook. This seems a natural starting point for someone with a solid theoretical background without any applied experience. I know many graduate students from outside our department start in a regression course. 
A good one is Sanford Weisberg's Applied Linear Regression. I believe it's on its fourth version. You could probably find relatively cheap older versions. 
http://users.stat.umn.edu/~sandy/alr4ed/
One nice thing about this textbook, particularly given your relative inexperience with R, is the R primer available via the above link. It provides sufficient instruction to recreate everything done in the book. This way, you can actually learn regression (in addition to some basics of GLM), without your lack of R programming holding you back (and you'll probably pick up many of the R basics along the way). 
If you want a comprehensive introduction to R, you may be better served going through Fox and Weisberg's An R Companion to Applied Regression, but it sounds like you would rather learn statistics than programming (if those two things can be thought of separately).
As far as your time commitment concern, I really don't think you would find this textbook or material overly difficult.  Unlike Casella-Berger, there won't be much in the way of proofs or derivations. It's generally pretty straightforward. 
As an aside, there seem to be solutions floating around online (or were at some point), so you could attempt problems, check solutions, and kind of speed work your way through the book.  
A: I'm trying in a roundabout way to be more of a statistician myself, but I'm primarily a psychologist who happens to have some quantitative and methodological interests. To do psychometric work properly, I've been studying advanced (for a psychologist) methods that I wouldn't dream of calculating manually (much less would I know how). I've been surprised at how accessible and convenient these methods have become through all the dedicated efforts of R package programmers over the past decade. I've been doing real-life analysis with new methods that I've learned to use in much less than 20 hours per method...I might spend that much time on a new method by the time I'm ready to publish a result using it, but there's certainly no need to make a part-time job of studying just to make progress like I have. Do what you can as you find the time for it; it's not an all-or-nothing pursuit if you don't need it to be.
I certainly haven't focused exclusively on any topic, let alone families of distributions; I doubt that any honest-to-goodness statistician would study so narrowly either. I've dabbled in theoretical distributions for maybe an hour per day on a few occasions over the past week; that's been plenty to prove useful in real data applications. As far as I can tell, the idea isn't so much to classify distributions strictly; it's to recognize distribution shapes that resemble theories and use them to help decide the appropriate analyses and understand basic dynamics. I've shared similar thoughts on my most recent answer to "Is it better to select distributions based on theory, fit or something else?"
You haven't said what analysis you want to perform in what I assume was your hypothetical worst- case scenario, but there are ways to study the sensitivity of any analysis to sampling error. If the CLT doesn't apply, there are still several statistical questions you can ask if you know how. Nonparametric methods generally make very limited assumptions about distributions, so prior knowledge of the shape of a population's distribution isn't necessarily a major problem.
Knowledge in general doesn't really evaporate all that quickly or completely, but if you don't use it, you will find it harder to recall freely. You will retain a recognition advantage much longer, which could still come in handy if you ever need to study topics you've studied several years before...but if you want to remain fluent in what you've learned, keep on using it, and keep on learning! R is definitely a good place to invest any spare study time you have. It should help with your pure math too: see another of my recent answers to "Best open source data visualization software to use with PowerPoint."
A: I stumbled upon this one in 2019. My two cents. 
I'm a statistics professor with an inclination to do data analysis of various kinds (that's why I chose statistics!). To pick up some practical knowledge, I recommend James, Witten, Hastie and Tibshirani "An Introduction to Statistical Learning". They even have a MOOC based on that. The book uses a lot of "real data" examples and is also R-based.
A: 
I do not think I will be able to give regular time investment to continue learning data analysis

I don't think Casella & Berger is a place to learn data much in the way of data analysis. It's a place to learn some of the tools of statistical theory. 

My experience so far telling me to be a statistican one needs to bear with a lot of tedious computation involving various distributions(Weibull, Cauchy, t, F...).

I've spent a lot of time as a statistician doing data analysis. It rarely (almost never) involves me doing tedious calculation. It sometimes involves a little simple algebra, but the common problems are usually solved and I don't need to expend any effort on replicating that each time.
The computer does all the tedious calculation.
If I am in a situation where I'm not prepared to assume a reasonably standard case (e.g. not prepared to use a GLM), I generally don't have enough information to assume any other distribution either, so the question of the calculations in LRT is usually moot (I can do them when I need to, they just either tend to be already solved or come up so rarely that it's an interesting diversion).
I tend to do a lot of simulation; I also frequently try to use resampling in some form either alongside or in place of parametric assumptions.

Will I need to spend 20hr+ per week on it like I used to be? 

It depends on what you want to be able to do and how soon you want to get good at it.
Data analysis is a skill, and it takes practice and a large base of knowledge. You'll have some of the knowledge you need already.
If you want to be a good practitioner at a wide variety of things, it will take a lot of time - but to my mind it's a lot more fun than the algebra and such of doing Casella and Berger exercises.
Some of the skills I built up on say regression problems are helpful with time series, say -- but a lot of new skills are needed. So learning to interpret residual plots and QQ plots is handy, but they don't tell me how much I need to worry about a little bump in a PACF plot and don't give me tools like the use of one-step-ahead prediction errors.
So for example, I don't need to expend effort figuring out how to do reasonably ML for typical gamma or weibull models, because they're standard enough to be solved problems that have already been largely put into a convenient form.
If you come to do research, you'll need a lot more of the skills you pick up in places like Casella & Berger (but even with those kind of skills, you should also read more than one book).

Some suggested things:
You should definitely build up some regression skills, even if you do nothing else.
There are a number of quite good books, but perhaps Draper & Smith Applied Regression Analysis plus Fox and Weisberg An R Companion to Applied Regression; I'd also suggest you consider following with Harrell's Regression Modelling Strategies
(You could substitute any number of good books for Draper and Smith - find one or two that suit you.)
The second book has a number of online additional chapters that are very much worth reading (and its own R-package)
--
A good second serving would be Venables & Ripley's Modern Applied Statistics with S.
That's some grounding in a fairly broad swathe of ideas.
It may turn out that you need some more basic material in some topics (I don't know your background).
Then you'd need to start thinking about what areas of statistics you want/need -- Bayesian stats, time series, multivariate analysis, etc etc
A: Answering for others who come to this question later…


real life data analysis

Learn databases (SQL), dplyr/pandas, unix tools (sed, grep), scraping, scripting, data cleaning, and software testing. The various specialised distributions have little value in industry.
An applied regression book like Angrist & Pischke, Faraway, or Weisberg, will be a more practical kind of theory.

most of the time we do not know what the distribution is for real life data, so what is the purpose for us to focus exclusively on various families of distributions

Hence the interest in nonparametric statistics. But at the same time nonparametric with no assumptions is too loose. To answer your question, the specialised families can be thought of as answers to simple questions that you might, maybe come across. For example I think of a Gaussian as a "smooth" point-estimate. Poisson answers another simple question. When people build mathematical models these special can be useful fulcrum points. (But academics do often take the quest for the master distribution the wrong way.)
OP: Hope you had fun with your PhD research!
