I've tried looking online and I can't seem to grasp the Rao-Blackwell theorem.
Could someone please give an intuitive explanation backed up by formulae.
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
First, let's recall what a sufficient statistic is: intuitively it captures all the information about a parameter there is in the sample. In other words, if we use the original sample, rather than the statistic we would not be able to estimate our parameter any better.
To formalize these two claims we can use factorization criterion to back up the first, and a Cramer-Rao lower bound for the second.
Rao-Blackwell theorem tells you that you can always improve (or an least not do worse) on your estimator's quality (measures by variance) if you condition your estimator on the sufficient statistic. Because this statistic provides all the information about this parameter from the sample, it contains all you know about the parameter. By conditioning you use all this available information. That's why it is intuitive, it should be the best (where by best we mean smallest variance and since it is unbiased also smallest mean squared error).
This is a chatty explanation but you asked for intuition not hard proofs:)
Hope this helps!