What, if anything, is wrong with the SE welcome wagon blog post's statistical analysis? It's been a while since I've done any major statistics, hence I have to base this question more on a gut feeling than a thorough analysis.
It's about this stack overflow blog post https://stackoverflow.blog/2018/12/04/welcome-wagon-community-and-comments-on-stack-overflow/
They are drawing a lot of conclusions from this study. However, the group selection seems heavily biased. The main conclusion that people more invested in SE find comments to be rude easier is based on the difference between employees and others, but to me, especially the difference in the not-employees group could easily be random noise. I see no test for significance anywhere.
From a purely academic perspective - is this good statistics, or is it not? 
 A: This is a great question, and it is nice to see the analysis conducted by StackExchange subjected to the rigour of its own contributing experts!  Nevertheless, it is a bit difficult to assess a blog post "from an academic perspective" since the level of rigour in blog posts differs substantially from those of published scholarly work in academia.
As a general opinion, I will first note that the analysis presented in the blog post is less rigorous than what I would expect in an academic paper, and there are many aspects of the study and analysis that are not specified.  That is not necessarily unreasonable, since a blog post is generally not aiming at the level of detail and rigour of an academic paper on the subject.  Blog posts are usually targeted to a lay audience that cannot be assumed to have any statistical knowledge or training, so it is not necessarily unreasonable that the kinds of information in a detailed statistical analysis would be stripped away for brevity.  The data they have collected looks to me to be a large and impressive project, so my overall impressions are favourable.
Another important thing to note is that it is important to distinguish exploratory data analysis and confirmatory data analysis.  This project appears to be in the former category, since it does not seek to test a pre-existing hypothesis.  In view of this, one must be careful with presenting formal statistical tests for hypotheses.  In particular, if hypotheses are formed by looking at the exploratory data then formal tests of these hypotheses using that same data will be biased towards acceptance of the hypotheses.  In EDA it is generally best to avoid formal hypothesis tests, except for general tests of differences that were reasonable to ask for prior to seeing the data.
Having said this, it would certainly be nice if the authors were able to back up this blog post with a more detailed exposition of the mechanics of their study and their modelling (e.g., in a published paper, or even just a more detailed post aimed at statisticians).  To get this up to a level that would be convincing to a statistician they would need to gave a bit more detail in some areas, particularly in regard to their sampling mechanisms and their modelling.  It would be even nicer if they shared the underlying data ---suitably anonymised--- so that others can model the data!  In terms of the drawbacks of the present post, I see a few main issues that would need to be tightened up to make this a more rigorous presentation.

Unclear sampling mechanisms: The mechanism by which comments were sampled and presented to the participants is not clear from the post.  Were all comments in a given time period presented for review, or was it just a sample?  If the latter, was it a randomised sample, or was it chosen in the discretion of the study researchers?
Unclear demarcation of group effect vs question effect: The metrics shown in the blog post are all measures per user, that do not have any adjustment to account for the comments rated by that user.  It is unclear whether there are systematic differences in the types of comments rated by different users, and if this can be filtered out via statistical methods.  We are told that the overall group differences are "...robust to comparing groups who were shown the same comments, who rated the same number of comments, and other analytical approaches."  Nevertheless, this latter analysis is not shown or described.
Measures of uncertainty: The post shows the median ratings of each group, and the graphs give a visual presentation of spread in the median ratings of each participant.  Nevertheless, aside from those presented graphically, there are no statistical metrics (e.g., confidence intervals, etc.) that estimate the "true" group means/medians.  There is also no presentation of the relative size of the variance in ratings for an individual participant, versus the spread in median ratings for different participants.
Hypothesis testing: The omission of formal testing in this analysis is probably a positive, since it appears to be an exploratory study rather than a confirmatory analysis of a pre-existing hypothesis.  From their observed data the authors state that "[o]ur project showed that the more deeply an individual is connected to Stack Overflow (as an employee, or a moderator), the more they are likely to see problems in comments like these."  For reasons elaborated at length by Tukey in his analysis of EDA and CDA (see e.g., Behrens 1997), I think this "conclusion" should actually be treated as an exploratory hypothesis formed by observation of the data, and so it should be tested with new data in a later study.

As a final comment, I would like to reiterate that the above issues are things that are drawbacks only from the point of view of trying to present a more rigorous analysis - in terms of a blog post for a lay audience, their omission may be perfectly reasonable.  The authors of the blog post are both data scientists with some background in statistical analysis, so I'm sure they're aware of these issues, and would deal with them accordingly if they decide to write their blog post up into a more rigorous paper.
