30
$\begingroup$

In mathematics, there are branches such as algebra, analysis, topology, etc. In machine learning there is supervised, unsupervised, and reinforcement learning. Within each of these branches, there are finer branches that further divide the methods.

I am having trouble drawing a parallel with statistics. What would be the main branches of statistics (and sub-branches)? A perfect partition is likely not possible, but anything is better than a big blank map.

Visual examples: enter image description here enter image description here

$\endgroup$
  • 6
    $\begingroup$ To offer yet another reason why this question is unanswerable (and why, perhaps the premise is misplaced): it's poorly understood that the objective of hard, theoretical science (like mathematics) is to generalize rather than to specialize. So if we were to visualize the trajectory of a successful inquiry into the field, we would not see it as though branching out to smaller, more refined branches, but rather a lens ever widening into more abstract concepts and thoughts. $\endgroup$ – AdamO Oct 8 at 16:15
  • $\begingroup$ @Rob Hyndman's answer still seems to me bang on. I am highly sceptical of any classification here. Further, this is as good a place as any to flag that a list of topics that occurs to someone falls far short of a a tree-based classification. And although dendrograms or hairball representations are mildly intriguing, what real use or interest do they serve beyond demonstrating the manifold nature of the field? $\endgroup$ – Nick Cox Oct 16 at 11:45
52
$\begingroup$

I find these classification systems extremely unhelpful and contradictory. For example:

  • neural networks is a form of supervised learning
  • Calculus is used in differential geometry
  • Probability theory can be formalized as a part of set theory

and so on. There are no unambiguous "branches" of mathematics, and nor should there be of statistics.

$\endgroup$
  • 11
    $\begingroup$ "neural networks is a form of supervised learning". That's not entirely true either, is it? I mean, one could use (and does use) NNs in supervised learning, unsupervised learning and even reinforcement learning! Well the concept of neural networks at least (it is just a huge nonlinear function that may be optimized through various optimization methods, among them SL, UL and RL). But maybe the terminology is simply used in the way you are using it, in which case.. anyone can be right. $\endgroup$ – BlueRine S Oct 7 at 12:12
  • 7
    $\begingroup$ Sure, there's no truth, but that's not really useful. Is there a model that satisfies the OP's needs? $\endgroup$ – Jay Schyler Raadt Oct 7 at 13:06
  • 3
    $\begingroup$ Rob is right. Decision trees are used in regression and AdaBoost is a classification method, but the map doesn't show this. $\endgroup$ – Zen Oct 7 at 13:34
  • 4
    $\begingroup$ I confess I don't really understand this perspective. A statistics textbook must also have the sequence of its chapters organised in some way, and its contents page reflects that organisation. The contents page's structure conveys at least some information about how the field's concepts are organised, and it does so in a much more limited way than a visualisation would allow. If nobody has a problem with the existence of textbook contents pages even though they don't capture the complexity of the field, I don't see why one would object to a visualisation like the one the OP is hoping for. $\endgroup$ – mkt - Reinstate Monica Oct 7 at 15:11
  • 4
    $\begingroup$ Textbooks are not structured hierarchically, they are structured linearly. Later in the book, links between early chapters are often developed showing that the topics introduced separately earlier are actually linked. To take an example, my own textbook on forecasting where we introduce dynamic regression models in a later chapter, linking regression and ARIMA models introduced earlier. $\endgroup$ – Rob Hyndman Oct 7 at 23:46
29
+100
$\begingroup$

This is a minor counterpoint to Rob Hyndman's answer. It started off as a comment and then grew too complex for one. If this is too far from addressing the main question, I apologise and will delete it.


Biology has been depicting hierarchical relationships since long before Darwin's first doodle (see Nick Cox's comment for a link). Most evolutionary relationships are still shown with this type of nice, clean, branching 'phylogenetic tree':

enter image description here However, we eventually realised that biology is messier than this. Occasionally there is genetic exchange (through interbreeding and other processes) between distinct species and genes present in one part of the tree 'jump' to a different part of the tree. Horizontal gene transfer moves genes around in a way that makes the simple tree depiction above inaccurate. However, we did not abandon trees, but merely created modifications to this type of visualisation:

Same as the previous figure, but with gene transfer across branches shown

This is harder to follow, but it conveys a more accurate picture of reality.

Another example:

enter image description here

However, we never introduce these more complex figures to start with, because they are hard to grasp without understanding the basic concepts. Instead, we teach the basic idea with the simple figure, and then present them with the more complex figure and the newer complications to the story.

Any 'map' of statistics would similarly be both inaccurate and a valuable teaching tool. Visualisations of the form OP suggests are very useful for students and should not be ignored just because they fail to capture reality in total. We can add more complexity to the picture once they have a basic framework in place.

$\endgroup$
  • 4
    $\begingroup$ FWIW, tree representations of the relationships between organisms long predate Darwin. I'll add a reference later. $\endgroup$ – Nick Cox Oct 7 at 10:40
  • 2
    $\begingroup$ jhupbooks.press.jhu.edu/title/trees-life is scholarly yet appealing. $\endgroup$ – Nick Cox Oct 7 at 10:55
  • 3
    $\begingroup$ Not so much a counterpoint than a supporting argument: calling into question the validity of trees. At least, with phylogeny, we use data to create such a structure, be it fossil record, gene expression, anything. Without data, we seriously ask who has the authority to choose the blocks and arrows that spread misinformation., $\endgroup$ – AdamO Oct 7 at 17:02
  • 2
    $\begingroup$ @AdamO I don't expect a single universal statistics 'map' to exist. It's perfectly reasonable for two people to use different structures and different sets of links, though one would expect the broad structure to be reasonably robust (low-level differences also occur between phylogenetic trees constructed from the same dataset, though at this point we're stretching the metaphor too far). I would say that the expertise (setting aside notions of authority for the moment) exists among the many people who have written general statistics textbooks, or even taught general statistics. $\endgroup$ – mkt - Reinstate Monica Oct 7 at 17:16
  • 2
    $\begingroup$ I liked the diagrams here enough to upvote this, but it doesn't really answer the question. $\endgroup$ – Nick Cox Oct 16 at 11:49
24
$\begingroup$

You could look into the keywords/tags of the Cross Validated website.


Branches as a network

One way to do this is to plot it as a network based on the relationships between the keywords (how often they coincide in the same post).

When you use this sql-script to get the data of the site from (data.stackexchange.com/stats/query/edit/1122036)

select Tags from Posts where PostTypeId = 1 and Score >2

Then you obtain a list of keywords for all questions with a score of 2 or higher.

You could explore that list by plotting something like the following:

relations between tags

Update: the same with color (based on eigenvectors of the relation matrix) and without the self-study tag

relations between tags

You could clean this graph up a bit further (e.g. take out the tags which do not relate to statistical concepts like software tags, in the above graph this is already done for the 'r' tag) and improve the visual representation, but I guess that this image above already shows a nice starting point.

R-code:

#the sql-script saved like an sql file
network <- read.csv("~/../Desktop/network.csv", stringsAsFactors = 0)
#it looks like this:
> network[1][1:5,]
 [1] "<r><biostatistics><bioinformatics>"                                 
 [2] "<hypothesis-testing><nonlinear-regression><regression-coefficients>"
 [3] "<aic>"                                                              
 [4] "<regression><nonparametric><kernel-smoothing>"                      
 [5] "<r><regression><experiment-design><simulation><random-generation>"  

l <- length(network[,1])
nk <- 1
keywords <- c("<r>")
M <- matrix(0,1)

for (j in 1:l) {                              # loop all lines in the text file
  s <- stringr::str_match_all(network[j,],"<.*?>")           # extract keywords
  m <- c(0)                                             
  for (is in s[[1]]) {
    if (sum(keywords == is) == 0) {           # check if there is a new keyword
      keywords <- c(keywords,is)              # add to the keywords table
      nk<-nk+1
      M <- cbind(M,rep(0,nk-1))               # expand the relation matrix with zero's
      M <- rbind(M,rep(0,nk))
    }
    m <- c(m, which(keywords == is))
    lm <- length(m)
    if (lm>2) {                               # for keywords >2 add +1 to the relations
      for (mi in m[-c(1,lm)]) {
        M[mi,m[lm]] <- M[mi,m[lm]]+1
        M[m[lm],mi] <- M[m[lm],mi]+1
      }
    }
  }
}


#getting rid of <  >
skeywords <- sub(c("<"),"",keywords)
skeywords <- sub(c(">"),"",skeywords) 


# plotting connections 

library(igraph)
library("visNetwork")

# reduces nodes and edges
Ms<-M[-1,-1]             # -1,-1 elliminates the 'r' tag which offsets the graph
Ms[which(Ms<50)] <- 0
ww <- colSums(Ms)
el <- which(ww==0)

# convert to data object for VisNetwork function
g <- graph.adjacency(Ms[-el,-el], weighted=TRUE, mode = "undirected")
data <- toVisNetworkData(g)

# adjust some plotting parameters some 
data$nodes['label'] <- skeywords[-1][-el]
data$nodes['title'] <- skeywords[-1][-el]
data$nodes['value'] <- colSums(Ms)[-el]
data$edges['width'] <- sqrt(data$edges['weight'])*1
data$nodes['font.size'] <- 20+log(ww[-el])*6
data$edges['color'] <- "#eeeeff"

#plot
visNetwork(nodes = data$nodes, edges = data$edges) %>%
visPhysics(solver = "forceAtlas2Based", stabilization = TRUE,
           forceAtlas2Based = list(nodeDistance=70, springConstant = 0.04,
                                   springLength = 50,
                                   avoidOverlap =1)
           )

Hierarchical branches

I believe that these type of network graphs above relate to some of the criticisms regarding a purely branched hierarchical structure. If you like, I guess that you could perform a hierarchical-clustering to force it into a hierarchical structure.

Below is an example of such hierarchical model. One would still need to find proper group names for the various clusters (but, I do not think that this hierarchical clustering is the good direction, so I leave it open).

hierarchical clustering

The distance measure for the clustering has been found by trial and error (making adjustments until the clusters appear nice.

#####
#####  cluster

library(cluster)

Ms<-M[-1,-1]
Ms[which(Ms<50)] <- 0
ww <- colSums(Ms)
el <- which(ww==0)

Ms<-M[-1,-1]
R <- (keycount[-1]^-1) %*% t(keycount[-1]^-1)
Ms <- log(Ms*R+0.00000001)

Mc <- Ms[-el,-el]
colnames(Mc) <- skeywords[-1][-el]

cmod <- agnes(-Mc, diss = TRUE)

plot(as.hclust(cmod), cex = 0.65, hang=-1, xlab = "", ylab ="")

Written by StackExchangeStrike

$\endgroup$
  • 2
    $\begingroup$ Maybe I will put some work into making the graphs more neat. It might be nice to have some clear graphs that map the topics on this website. $\endgroup$ – Sextus Empiricus Oct 8 at 17:22
  • 1
    $\begingroup$ This is a great approach! Nicely done. $\endgroup$ – Andrew Brēza Oct 9 at 1:30
  • $\begingroup$ From your colored graph, the three big areas are probability, regression and machine learning. $\endgroup$ – Matt F. Oct 17 at 9:24
  • $\begingroup$ @MattF. the graph is actually a bit problematic and corresponds more to the frequency of use. I tried scaling the matrix by the frequency (like switching from covariance matrix to correlation matrix), but it did not change much. The 2D graph does not show the structure very nicely and the physical model, which treats the paths as strings, places the nodes in a hexagonal/triangular shape (which is most efficient). $\endgroup$ – Sextus Empiricus Oct 17 at 9:33
  • 1
    $\begingroup$ I'd say that on stackoverflow it is five main categories: probability, regression, machine learning, but also hypothesis testing and time-series. $\endgroup$ – Sextus Empiricus Oct 17 at 9:36
9
$\begingroup$

An easy way to go about answering your question is to look up the common classification tables. For instance, 2010 Mathematics Subject Classification is used by some publications to classify papers. These are relevant because that's how a lot of authors classify their own papers. enter image description here

There are many examples of similar classifications, e.g. arxiv's classification or Russian education ministry's UDK (universal decimal classifictaion) which is used widely for all publications and research.

enter image description here

Another example is JEL Claasification System of American Economic Association. Rob Hyndman's paper "Automatic time series forecasting: the forecast package for R." It's classified as C53,C22,C52 according to JEL. Hyndman has a point though in criticizing the tree classifications. A better approach could be tagging, e.g. the keywords in his paper are: "ARIMA models, automatic forecasting, exponential smoothing, prediction intervals, state space models, time series, R." One could argue that these are better way to classify the papers, as they're not hierarchical and multiple hierarchies could be built.

@whuber made a good point that some latest advances such as machine learning will not be under statistics in current classifications. For instance, take a look at the paper "Deep Learning: An Introduction for Applied Mathematicians" by Catherine F. Higham, Desmond J. Higham. They classified their paper under aforementioned MSC as 97R40, 68T01, 65K10, 62M45. these are under computer science, math education and numerical analysis in addition to stats

$\endgroup$
  • 3
    $\begingroup$ I think it would be more accurate to say this is how a lot of authors are asked to classify their papers. I know I am never quite satisfied when asked to employ such a priori categories to my work. $\endgroup$ – Alexis Oct 8 at 16:25
  • 6
    $\begingroup$ This is a good basis to identify the branches of mathematical statistics. Knowing that helps us identify what has been left out, which includes many parts of machine learning. Indeed, it may be fair to characterize the 2010 math subject classification as describing "statistics as of 1950" and then throw in everything that emerged later, such as geostatistics, genomics, bootstrapping, and so on (some of which may fall under those old categories, perhaps). $\endgroup$ – whuber Oct 8 at 17:02
4
$\begingroup$

One way to approach the problem is look at citation and co-authorship networks in statistics journals, such as the Annals of Statistics, Biometrika, JASA, and JRSS-B. This was done by:

Ji, P., & Jin, J. (2016). Coauthorship and citation networks for statisticians. The Annals of Applied Statistics, 10(4), 1779-1812.

They identified communities of statisticians and used their domain understanding to label the communities as:

  • High-Dimensional Data Analysis (HDDA-Coau-A)
  • Theoretical Machine Learning
  • Dimension Reduction
  • Johns Hopkins
  • Duke
  • Stanford
  • Quantile Regression
  • Experimental Design
  • Objective Bayes
  • Biostatistics
  • High-Dimensional Data Analysis (HDDA-Coau-B)
  • Large-Scale Multiple Testing
  • Variable Selection
  • Spatial & Semi-parametric/Non-parametric Statistics

The paper includes a detailed discussion of the communities along with decompositions of the bigger ones into further subcommunities.

This may not entirely answer the question, since it's concerning the fields of researching statisticians rather than all fields, including ones which are no longer active. Hopefully it nonetheless is helpful. Of course, there's other caveats (such as only considering these four journals) which are discussed further in the paper.

$\endgroup$
  • 2
    $\begingroup$ I was thinking about doing this for this website. Defining "co-authorship" as people that respond/answer to the same questions. $\endgroup$ – Sextus Empiricus Oct 10 at 9:22
  • $\begingroup$ @MartijnWeterings Yeah, your answer seems to be in a very similar direction as this approach! $\endgroup$ – user257566 Oct 10 at 20:07
2
$\begingroup$

I see many amazing answers, and I don't know how an humble self made classification may be received, but I don't know any all-comprensive book of all statistics to show the summary of, and I do think that, as @mkt brillantly commented, a classification of a study field can be useful. So, here is my shot:

  • descriptive statistics
    • simple inference
      • simple hypothesis testing
    • plotting/data visualization
  • sampling design
    • experimental design
    • survey design
  • multivariate statistics (unsipervised)
    • clustering
    • component analysis
    • latent variables models
  • linear models (which are actually multivariate as well)
    • ordinary least squares
    • generalized linear models
      • logit model
    • other linear models
      • Cox model
      • quantile regression
    • multivariate inference
      • multiple hypothesis testing
      • adjusted hypothesis testing
    • models for structured data
      • mixed effects models
      • spacial models
      • time series models
    • non linear extensions
      • generalized additive models
  • bayesian statistics (actually bayesian methods exist for many things I already listed)
  • non parametric regression and classification
    • many machine learning methods fit here

Of course this is over-simplicistic, it is only meant to get some idea straight to someone who barely knows the field, each of us here surely knows that there are a lot of methods in between the categories up here, many others I didn't list because they are less famous or because I simply forgot. Hope you like it.

$\endgroup$
1
$\begingroup$

One way to organize this information is to find a good book and look at the table of contents. This is a paradox because you specifically asked about statistics, whereas most introductory graduate level texts on the topic are for statistics and probability theory together. A book I am reading on regression now has the following TOC:

  • Frequentist Inference
  • Bayesian Inference
  • Hypothesis Testing and Variable Selection
  • Linear Models
  • General Regression Models
  • Binary Data Models

  • General Regression Models

  • Preliminaries for Nonparametric Regression [a precursor to...]
  • Spline and Kernel Methods
  • Nonparametric Regression with Multiple Predictors

(The remaining sections are supporting mathematics and probability theory)

  • Differentiation of Matrix Expressions
  • Matrix Results
  • Some Linear Algebra
  • Probability Distributions and Generating Functions
  • Functions of Normal Random Variables
  • Some Results from Classical Statistics
  • Basic Large Sample Theory
$\endgroup$
  • 2
    $\begingroup$ One might consider such a book to convey a part of one branch of a discipline. Unless it purports to be an encyclopedic survey of all of statistics, though, its chapter headings could scarcely be considered major branches of the field! $\endgroup$ – whuber Oct 8 at 15:18
  • 3
    $\begingroup$ @whuber agreed. I was careful to mention the book was on regression, and that I don't consider any book on the topic of "statistics" to be general-enough or at the appropriate level for a statistician to consider the topics laid out as being sufficient. This specific example is from Wakefield's text and is a remarkably general treatment (the T-test with unequal variance estimation is discussed in the context of linear regression with binary covariate and robust error estimation, for instance). $\endgroup$ – AdamO Oct 8 at 16:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.