Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Does frequentist statistics take a stand on whether the universe / world (or at least the processes that are being modeled) is deterministic or stochastic?
If so, where in the methodology does that matter?
For a related question about the Bayesian point of view, see "Deterministic or stochastic universe in Bayesian statistics?".
I respectfully disagree with the answer by Frans; there is nothing in the frequentist methodology that takes any position on whether data-generating processes, as modeled by statistics, are deterministic or not. (While Wikipedia is a useful source for some statistical material, this unreferenced sentence does not hold any weight for me.) Frequentism defines the "probability" of an event in the context of a repeatable sequence of trials as the limiting relative frequency of that event over a sequence of trials.$^\dagger$ Hence, within this framework, the application of probabilistic models implies only that the user is satisfied that the event can be placed in a sequence of theoretically repeatable trials. The occurrence of events in the sequence, and the existence of a limiting relative frequency, are not affected by whether the process is deterministic or not.
In view of this, within the frequentist paradigm, one should not imbue references to "probability" or "stochastic" with any non-deterministic implications (in a metaphysical sense). Within this paradigm the notion of probability refers merely to limiting relative frequencies of events, and a "stochastic" model is just one that is described with the use of probability (i.e., with the appeal to limiting relative frequencies of events). A frequentist statistical model is only "non-deterministic" in a mathematical sense ---i.e., that the specification of the parameters does not logically imply the outcome of the individual random variable. (Or to put it another way, the limiting relative frequency of an event does not logically imply the occurrence or non-occurrence of the event at any particular point in the sequence.)
One could believe in randomness in the universe, or determinism, and apply frequentist methods and interpretations. Under this paradigm the observable values are considered to be results from a repeatable experiment and thus they are considered to be contained within a (hypothetical) infinite sequence, with limiting empirical distribution described by one or more "parameters". These latter objects are treated as "unknown constants" even if the user adopts a non-deterministic aleatory view which holds that the parameter is non-deterministic.
$^\dagger$ My view is that this frequentist definition is problematic, since it takes the concept of a repeatable experiment to be preliminary to probability, and it therefore has trouble explaining the conditions that constitute repeatability (since it cannot appeal to any probabilistic condition). This notion is actually well-described within Bayesian theory by the concept of an exchangeable sequence of values, where the condition of exchangeability corresponds to repeatability. Within the Bayesian framework the representation theorem of de Finetti then establishes that the probability corresponds to the limiting relative frequency as a mathematical consequence of exchangeability, rather than as a definition.
Interesting question!
I would say from a statistical modelling perspective, all data are assumed to come from a combination of systematic components and a stochastic component, which means that data-generating processes as modeled by statistics, are assumed to be non-deterministic in nature. Wikipedia even states that:
A statistical model is a special class of mathematical model. What distinguishes a statistical model from other mathematical models is that a statistical model is non-deterministic.
I will also give you my perspective as a former biologist, where we usually explain that statistical modelling (at least frequentist) is something like the following:
Which ironically you could interpret as the stochastic part being only 'apparently stochastic', since it relies on things that we did not or cannot measure.$^*$ Even 'random mutations' are eventually caused by a large sum of things we cannot measure, such as exposure to sunlight, failure of repair mechanisms, etc., but for all practical intents and purposes, it might as well be non-deterministic.
However, even if we could measure all those things perfectly, all down to the molecular level, we would have to concede that effects on a molecular scale are in turn influenced by things on a quantum scale, and... well,
there are limits to the precision with which quantities can be measured (uncertainty principle).
We run into the uncertainty principle, which would imply that in the end, the processes we model with statistics are random (if my understanding of it is correct).
That being said, a model is just that, a model. And I don't think statistics as a field takes a point of view on the nature of the universe. After all, that is not our field of research. At best you could argue that you implicitly assume the data-generating process is random by making use of statistical models.
$^*$ In other words, the sum of a large number of unobserved, independent effects create an (apparently) random deviation from the systematic effects. (In case of a large sum of uniformly distributed effects this nicely explains why we often assume normality, but I'm getting off track.)
Where in the methodology does it matter?
Well, basically for everything! A model without an stochastic part is not considered a statistical model. This even applies to those who do not consider machine learning and statistics to be the same. Even from the point of stochastic optimization (the name gives it away a bit), a model cannot be trained further from new data if the loss function is already exactly zero.
