A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [philosophical]

For questions about PHILOSOPHY of statistics or probability: interpretations of probability, foundational issues with frequentist/Bayesian statistics, etc. Do not use this tag for generally speculative (aka "philosophical") questions.

Filter by
Sorted by
Tagged with
2
votes
0answers
28 views

What's the rationale behind a normality test followed by a $t$-test?

Correct me if I'm wrong, but from my understanding, the standard procedure of testing whether data from an unknown source have a specific mean is to (a) perform a normality test to see if the data are ...
2
votes
1answer
42 views
+50

Calibration with or without temporal structure

Setup I have a sensor which measures some air property outside. It outputs values on a continuous range. The sensor is expensive, so I want to replace it with multiple cheap sensors, which may give ...
3
votes
1answer
50 views

Two meanings of entropy?

In the world of physics, entropy seems to mean something different from stats + information theory world. So, I assumed that there are two definitions to the word entropy. Indeed, I looked up how ...
17
votes
5answers
961 views

Is it appropriate to use “time” as a causal variable in a DAG?

This question might be better suited for philosophy.SE, but I will post it here in the first instance, since it involves technical aspects that are best understood by users on this site. The title ...
0
votes
0answers
35 views

When computing MLE for linear regression, where does the uncertainty come from?

In my Machine Learning course, when computing MLE for a linear regression problem, we modeled the likelihood function as a Gaussian. I have trouble understanding why. Where does the uncertainty come ...
3
votes
0answers
78 views

Constructive mathematics and statistics [closed]

Usual non-constructive mathematics leads to some paradoxes (e.g. the Banach-Tarski paradox), which are directly related to the axiom of choice. In non-constructive mathematics, the axiom of choice (as ...
14
votes
3answers
2k views

What does Fisher mean by this quote?

I keep seeing this famous quote everywhere, but fail to understand the emphasized part every single time. A man who ‘rejects’ a hypothesis provisionally, as a matter of habitual practice, when ...
7
votes
3answers
123 views

Why don't people trade significance level for power?

As a convention, we have a lot of studies whose significance level is $0.05$ and a power of $0.8$. However, it is extremely rare to find a study whose $\alpha = 0.2$ with a power of $0.95$. From my ...
26
votes
5answers
5k views

When does Fisher's “go get more data” approach make sense?

Quoting gung's great answer Allegedly, a researcher once approached Fisher with 'non-significant' results, asking him what he should do, and Fisher said, 'go get more data'. From a Neyman-Pearson ...
3
votes
0answers
86 views

Do probabilities exist in reality? [closed]

I came across the seemingly interesting work of William Briggs in relation to probability and statistics. He make some bold claims, like the following: Probability does not exist as a thing, as a ...
16
votes
4answers
627 views

On George Box, Galit Shmueli and the scientific method?

(This question might seem like it is better suited for the Philosophy SE. I am hoping that statisticians can clarify my misconceptions about Box's and Shmueli's statements, hence I am posting it here)....
3
votes
0answers
53 views

What are the main approaches to the foundation of statistics without probability

The frequentist, likelihood and, to an even greater extent, Bayesian approaches to statistics are all based on probability. Without probability, it seems difficult to use a data sample ("seen" cases), ...
0
votes
0answers
15 views

Verify that data have property $X$ with hypothesis testing

In data analysis, one usually need to verify that data have property $X$ before applying method $Y$, which takes $X$ as a prerequisite. To illustrate, possible values of $(X, Y)$ include $(\text{...
1
vote
2answers
25 views

How is probability distribution type decided before applying maximum likelihood estimation?

What I can't grasp for a long time is how is the assumption about the probability distribution type made? Why do we assume that the human height is distributed exactly normally? Why not any other ...
7
votes
3answers
151 views

What is the $p$ in Bernoulli distribution?

In the Bayesian theory of probability, probability is our expression of knowledge about a certain thing, not a property of that thing. However, I always see people treat $p$ as a parameter that needs ...
0
votes
1answer
59 views

What would be an ignorance prior of AB, given the probabilities of A and B?

Let us have two events, $A$ and $B$ whose probabilities are $P(A)$ and $P(B)$. In the absence of any other information, what would be a reasonable probability to assign to $AB$, that is, $A$ and $B$ ...
0
votes
1answer
62 views

Iterating Bayes rule over time

$\require{cancel}$ In a online bayesian inference procedure one is iteratively changing the prior with a new posterior, calculated given a new set of observations. Does it mean we capture time ...
4
votes
0answers
38 views

Best textbook on philosophical background of interpretations (for beginners)

I'm looking for the best textbook teaches basics of statistics for a general researcher (undergraduate) without totally ignoring the mathematical proofs and theories while the main emphasizes is on ...
20
votes
5answers
696 views

An example where the likelihood principle *really* matters?

Is there an example where two different defensible tests with proportional likelihoods would lead one to markedly different (and equally defensible) inferences, for instance, where the p-values are ...
0
votes
1answer
43 views

Frequentist definition of probability and prediction?

The frequentist definition of probability states that: The probability of an event is the ratio of the number of cases favorable to it, to the number of all cases possible when nothing leads us to ...
2
votes
1answer
47 views

Can every parameter $\Theta$ in Bayesian modelling be explained via De Finetti`s representation theorem

My question is the following: I recently got to know (and love) De Finetti`s representation theorem and I now started to read a Book an Bayesian statistics. However this book simply takes as the ...
3
votes
0answers
50 views

Overview articles on Bayesian philosophy and methodology

While we have questions about Bayesian textbooks (1 and several other) and the philosophy behind the Bayesian thinking (2), I am interested papers (or sources of similar length, e.g. blog posts or ...
3
votes
2answers
98 views

Deterministic or stochastic universe in frequentist statistics

Does frequentist statistics take a stand on whether the universe (or at least the processes that are being modeled) is deterministic or stochastic? If so, where in the methodology does that matter?
2
votes
1answer
57 views

Probability based on Observed Data

I'm trying to figure if my reasoning and application of probability is correct in this made up example using R. Example data: ...
2
votes
1answer
149 views

Covariance versus correlation: which is a “deeper” or more “structural” property of the data?

It might seem obvious that the covariance is a "deeper" property of the data generation process (DGP), since normally the specification of a joint distribution is done in terms of its mean vector and ...
1
vote
0answers
78 views

What can we infer about an individual from a distribution over the population?

This is a somewhat philosophical question, or maybe it has to do with the interpretation of statistics. The question is: if we have a distribution of some property of a large population, but then look ...
1
vote
1answer
498 views

How can (L1 / L2) regularization be equivalent to using a prior when priors can't be changed?

I understand the argument for how training with an L1/L2 regularizer is the same thing as finding the MAP estimate when the prior is Gaussian/Laplace. But there's a crucial difference. In Bayes' ...
5
votes
3answers
165 views

Intuition about the deep meaning of Bayesian priors and its influence on posteriors

In estimating posterior distributions, Bayesians rely on the idea of the prior distribution. In many examples, I see this being set fairly arbitrarily, ie ~N(0,1). It's clear that the posterior is ...
0
votes
0answers
19 views

Can we get uniform distributions on infinite spaces by giving up infinite additivity

I am wondering whether it is possible to translate the idea of drawing a number randomly from the set of all natural numbers. If we have infinite additivity as an axiom this obviously does not work. ...
2
votes
1answer
43 views

Assigning percentages to different causes

There was recently a celebrated spat between Cathy Newman, one of the main presenters of Channel 4 News in the UK, and Jordan Peterson, a well-known academic and clinical psychologist, on the causes ...
8
votes
3answers
246 views

How does a Bayesian update his belief when something with probability 0 happened?

Define $X:=$ "coin has probability 1 to land heads" Assume that one has the prior belief: $P(X)= 1$. However after tossing the coin once it lands tails ($E:= $ "coin landed tails"). How should a ...
4
votes
0answers
79 views

Is it a problem that limiting frequencies (can) violate countable additivity?

I`ve stumbled upon the following paper by Alán Hajek https://www.jstor.org/stable/40267419, in which the author states that the Frequentist interpretation of probabilities as limiting frequencies ...
16
votes
4answers
1k views

How do Bayesian Statistics handle the absence of priors?

This question was inspired by two recent interactions I had, one here in CV, the other over at economics.se. There, I had posted an answer to the well-known "Envelope Paradox" (mind you, not as the "...
4
votes
1answer
400 views

What is the difference between classical frequentist methods and likelihood methods?

You may assume that I'm familiar with the material in Casella and Berger. This question is identical to What is the difference between Fisherian vs frequentist statistics?; however, the question was ...
3
votes
2answers
310 views

Is there a reason other than conventions why a CDF must be defined for all real numbers

There are many cases when the sample space is not the entire set of real numbers (for instance a Bernoulli trial or sampling from an interval). On the one hand: for the definition $F_X(x) = P(X \leq ...
3
votes
0answers
347 views

What is the difference between Fisherian vs frequentist statistics? [closed]

I just read a research paper that said implicitly that there was a difference between the two. I thought that Fisherian statistics was another word for frequentist statistics.
1
vote
2answers
57 views

What are some basic principles to handle “none of the above” events in Bayesian statistics?

Suppose you want to assign a noninformative prior to the following event: The next tree that we will encounter is a: Spruce Pine None of the above We don't have any prior information, so we are ...
3
votes
1answer
355 views

Isn't the Solomonoff Universal Prior Biased Like Any Other?

The Solomonoff universal prior is fixed relative to a specific choice of universal Turing machine (UTM). Now, I understand that a UTM can simulate any other UTM, so that they assign a complexity to ...
3
votes
1answer
73 views

Is a frequentist approach to inference appropriate when working with non-repeatable data?

Jackman (2009) writes on p.xxxi-xxxii: Consider researchers analyzing cross-national data in economics, political science, or sociology, say, using national accounts data from the OECD. [...] ...
1
vote
0answers
90 views

Applying machine learning to dynamic complex systems (e.g. weather prediction)

Would it be correct to say that: Physics-based, domain specific models are more widely used and are more practical in (longer term) weather forecasts than pure machine learning approaches The reason ...
7
votes
2answers
187 views

Is testing model assumptions considered p-hacking/fishing?

"P-hacking", "fishing", and "garden of forking paths" as explained here and here describes an exploratory data analysis-like style of doing research that produces biased estimates. Does testing model ...
2
votes
0answers
43 views

Does it ever make sense to talk about the confidence (faith) in a probability value?

Let us suppose that we want to know what weather will be tomorrow. We ask two meteorologists and both give us an identical probabilistic answer: It will rain with probability of 30% It will snow ...
0
votes
1answer
71 views

What are the problems with existence of an “Omniscience Machine” within the constraints of collective human knowledge? [closed]

"If a machine can read and process written texts, manuscripts, journal articles and books, it would be able to predict results of new scientific experiments and construct new scientific theories." ...
1
vote
0answers
47 views

Why can scientists that refuse to bound the prior probability declare discoveries?

Summary: There appears to be scientists that refuse to put prior probabilities on some statements, such as the existence of the Higgs Boson. This is an understandable position. These scientists, ...
20
votes
4answers
2k views

Is everyday probability just a way of dealing with the unknown (not talking quantum physics here)?

It seems like in everyday probability (not quantum physics), probabilities are really just a substitute for an unknown. Take a coin flip for example. We say it's "random," a 50% change of head and a ...
7
votes
3answers
374 views

Is there an example of two causally dependent events being logically (probabilistically) independent?

Two events $A,B$ are independent when $P(A \cap B ) = P(A)P(B)$ I am trying to drill into this definition and to try to reconcile it with our intuitive idea of independence in the real world. I feel ...
9
votes
1answer
174 views

P-values and likelihood principle

This question came up in class: If we use p-values to evaluate hypotheses on an experiment, which part of the Likelihood Principle are we not obeying: Sufficiency or Conditionality? My intuition ...
2
votes
1answer
128 views

How important are interpretations of probability to the practice of statistics?

I know that the frequentist interpretation of probability is associated with classical statistics and maximum likelihood estimation, and that a subjective interpretation of probability is considered ...
70
votes
11answers
8k views

Why should I be Bayesian when my model is wrong?

Edits: I have added a simple example: inference of the mean of the $X_i$. I have also slightly clarified why the credible intervals not matching confidence intervals is bad. I, a fairly devout ...
3
votes
1answer
139 views

HMM: Why are observations conditional on the latent state and not vice versa?

The model of a HMM consists of a latent Markov chain with state $X$ and transition probabilities $P(X^t \mid X^{t-1})$, and observation variables $Y$ that depend on the current latent state via $P(Y^t ...