Questions tagged [philosophical]
For questions about PHILOSOPHY of statistics or probability: interpretations of probability, foundational issues with frequentist/Bayesian statistics, etc. Do not use this tag for generally speculative (aka "philosophical") questions.
191 questions
2
votes
1
answer
90
views
Birnbaum's Theorem: Strong belief in a model $\implies$ the likelihood function must be used as a data reduction device?
Working through understanding section 6.3.2 (pg. 292-294) in Casella and Berger's Statistical Inference (2nd-ed).
The following definitions and principles are given:
Definition (Experiment): An ...
0
votes
0
answers
49
views
Help with completing a derivation of usefulness of cross-validation
This question is raised as a result of my attempt to answer this other question of mine.
Let's refer to all our prior knowledge, both explicit and implicit, as $X_\text{true}$. Almost always, we are ...
- 187
2
votes
0
answers
70
views
Can an outcome variable be used twice in the same model?
When is it appropriate to use the same outcome variable in two likelihoods in the same model framework?
Here is a specific example:
...
6
votes
3
answers
426
views
The rationale for when significance or null hypothesis testing is needed
Why do people sometimes claim that an effect is so huge and "obvious" that it does not warrant any inferential statistics calculation, even though the sample size is not large?
This is ...
- 175
2
votes
0
answers
38
views
What is the role of determinism in stochastics?
What is the role of determinism in stochastics?
In my opinion, determinism, much like the continuum, is merely a concept to describe an ideal world. In this idealized world we can calculate and argue. ...
- 21
11
votes
7
answers
2k
views
Are “Data are fixed” in Bayesian viewpoint and “Data are random” in frequentist viewpoint talking about the same thing mathematically?
In my opinion, in BOTH Bayesian and Frequentist inferences, observational data $x$ are modelled as the observed value of a random variable $X$ which follows a certain probability distribution. ...
- 475
10
votes
5
answers
2k
views
How would a Bayesian define a fair coin?
In the frequentist worldview, probabilities are long-run relative frequencies. Hence, a fair coin can be defined as a coin, for which the long run relative frequency of each of the sides approaches 0....
- 777
13
votes
7
answers
6k
views
To a Bayesian, does a trick coin with two heads have 50% chance of flipping heads if they don't know that it has two heads?
And to expand on this idea, would uncertainty and complexity be the same thing to a Bayesian? For example, games with much randomness like poker and games that are more complexity based like chess -- ...
- 133
1
vote
1
answer
88
views
Why not conflating a distribution over distributions with a distribution?
This question is more like a philosophical one. I constantly find the notion of a distribution over distributions and redundant. Why don't we conflate that with the notion of a single distribution (by ...
- 235
15
votes
4
answers
1k
views
Do testability and falsifiability have statistical definitions?
Psychology: the Core Concepts says
Psychology differs from the pseudosciences in that it employs
the scientific method to test its ideas empirically. The scientific
method relies on ...
- 19.8k
8
votes
4
answers
419
views
iid data (Bayesian) vs iid random variables (Frequentist)?
I've been pondering the differences in notation / language used in some of the resources I've read for statistics / machine learning.
Warning: this might be embarrassingly obvious to any decent ...
- 435
1
vote
0
answers
39
views
Comparing scores on different dimensions in factor models
So I'm currently conducting some research where I'm using item response theory (IRT) to estimate the difficulty of school subjects in Norway. It turns out that a two-dimensional, simple-structure ...
- 11
9
votes
4
answers
791
views
What are some good books on the philosophy of statistics?
I am a PhD in biological sciences with some background in graduate-level probability. I am interested in questions like what does it mean for an event to have probability $x$ at a philosophical level.
...
1
vote
1
answer
487
views
What is the Statistical Method?
I apologize in advance for the possibly philosophical nature of the question, however I would like to get answers from this website rather than from Philosophy exchange at the moment. I also come from ...
- 267
1
vote
1
answer
96
views
How is Bayes Stats supposedly more "intuitive" when it requires us to think probabilistically which IS NOT intuitive for most folks? [closed]
People don’t naturally think in probabilistic ways, they often form priors through point estimates because it’s less cognitively taxing to reduce everything down to single numbers. So if the ...
- 11
2
votes
1
answer
35
views
Machine learning and empiricism: does prediction translate to evidence?
Let $X_1, ..., X_n$ be a series of idd random variables. Imagine we are training a machine learning classifier over these variables with the task of predicting a feature $F$. A typical scenario might ...
- 235
2
votes
1
answer
81
views
Arbitrariness in statistical tests
I am new to statistics and find the following procedure unnatural
and arbitrary. Could someone point out what I miss, and how I
should be thinking?
Assumptions
I have two vectors of real numbers $x_{i}...
- 235
0
votes
0
answers
41
views
Understanding Bayes' Theorem, or: Are all pluviophobes hermits?
Let $A$ be the event "I go out", $B$ be the event "It rains". Then Bayes' Theorem tells us that $$P[A|B] = \frac{P[B|A]P[A]}{P[B]}.$$
I think the weather doesn't care what I do, so ...
- 599
1
vote
1
answer
124
views
Bias of an estimator depends on whether you take expectation of the estimator or its inverse
(Please read until the end)
Consider two ways of writing the exponential distribution:
(A) $\frac{1}{\beta} e^{-\frac{x}{\beta}}$ and
(B) $\theta e^{-x\theta}$
If I estimate $\beta$ or $\theta$ ...
- 11
4
votes
0
answers
116
views
A misspecification error with linear models that can complete reverse the direction of an effect, has this been described, has this a name?
Linear models are ubiquitous in economic, social, health and nutritional sciences and the starting point for much research and many articles.
However, there is a problem with linear models. When the ...
- 86.4k
6
votes
2
answers
366
views
Frequentist inference with a null hypothesis that reflects theory a good-enough belt around it
TL;DR:
With frequentist statistics, does it make sense to 1) no longer use significance testing, 2) set the point null hypothesis to reflect theory and decide a priori when to refute it, and 3) use a ...
- 319
20
votes
3
answers
2k
views
Within the frequentist "school of thought" how are beliefs updated?
Background
Edit: I realize my use of the word "hypothesis" is confusing, I do not mean specifically a null hypothesis. I mean a proposition that something is true.
From my limited ...
- 387
2
votes
1
answer
53
views
How could one get a difference between expected and observed probabilities with rare events?
Say we have a car with an electronic ignition system. Our engineers have deemed that due to mechanical failure possibilities, there is a 1 in a billion (or some huge number) chance that the car will ...
- 23
12
votes
4
answers
2k
views
Is anything inherently random?
Is anything inherently random? Or is all randomness observed in data either "errors in measurement" or "lack of understanding"? Assume we could measure everything with infinite ...
- 4,792
18
votes
5
answers
2k
views
Do all observations arise from probability distributions?
Below is the quote from Karl Pearson in the book: “The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century” by David Salsburg:
Over a hundred years ago, Karl Pearson ...
11
votes
4
answers
1k
views
Bayesian analysis used merely as a computational tool?
I have sometimes seen some statisticians used bayesian analysis and related techniques such as MCMC simply as a tool when a frequentist approach is not satisfying, typically for example when the ...
- 397
21
votes
3
answers
3k
views
Does the rejection of the null hypothesis have anything to do with Popper's theory of falsification?
According to Popper, we cannot verify a hypothesis due to the problem of induction - we can only aim to falsify it. If we are repeatedly unable to falsify it, the hypothesis is said to be tentatively ...
- 311
1
vote
0
answers
36
views
Understanding Countability in Sample Spaces [duplicate]
In [Casella, Berger] Statistical Inference there is a short discussion on countability of sample spaces and its implications:
This distinction between countable and uncountable sample spaces is ...
1
vote
0
answers
66
views
Is Probability just Math? [closed]
Is Probability just a "branch of mathematics" as wikipedia suggests or is it something larger than that?
More like we use math for real world problems like engineering, medicine, meteorology ...
- 123
1
vote
0
answers
210
views
Deterministic or stochastic universe in Bayesian statistics?
Dave Harris says the following in "Knightian uncertainty versus Black Swan event":
In Bayesian thinking, chance doesn't really exist. What does exist is a system that is too complicated and ...
- 69.5k
4
votes
1
answer
3k
views
What is meant by divergence in statistics?
I have learned about the Intuition on the Kullback-Leibler (KL) Divergence as how much a model distribution function differs from the theoretical/true distribution of the data.
The two most important ...
- 3,005
11
votes
5
answers
982
views
How do you know something isn't random?
Suppose I made a random number generator that's supposed to return a number 1-10, but I made it always return 4, and didn't tell you.
How would you know with 100% certainty it wasn't random?
Even if ...
5
votes
2
answers
794
views
What is Cromwell's rule and why is it important for Bayesians?
I have just heard of Cromwell's rule, but I'm not sure I understand it very well. What is Cromwell's rule and why is it important for Bayesian statistics?
47
votes
6
answers
5k
views
How seriously should I think about the different philosophies of statistics?
I've just finished a module where we covered the different approaches to statistical problems – mainly Bayesian vs frequentist. The lecturer also announced that she is a frequentist. We covered some ...
- 423
5
votes
1
answer
66
views
Does using a probabilistic model for a real-world event make it harder to identify its causes?
I recently read this odd critique of statistics (the author calls it a critique of probability theory, but I think he doesn't understand the difference probability theory and statistics).
http://...
- 18.4k
8
votes
3
answers
240
views
How to answer critiques about the inapplicability of the framework of frequentist statistics to the real world?
I often hear the argument that frequentist stats is useless or contorted because no event is precisely repeatable, let alone repeatable infinitely many times, and because there are no iid sequences in ...
- 18.4k
2
votes
2
answers
124
views
Why is a 100 heads run surprising? [closed]
Assume we have a fair coin. We flip it 100 times. The outcome is all heads.
Why is it that all heads outcome is more surprising to us than a "more random looking" outcome with less ...
- 121
2
votes
2
answers
964
views
Bayesian Probability of Zero?
I've been reading a few different philosophical papers/books which have mentioned a "Bayesian belief". Within these texts I've been basically inferring that within the Bayesian theorem, ...
- 21
2
votes
1
answer
49
views
Should a feature importance score be invariant to transformations of the response?
This is more of a philosophical question that came up in a discussion with a friend - consider some 'feature importance' procedure associated to a model (say a regression model). You run your model ...
- 1,531
2
votes
1
answer
2k
views
Why can't we say that the probability of the true parameter being within a 90% confidence interval is 90%? [duplicate]
I've been reading a bit about the confidence intervals on Wikipedia. The section on misunderstandings says:
A 95% confidence level does not mean that for a given realized
interval there is a 95% ...
1
vote
0
answers
51
views
How to deal with different opinions in statistics and data analysis? [closed]
As I see it, there are a gap between theoretical work in statistics and real-world data analysis; and differences in opinions among applied statisticians with regards to their approaches to data ...
- 133
9
votes
4
answers
3k
views
The explosive AR(1) process with $\varphi>1$, where was this first represented as a stationary, but non-causal, time-series?
According to this question and answer Explosive AR(MA) processes are stationary? the AR(1) process (with $e_t$ white noise):
$$X_{t}=\varphi X_{t-1}+e_{t} \qquad , e_t \sim WN(0,\sigma)$$
is a ...
- 86.4k
1
vote
0
answers
107
views
Does bayesians' critique to frequentists apply to themselves too?
I've been reading about bayesians versus frequentists, including articles in this forum (like this one). Key is of course the issue of "priors". The bayesian critique being that frequentists ...
- 127
12
votes
2
answers
3k
views
What does "Parameters are fixed and data vary" in frequentists' term and "Parameters vary and data are fixed" in Bayesians' term exactly mean?
I hear the sentence in my question a lot, I kind of understand what it means but never have a clear picture of it. Hope to get the clear picture of what the sentence exactly mean.
- 3,585
1
vote
0
answers
27
views
Science practice: Where to introduce approximations?
In my work, I am using an algorithm which relies on estimates of the gradient of the log-posterior at a collection of Monte Carlo samples. Since this gradient is not available in closed form, I must ...
- 585
2
votes
1
answer
364
views
When is it okay to not use model selection
If I have a model in mind, to ask a very specific question, do I have to do some form of model/variable selection?
There are many papers describing different ways to do model selection, why some are ...
- 269
9
votes
2
answers
628
views
Alternatives to the null hypothesis significance testing framework
How did academics support hypotheses before the null hypothesis significance testing (NHST) framework was, in part, introduced and democratized by Fisher/Neyman & Pearson? Suppose NHST was never a ...
- 2,540
3
votes
1
answer
48
views
Crossing Frequentism and Bayesian Analysis
Has anyone considered giving the posteriors of an analysis a sampling distribution and seeing where, methodologically, things could go from there?
For details, check out: https://sdba-stats.weebly.com
- 31
3
votes
1
answer
176
views
Structural complexity versus ontological complexity
From the article https://en.wikipedia.org/wiki/Occam%27s_razor:
Another contentious aspect of the razor is that a theory can become more complex in terms of its structure (or syntax), while its ...
- 679
3
votes
1
answer
173
views
Are causal effects constant over time?
The possibility that correlations are unstable over time is a matter of fact. Just for example we can consider that models included in these articles: https://www.sciencedirect.com/science/article/abs/...
- 5,779