Questions tagged [philosophical]
For questions about PHILOSOPHY of statistics or probability: interpretations of probability, foundational issues with frequentist/Bayesian statistics, etc. Do not use this tag for generally speculative (aka "philosophical") questions.
185
questions
10
votes
7
answers
4k
views
To a Bayesian, does a trick coin with two heads have 50% chance of flipping heads if they don't know that it has two heads?
And to expand on this idea, would uncertainty and complexity be the same thing to a Bayesian? For example, games with much randomness like poker and games that are more complexity based like chess -- ...
- 103
1
vote
1
answer
65
views
Why not conflating a distribution over distributions with a distribution?
This question is more like a philosophical one. I constantly find the notion of a distribution over distributions and redundant. Why don't we conflate that with the notion of a single distribution (by ...
- 235
12
votes
4
answers
1k
views
Do testability and falsifiability have statistical definitions?
Psychology: the Core Concepts says
Psychology differs from the pseudosciences in that it employs
the scientific method to test its ideas empirically. The scientific
method relies on ...
- 18.9k
6
votes
4
answers
175
views
iid data (Bayesian) vs iid random variables (Frequentist)?
I've been pondering the differences in notation / language used in some of the resources I've read for statistics / machine learning.
Warning: this might be embarrassingly obvious to any decent ...
- 313
0
votes
0
answers
24
views
Can you use your experimental predictor as an outcome variable in a time series statistical model?
I asked chatGPT and it said that of course I can, but I thought having a human in the loop would be a good idea to confirm.
The main question is: Can you use your experimental predictor as an outcome ...
- 101
1
vote
0
answers
36
views
Comparing scores on different dimensions in factor models
So I'm currently conducting some research where I'm using item response theory (IRT) to estimate the difficulty of school subjects in Norway. It turns out that a two-dimensional, simple-structure ...
- 11
4
votes
1
answer
202
views
What are some good books on the philosophy of statistics
I am a PhD in biological sciences with some background in graduate level probability
I am interested in questions like what does it mean for an event to have probability x at a philosophical level
I ...
0
votes
1
answer
447
views
What is the Statistical Method?
I apologize in advance for the possibly philosophical nature of the question, however I would like to get answers from this website rather than from Philosophy exchange at the moment. I also come from ...
- 227
1
vote
1
answer
54
views
How is Bayes Stats supposedly more "intuitive" when it requires us to think probabilistically which IS NOT intuitive for most folks? [closed]
People don’t naturally think in probabilistic ways, they often form priors through point estimates because it’s less cognitively taxing to reduce everything down to single numbers. So if the ...
- 11
1
vote
1
answer
29
views
Machine learning and empiricism: does prediction translate to evidence?
Let $X_1, ..., X_n$ be a series of idd random variables. Imagine we are training a machine learning classifier over these variables with the task of predicting a feature $F$. A typical scenario might ...
- 153
2
votes
1
answer
52
views
Arbitrariness in statistical tests
I am new to statistics and find the following procedure unnatural
and arbitrary. Could someone point out what I miss, and how I
should be thinking?
Assumptions
I have two vectors of real numbers $x_{i}...
- 235
0
votes
0
answers
39
views
Understanding Bayes' Theorem, or: Are all pluviophobes hermits?
Let $A$ be the event "I go out", $B$ be the event "It rains". Then Bayes' Theorem tells us that $$P[A|B] = \frac{P[B|A]P[A]}{P[B]}.$$
I think the weather doesn't care what I do, so ...
- 535
1
vote
0
answers
62
views
Biasness of an estimator depends on whether you take expectation of the estimator or its inverse
(Please read until the end)
know
Consider two ways of writing the exponential distribution-
(A) $\frac{1}{\beta} e^{-\frac{x}{\beta}}$ and
(B) $\theta e^{-x\theta}$
If I estimate $\beta$ or $\theta$...
- 11
3
votes
0
answers
77
views
A misspecification error with linear models that can complete reverse the direction of an effect, has this been described, has this a name?
Linear models are ubiquitous in economic, social, health and nutritional sciences and the starting point for much research and many articles.
However, there is a problem with linear models. When the ...
- 68.2k
6
votes
2
answers
295
views
Frequentist inference with a null hypothesis that reflects theory a good-enough belt around it
TL;DR:
With frequentist statistics, does it make sense to 1) no longer use significance testing, 2) set the point null hypothesis to reflect theory and decide a priori when to refute it, and 3) use a ...
- 319
20
votes
3
answers
2k
views
Within the frequentist "school of thought" how are beliefs updated?
Background
Edit: I realize my use of the word "hypothesis" is confusing, I do not mean specifically a null hypothesis. I mean a proposition that something is true.
From my limited ...
- 367
2
votes
1
answer
40
views
How could one get a difference between expected and observed probabilities with rare events?
Say we have a car with an electronic ignition system. Our engineers have deemed that due to mechanical failure possibilities, there is a 1 in a billion (or some huge number) chance that the car will ...
- 23
12
votes
4
answers
1k
views
Is anything inherently random?
Is anything inherently random? Or is all randomness observed in data either "errors in measurement" or "lack of understanding"? Assume we could measure everything with infinite ...
- 4,156
17
votes
5
answers
2k
views
Do all observations arise from probability distributions?
Below is the quote from Karl Pearson in the book: “The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century” by David Salsburg:
Over a hundred years ago, Karl Pearson ...
11
votes
4
answers
1k
views
Bayesian analysis used merely as a computational tool?
I have sometimes seen some statisticians used bayesian analysis and related techniques such as MCMC simply as a tool when a frequentist approach is not satisfying, typically for example when the ...
- 397
20
votes
3
answers
2k
views
Does the rejection of the null hypothesis have anything to do with Popper's theory of falsification?
According to Popper, we cannot verify a hypothesis due to the problem of induction - we can only aim to falsify it. If we are repeatedly unable to falsify it, the hypothesis is said to be tentatively ...
- 301
1
vote
0
answers
31
views
Understanding Countability in Sample Spaces [duplicate]
In [Casella, Berger] Statistical Inference there is a short discussion on countability of sample spaces and its implications:
This distinction between countable and uncountable sample spaces is ...
1
vote
0
answers
64
views
Is Probability just Math? [closed]
Is Probability just a "branch of mathematics" as wikipedia suggests or is it something larger than that?
More like we use math for real world problems like engineering, medicine, meteorology ...
- 123
1
vote
0
answers
135
views
Deterministic or stochastic universe in Bayesian statistics?
Dave Harris says the following in "Knightian uncertainty versus Black Swan event":
In Bayesian thinking, chance doesn't really exist. What does exist is a system that is too complicated and ...
- 63.6k
4
votes
1
answer
2k
views
What is meant by divergence in statistics?
I have learned about the Intuition on the Kullback-Leibler (KL) Divergence as how much a model distribution function differs from the theoretical/true distribution of the data.
The two most important ...
- 2,775
11
votes
5
answers
776
views
How do you know something isn't random?
Suppose I made a random number generator that's supposed to return a number 1-10, but I made it always return 4, and didn't tell you.
How would you know with 100% certainty it wasn't random?
Even if ...
5
votes
2
answers
506
views
What is Cromwell's rule and why is it important for Bayesians?
I have just heard of Cromwell's rule, but I'm not sure I understand it very well. What is Cromwell's rule and why is it important for Bayesian statistics?
46
votes
6
answers
5k
views
How seriously should I think about the different philosophies of statistics?
I've just finished a module where we covered the different approaches to statistical problems – mainly Bayesian vs frequentist. The lecturer also announced that she is a frequentist. We covered some ...
- 413
5
votes
1
answer
60
views
Does using a probabilistic model for a real-world event make it harder to identify its causes?
I recently read this odd critique of statistics (the author calls it a critique of probability theory, but I think he doesn't understand the difference probability theory and statistics).
http://...
- 17.3k
6
votes
3
answers
193
views
How to answer critiques about the inapplicability of the framework of frequentist statistics to the real world?
I often hear the argument that frequentist stats is useless or contorted because no event is precisely repeatable, let alone repeatable infinitely many times, and because there are no iid sequences in ...
- 17.3k
1
vote
2
answers
108
views
Why is a 100 heads run surprising? [closed]
Assume we have a fair coin. We flip it 100 times. The outcome is all heads.
Why is it that all heads outcome is more surprising to us than a "more random looking" outcome with less ...
- 111
2
votes
2
answers
485
views
Bayesian Probability of Zero?
I've been reading a few different philosophical papers/books which have mentioned a "Bayesian belief". Within these texts I've been basically inferring that within the Bayesian theorem, ...
- 21
2
votes
1
answer
38
views
Should a feature importance score be invariant to transformations of the response?
This is more of a philosophical question that came up in a discussion with a friend - consider some 'feature importance' procedure associated to a model (say a regression model). You run your model ...
- 1,431
2
votes
1
answer
1k
views
Why can't we say that the probability of the true parameter being within a 90% confidence interval is 90%? [duplicate]
I've been reading a bit about the confidence intervals on Wikipedia. The section on misunderstandings says:
A 95% confidence level does not mean that for a given realized
interval there is a 95% ...
1
vote
0
answers
39
views
How to deal with different opinions in statistics and data analysis? [closed]
As I see it, there are a gap between theoretical work in statistics and real-world data analysis; and differences in opinions among applied statisticians with regards to their approaches to data ...
- 123
8
votes
3
answers
2k
views
The explosive AR(1) process with $\varphi>1$, where was this first represented as a stationary, but non-causal, time-series?
According to this question and answer Explosive AR(MA) processes are stationary? the AR(1) process (with $e_t$ white noise):
$$X_{t}=\varphi X_{t-1}+e_{t} \qquad , e_t \sim WN(0,\sigma)$$
is a ...
- 68.2k
1
vote
0
answers
104
views
Does bayesians' critique to frequentists apply to themselves too?
I've been reading about bayesians versus frequentists, including articles in this forum (like this one). Key is of course the issue of "priors". The bayesian critique being that frequentists ...
- 127
11
votes
2
answers
2k
views
What does "Parameters are fixed and data vary" in frequentists' term and "Parameters vary and data are fixed" in Bayesians' term exactly mean?
I hear the sentence in my question a lot, I kind of understand what it means but never have a clear picture of it. Hope to get the clear picture of what the sentence exactly mean.
- 3,145
1
vote
0
answers
26
views
Science practice: Where to introduce approximations?
In my work, I am using an algorithm which relies on estimates of the gradient of the log-posterior at a collection of Monte Carlo samples. Since this gradient is not available in closed form, I must ...
- 545
2
votes
1
answer
199
views
When is it okay to not use model selection
If I have a model in mind, to ask a very specific question, do I have to do some form of model/variable selection?
There are many papers describing different ways to do model selection, why some are ...
- 249
9
votes
1
answer
555
views
Alternatives to the null hypothesis significance testing framework
How did academics support hypotheses before the null hypothesis significance testing (NHST) framework was, in part, introduced and democratized by Fisher/Neyman & Pearson? Suppose NHST was never a ...
- 2,386
3
votes
1
answer
48
views
Crossing Frequentism and Bayesian Analysis
Has anyone considered giving the posteriors of an analysis a sampling distribution and seeing where, methodologically, things could go from there?
For details, check out: https://sdba-stats.weebly.com
- 31
3
votes
1
answer
164
views
Structural complexity versus ontological complexity
From the article https://en.wikipedia.org/wiki/Occam%27s_razor:
Another contentious aspect of the razor is that a theory can become more complex in terms of its structure (or syntax), while its ...
- 534
3
votes
1
answer
135
views
Are causal effects constant over time?
The possibility that correlations are unstable over time is a matter of fact. Just for example we can consider that models included in these articles: https://www.sciencedirect.com/science/article/abs/...
- 4,896
0
votes
0
answers
102
views
Bayes estimates and model misspecification
Consider a misspecified model:
$$P \sim \text{SegmentedUniform}(0,1) \\ Y \mid P \sim \text{Binomial}(N,P).$$
Where SegmentedUniform has uniform density on intervals $$(0.1, 0.2), (0.3, 0.4), (0.5, 0....
- 2,386
1
vote
0
answers
32
views
moderate size of the data sample [closed]
It is often that the authors of papers in AI, ML and Statistics write that their methods are proved to perform good for "moderate sample sizes". I saw this statement in very high ranking ...
- 426
4
votes
3
answers
218
views
Why does a small $p$-value indicate incompatibility with the null?
Let's take, as a simple example, the two-tailed one-sample hypothesis test on the population mean. Suppose we've determined an $\alpha$-level a priori.
Let $X_1, \dots, X_n \overset{\text{iid}}{\sim}\...
- 4,274
69
votes
32
answers
5k
views
What are the worst (commonly adopted) ideas/principles in statistics?
In my statistical teaching, I encounter some stubborn ideas/principles relating to statistics that have become popularised, yet seem to me to be misleading, or in some cases utterly without merit. I ...
3
votes
0
answers
75
views
Interpretation of sampling distribution as the main distinction between Bayesian and classical statistics (Leamer)
In Hendry et al. (1990) p. 187-188, Edward Leamer says:
To me the essential difference between the Bayesian and a classical point of view is not that the parameters are treated as random variables, ...
- 63.6k
1
vote
1
answer
23
views
Should we try imputation on cases with slightly problematic datasets or prefer ommiting observations
I would like to ask a general question which makes me worry when I try to impute NA values.
We know that most of the imputation methods are based on the rest non-NA values. However, if we know that ...
- 2,142