Skip to main content

Questions tagged [philosophical]

For questions about PHILOSOPHY of statistics or probability: interpretations of probability, foundational issues with frequentist/Bayesian statistics, etc. Do not use this tag for generally speculative (aka "philosophical") questions.

Filter by
Sorted by
Tagged with
1 vote
1 answer
52 views

Birnbaum's Theorem: Strong belief in a model $\implies$ the likelihood function must be used as a data reduction device?

Working through understanding section 6.3.2 (pg. 292-294) in Casella and Berger's Statistical Inference (2nd-ed). The following definitions and principles are given: Definition (Experiment): An ...
Aaron Hendrickson's user avatar
0 votes
0 answers
45 views

Help with completing a derivation of usefulness of cross-validation

This question is raised as a result of my attempt to answer this other question of mine. Let's refer to all our prior knowledge, both explicit and implicit, as $X_\text{true}$. Almost always, we are ...
Feri's user avatar
  • 197
2 votes
0 answers
65 views

Can an outcome variable be used twice in the same model?

When is it appropriate to use the same outcome variable in two likelihoods in the same model framework? Here is a specific example: ...
Benny Borremans's user avatar
6 votes
3 answers
400 views

The rationale for when significance or null hypothesis testing is needed

Why do people sometimes claim that an effect is so huge and "obvious" that it does not warrant any inferential statistics calculation, even though the sample size is not large? This is ...
BioLeal's user avatar
  • 175
2 votes
0 answers
37 views

What is the role of determinism in stochastics?

What is the role of determinism in stochastics? In my opinion, determinism, much like the continuum, is merely a concept to describe an ideal world. In this idealized world we can calculate and argue. ...
Henning's user avatar
  • 21
12 votes
7 answers
2k views

Are “Data are fixed” in Bayesian viewpoint and “Data are random” in frequentist viewpoint talking about the same thing mathematically?

In my opinion, in BOTH Bayesian and Frequentist inferences, observational data $x$ are modelled as the observed value of a random variable $X$ which follows a certain probability distribution. ...
Ken T's user avatar
  • 485
11 votes
5 answers
1k views

How would a Bayesian define a fair coin?

In the frequentist worldview, probabilities are long-run relative frequencies. Hence, a fair coin can be defined as a coin, for which the long run relative frequency of each of the sides approaches 0....
Sam's user avatar
  • 755
12 votes
7 answers
6k views

To a Bayesian, does a trick coin with two heads have 50% chance of flipping heads if they don't know that it has two heads?

And to expand on this idea, would uncertainty and complexity be the same thing to a Bayesian? For example, games with much randomness like poker and games that are more complexity based like chess -- ...
setblack7's user avatar
  • 123
1 vote
1 answer
79 views

Why not conflating a distribution over distributions with a distribution?

This question is more like a philosophical one. I constantly find the notion of a distribution over distributions and redundant. Why don't we conflate that with the notion of a single distribution (by ...
Student's user avatar
  • 235
16 votes
4 answers
1k views

Do testability and falsifiability have statistical definitions?

Psychology: the Core Concepts says Psychology differs from the pseudosciences in that it employs the scientific method to test its ideas empirically. The scientific method relies on ...
Tim's user avatar
  • 19.6k
8 votes
4 answers
327 views

iid data (Bayesian) vs iid random variables (Frequentist)?

I've been pondering the differences in notation / language used in some of the resources I've read for statistics / machine learning. Warning: this might be embarrassingly obvious to any decent ...
paul's user avatar
  • 415
1 vote
0 answers
39 views

Comparing scores on different dimensions in factor models

So I'm currently conducting some research where I'm using item response theory (IRT) to estimate the difficulty of school subjects in Norway. It turns out that a two-dimensional, simple-structure ...
Sverdo's user avatar
  • 11
9 votes
4 answers
650 views

What are some good books on the philosophy of statistics?

I am a PhD in biological sciences with some background in graduate-level probability. I am interested in questions like what does it mean for an event to have probability $x$ at a philosophical level. ...
0 votes
1 answer
481 views

What is the Statistical Method?

I apologize in advance for the possibly philosophical nature of the question, however I would like to get answers from this website rather than from Philosophy exchange at the moment. I also come from ...
Amr's user avatar
  • 237
1 vote
1 answer
72 views

How is Bayes Stats supposedly more "intuitive" when it requires us to think probabilistically which IS NOT intuitive for most folks? [closed]

People don’t naturally think in probabilistic ways, they often form priors through point estimates because it’s less cognitively taxing to reduce everything down to single numbers. So if the ...
Ruiz's user avatar
  • 11
2 votes
1 answer
35 views

Machine learning and empiricism: does prediction translate to evidence?

Let $X_1, ..., X_n$ be a series of idd random variables. Imagine we are training a machine learning classifier over these variables with the task of predicting a feature $F$. A typical scenario might ...
lafinur's user avatar
  • 235
2 votes
1 answer
76 views

Arbitrariness in statistical tests

I am new to statistics and find the following procedure unnatural and arbitrary. Could someone point out what I miss, and how I should be thinking? Assumptions I have two vectors of real numbers $x_{i}...
Student's user avatar
  • 235
0 votes
0 answers
41 views

Understanding Bayes' Theorem, or: Are all pluviophobes hermits?

Let $A$ be the event "I go out", $B$ be the event "It rains". Then Bayes' Theorem tells us that $$P[A|B] = \frac{P[B|A]P[A]}{P[B]}.$$ I think the weather doesn't care what I do, so ...
mjc's user avatar
  • 599
1 vote
1 answer
112 views

Bias of an estimator depends on whether you take expectation of the estimator or its inverse

(Please read until the end) Consider two ways of writing the exponential distribution: (A) $\frac{1}{\beta} e^{-\frac{x}{\beta}}$ and (B) $\theta e^{-x\theta}$ If I estimate $\beta$ or $\theta$ ...
learner's user avatar
  • 11
3 votes
0 answers
107 views

A misspecification error with linear models that can complete reverse the direction of an effect, has this been described, has this a name?

Linear models are ubiquitous in economic, social, health and nutritional sciences and the starting point for much research and many articles. However, there is a problem with linear models. When the ...
Sextus Empiricus's user avatar
6 votes
2 answers
360 views

Frequentist inference with a null hypothesis that reflects theory a good-enough belt around it

TL;DR: With frequentist statistics, does it make sense to 1) no longer use significance testing, 2) set the point null hypothesis to reflect theory and decide a priori when to refute it, and 3) use a ...
Raoul Van Oosten's user avatar
20 votes
3 answers
2k views

Within the frequentist "school of thought" how are beliefs updated?

Background Edit: I realize my use of the word "hypothesis" is confusing, I do not mean specifically a null hypothesis. I mean a proposition that something is true. From my limited ...
DancingIceCream's user avatar
2 votes
1 answer
48 views

How could one get a difference between expected and observed probabilities with rare events?

Say we have a car with an electronic ignition system. Our engineers have deemed that due to mechanical failure possibilities, there is a 1 in a billion (or some huge number) chance that the car will ...
Aaron's user avatar
  • 23
12 votes
4 answers
1k views

Is anything inherently random?

Is anything inherently random? Or is all randomness observed in data either "errors in measurement" or "lack of understanding"? Assume we could measure everything with infinite ...
ColorStatistics's user avatar
18 votes
5 answers
2k views

Do all observations arise from probability distributions?

Below is the quote from Karl Pearson in the book: “The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century” by David Salsburg: Over a hundred years ago, Karl Pearson ...
11 votes
4 answers
1k views

Bayesian analysis used merely as a computational tool?

I have sometimes seen some statisticians used bayesian analysis and related techniques such as MCMC simply as a tool when a frequentist approach is not satisfying, typically for example when the ...
rosas's user avatar
  • 397
21 votes
3 answers
3k views

Does the rejection of the null hypothesis have anything to do with Popper's theory of falsification?

According to Popper, we cannot verify a hypothesis due to the problem of induction - we can only aim to falsify it. If we are repeatedly unable to falsify it, the hypothesis is said to be tentatively ...
Rainer_Zoufal's user avatar
1 vote
0 answers
34 views

Understanding Countability in Sample Spaces [duplicate]

In [Casella, Berger] Statistical Inference there is a short discussion on countability of sample spaces and its implications: This distinction between countable and uncountable sample spaces is ...
Shivashriganesh Mahato's user avatar
1 vote
0 answers
65 views

Is Probability just Math? [closed]

Is Probability just a "branch of mathematics" as wikipedia suggests or is it something larger than that? More like we use math for real world problems like engineering, medicine, meteorology ...
IXN's user avatar
  • 123
1 vote
0 answers
194 views

Deterministic or stochastic universe in Bayesian statistics?

Dave Harris says the following in "Knightian uncertainty versus Black Swan event": In Bayesian thinking, chance doesn't really exist. What does exist is a system that is too complicated and ...
Richard Hardy's user avatar
4 votes
1 answer
3k views

What is meant by divergence in statistics?

I have learned about the Intuition on the Kullback-Leibler (KL) Divergence as how much a model distribution function differs from the theoretical/true distribution of the data. The two most important ...
Thalassophile's user avatar
11 votes
5 answers
923 views

How do you know something isn't random?

Suppose I made a random number generator that's supposed to return a number 1-10, but I made it always return 4, and didn't tell you. How would you know with 100% certainty it wasn't random? Even if ...
QuestionLimitGoBrrrrr's user avatar
5 votes
2 answers
724 views

What is Cromwell's rule and why is it important for Bayesians?

I have just heard of Cromwell's rule, but I'm not sure I understand it very well. What is Cromwell's rule and why is it important for Bayesian statistics?
Lennis Dindley's user avatar
47 votes
6 answers
5k views

How seriously should I think about the different philosophies of statistics?

I've just finished a module where we covered the different approaches to statistical problems – mainly Bayesian vs frequentist. The lecturer also announced that she is a frequentist. We covered some ...
DerrYe's user avatar
  • 423
5 votes
1 answer
63 views

Does using a probabilistic model for a real-world event make it harder to identify its causes?

I recently read this odd critique of statistics (the author calls it a critique of probability theory, but I think he doesn't understand the difference probability theory and statistics). http://...
DeltaIV's user avatar
  • 18.2k
8 votes
3 answers
226 views

How to answer critiques about the inapplicability of the framework of frequentist statistics to the real world?

I often hear the argument that frequentist stats is useless or contorted because no event is precisely repeatable, let alone repeatable infinitely many times, and because there are no iid sequences in ...
DeltaIV's user avatar
  • 18.2k
1 vote
2 answers
115 views

Why is a 100 heads run surprising? [closed]

Assume we have a fair coin. We flip it 100 times. The outcome is all heads. Why is it that all heads outcome is more surprising to us than a "more random looking" outcome with less ...
Kaveh's user avatar
  • 111
2 votes
2 answers
822 views

Bayesian Probability of Zero?

I've been reading a few different philosophical papers/books which have mentioned a "Bayesian belief". Within these texts I've been basically inferring that within the Bayesian theorem, ...
CompSciFly's user avatar
2 votes
1 answer
44 views

Should a feature importance score be invariant to transformations of the response?

This is more of a philosophical question that came up in a discussion with a friend - consider some 'feature importance' procedure associated to a model (say a regression model). You run your model ...
WeakLearner's user avatar
  • 1,501
2 votes
1 answer
2k views

Why can't we say that the probability of the true parameter being within a 90% confidence interval is 90%? [duplicate]

I've been reading a bit about the confidence intervals on Wikipedia. The section on misunderstandings says: A 95% confidence level does not mean that for a given realized interval there is a 95% ...
relatively_random's user avatar
1 vote
0 answers
46 views

How to deal with different opinions in statistics and data analysis? [closed]

As I see it, there are a gap between theoretical work in statistics and real-world data analysis; and differences in opinions among applied statisticians with regards to their approaches to data ...
Ahmed Ali's user avatar
  • 133
9 votes
4 answers
3k views

The explosive AR(1) process with $\varphi>1$, where was this first represented as a stationary, but non-causal, time-series?

According to this question and answer Explosive AR(MA) processes are stationary? the AR(1) process (with $e_t$ white noise): $$X_{t}=\varphi X_{t-1}+e_{t} \qquad , e_t \sim WN(0,\sigma)$$ is a ...
Sextus Empiricus's user avatar
1 vote
0 answers
107 views

Does bayesians' critique to frequentists apply to themselves too?

I've been reading about bayesians versus frequentists, including articles in this forum (like this one). Key is of course the issue of "priors". The bayesian critique being that frequentists ...
chatGPT's user avatar
  • 127
11 votes
2 answers
3k views

What does "Parameters are fixed and data vary" in frequentists' term and "Parameters vary and data are fixed" in Bayesians' term exactly mean?

I hear the sentence in my question a lot, I kind of understand what it means but never have a clear picture of it. Hope to get the clear picture of what the sentence exactly mean.
xabzakabecd's user avatar
  • 3,525
1 vote
0 answers
27 views

Science practice: Where to introduce approximations?

In my work, I am using an algorithm which relies on estimates of the gradient of the log-posterior at a collection of Monte Carlo samples. Since this gradient is not available in closed form, I must ...
J.Galt's user avatar
  • 565
2 votes
1 answer
293 views

When is it okay to not use model selection

If I have a model in mind, to ask a very specific question, do I have to do some form of model/variable selection? There are many papers describing different ways to do model selection, why some are ...
Dylan_Gomes's user avatar
10 votes
2 answers
609 views

Alternatives to the null hypothesis significance testing framework

How did academics support hypotheses before the null hypothesis significance testing (NHST) framework was, in part, introduced and democratized by Fisher/Neyman & Pearson? Suppose NHST was never a ...
fool's user avatar
  • 2,480
3 votes
1 answer
48 views

Crossing Frequentism and Bayesian Analysis

Has anyone considered giving the posteriors of an analysis a sampling distribution and seeing where, methodologically, things could go from there? For details, check out: https://sdba-stats.weebly.com
Hiram Lau's user avatar
3 votes
1 answer
172 views

Structural complexity versus ontological complexity

From the article https://en.wikipedia.org/wiki/Occam%27s_razor: Another contentious aspect of the razor is that a theory can become more complex in terms of its structure (or syntax), while its ...
Single Malt's user avatar
3 votes
1 answer
163 views

Are causal effects constant over time?

The possibility that correlations are unstable over time is a matter of fact. Just for example we can consider that models included in these articles: https://www.sciencedirect.com/science/article/abs/...
markowitz's user avatar
  • 5,729