Questions tagged [philosophical]

For questions about PHILOSOPHY of statistics or probability: interpretations of probability, foundational issues with frequentist/Bayesian statistics, etc. Do not use this tag for generally speculative (aka "philosophical") questions.

Filter by
Sorted by
Tagged with
0 votes
0 answers
36 views

Understanding Bayes' Theorem, or: Are all pluviophobes hermits?

Let $A$ be the event "I go out", $B$ be the event "It rains". Then Bayes' Theorem tells us that $$P[A|B] = \frac{P[B|A]P[A]}{P[B]}.$$ I think the weather doesn't care what I do, so ...
  • 341
1 vote
0 answers
50 views

Why are there n interpretations of probability, yet only two of those interpretations led to philosophies in statistical inference?

Why are the subjectivist (bayesians) and frequentist (objectivist) statisticians but no propensity statisticians? It seems that every interpretation of probability should yield its own branch of ...
1 vote
0 answers
49 views

Biasness of an estimator depends on whether you take expectation of the estimator or its inverse

(Please read until the end) know Consider two ways of writing the exponential distribution- (A) $\frac{1}{\beta} e^{-\frac{x}{\beta}}$ and (B) $\theta e^{-x\theta}$ If I estimate $\beta$ or $\theta$...
  • 11
3 votes
0 answers
49 views

A misspecification error with linear models that can complete reverse the direction of an effect, has this been described, has this a name?

Linear models are ubiquitous in economic, social, health and nutritional sciences and the starting point for much research and many articles. However, there is a problem with linear models. When the ...
5 votes
2 answers
263 views

Frequentist inference with a null hypothesis that reflects theory a good-enough belt around it

TL;DR: With frequentist statistics, does it make sense to 1) no longer use significance testing, 2) set the point null hypothesis to reflect theory and decide a priori when to refute it, and 3) use a ...
20 votes
3 answers
2k views

Within the frequentist "school of thought" how are beliefs updated?

Background Edit: I realize my use of the word "hypothesis" is confusing, I do not mean specifically a null hypothesis. I mean a proposition that something is true. From my limited ...
2 votes
1 answer
34 views

How could one get a difference between expected and observed probabilities with rare events?

Say we have a car with an electronic ignition system. Our engineers have deemed that due to mechanical failure possibilities, there is a 1 in a billion (or some huge number) chance that the car will ...
  • 23
9 votes
4 answers
983 views

Is anything inherently random?

Is anything inherently random? Or is all randomness observed in data either "errors in measurement" or "lack of understanding"? Assume we could measure everything with infinite ...
15 votes
5 answers
1k views

Do all observations arise from probability distributions?

Below is the quote from Karl Pearson in the book: “The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century” by David Salsburg: Over a hundred years ago, Karl Pearson ...
11 votes
4 answers
1k views

Bayesian analysis used merely as a computational tool?

I have sometimes seen some statisticians used bayesian analysis and related techniques such as MCMC simply as a tool when a frequentist approach is not satisfying, typically for example when the ...
  • 387
20 votes
3 answers
2k views

Does the rejection of the null hypothesis have anything to do with Popper's theory of falsification?

According to Popper, we cannot verify a hypothesis due to the problem of induction - we can only aim to falsify it. If we are repeatedly unable to falsify it, the hypothesis is said to be tentatively ...
1 vote
0 answers
29 views

Understanding Countability in Sample Spaces [duplicate]

In [Casella, Berger] Statistical Inference there is a short discussion on countability of sample spaces and its implications: This distinction between countable and uncountable sample spaces is ...
1 vote
0 answers
62 views

Is Probability just Math? [closed]

Is Probability just a "branch of mathematics" as wikipedia suggests or is it something larger than that? More like we use math for real world problems like engineering, medicine, meteorology ...
  • 123
0 votes
0 answers
69 views

Deterministic or stochastic universe in Bayesian statistics?

Dave Harris says the following in "Knightian uncertainty versus Black Swan event": In Bayesian thinking, chance doesn't really exist. What does exist is a system that is too complicated and ...
3 votes
1 answer
995 views

What is meant by divergence in statistics?

I have learned about the Intuition on the Kullback-Leibler (KL) Divergence as how much a model distribution function differs from the theoretical/true distribution of the data. The two most important ...
  • 2,461
10 votes
5 answers
392 views

How do you know something isn't random?

Suppose I made a random number generator that's supposed to return a number 1-10, but I made it always return 4, and didn't tell you. How would you know with 100% certainty it wasn't random? Even if ...
4 votes
2 answers
323 views

What is Cromwell's rule and why is it important for Bayesians?

I have just heard of Cromwell's rule, but I'm not sure I understand it very well. What is Cromwell's rule and why is it important for Bayesian statistics?
46 votes
6 answers
5k views

How seriously should I think about the different philosophies of statistics?

I've just finished a module where we covered the different approaches to statistical problems – mainly Bayesian vs frequentist. The lecturer also announced that she is a frequentist. We covered some ...
  • 413
5 votes
1 answer
55 views

Does using a probabilistic model for a real-world event make it harder to identify its causes?

I recently read this odd critique of statistics (the author calls it a critique of probability theory, but I think he doesn't understand the difference probability theory and statistics). http://...
  • 16.2k
6 votes
3 answers
172 views

How to answer critiques about the inapplicability of the framework of frequentist statistics to the real world?

I often hear the argument that frequentist stats is useless or contorted because no event is precisely repeatable, let alone repeatable infinitely many times, and because there are no iid sequences in ...
  • 16.2k
1 vote
2 answers
106 views

Why is a 100 heads run surprising? [closed]

Assume we have a fair coin. We flip it 100 times. The outcome is all heads. Why is it that all heads outcome is more surprising to us than a "more random looking" outcome with less ...
  • 111
2 votes
2 answers
177 views

Bayesian Probability of Zero?

I've been reading a few different philosophical papers/books which have mentioned a "Bayesian belief". Within these texts I've been basically inferring that within the Bayesian theorem, ...
1 vote
1 answer
32 views

Should a feature importance score be invariant to transformations of the response?

This is more of a philosophical question that came up in a discussion with a friend - consider some 'feature importance' procedure associated to a model (say a regression model). You run your model ...
  • 1,146
2 votes
1 answer
381 views

Why can't we say that the probability of the true parameter being within a 90% confidence interval is 90%? [duplicate]

I've been reading a bit about the confidence intervals on Wikipedia. The section on misunderstandings says: A 95% confidence level does not mean that for a given realized interval there is a 95% ...
1 vote
0 answers
35 views

How to deal with different opinions in statistics and data analysis? [closed]

As I see it, there are a gap between theoretical work in statistics and real-world data analysis; and differences in opinions among applied statisticians with regards to their approaches to data ...
  • 123
8 votes
3 answers
1k views

The explosive AR(1) process with $\varphi>1$, where was this first represented as a stationary, but non-causal, time-series?

According to this question and answer Explosive AR(MA) processes are stationary? the AR(1) process (with $e_t$ white noise): $$X_{t}=\varphi X_{t-1}+e_{t} \qquad , e_t \sim WN(0,\sigma)$$ is a ...
2 votes
0 answers
94 views

Does bayesians' critique to frequentists apply to themselves too?

I've been reading about bayesians versus frequentists, including articles in this forum (like this one). Key is of course the issue of "priors". The bayesian critique being that frequentists ...
10 votes
2 answers
2k views

What does "Parameters are fixed and data vary" in frequentists' term and "Parameters vary and data are fixed" in Bayesians' term exactly mean?

I hear the sentence in my question a lot, I kind of understand what it means but never have a clear picture of it. Hope to get the clear picture of what the sentence exactly mean.
  • 2,645
1 vote
0 answers
26 views

Science practice: Where to introduce approximations?

In my work, I am using an algorithm which relies on estimates of the gradient of the log-posterior at a collection of Monte Carlo samples. Since this gradient is not available in closed form, I must ...
  • 473
1 vote
1 answer
114 views

When is it okay to not use model selection

If I have a model in mind, to ask a very specific question, do I have to do some form of model/variable selection? There are many papers describing different ways to do model selection, why some are ...
9 votes
1 answer
482 views

Alternatives to the null hypothesis significance testing framework

How did academics support hypotheses before the null hypothesis significance testing (NHST) framework was, in part, introduced and democratized by Fisher/Neyman & Pearson? Suppose NHST was never a ...
  • 2,141
3 votes
1 answer
46 views

Crossing Frequentism and Bayesian Analysis

Has anyone considered giving the posteriors of an analysis a sampling distribution and seeing where, methodologically, things could go from there? For details, check out: https://sdba-stats.weebly.com
3 votes
1 answer
145 views

Structural complexity versus ontological complexity

From the article https://en.wikipedia.org/wiki/Occam%27s_razor: Another contentious aspect of the razor is that a theory can become more complex in terms of its structure (or syntax), while its ...
3 votes
1 answer
98 views

Are causal effects constant over time?

The possibility that correlations are unstable over time is a matter of fact. Just for example we can consider that models included in these articles: https://www.sciencedirect.com/science/article/abs/...
  • 4,369
0 votes
0 answers
72 views

Bayes estimates and model misspecification

Consider a misspecified model: $$P \sim \text{SegmentedUniform}(0,1) \\ Y \mid P \sim \text{Binomial}(N,P).$$ Where SegmentedUniform has uniform density on intervals $$(0.1, 0.2), (0.3, 0.4), (0.5, 0....
  • 2,141
1 vote
0 answers
30 views

moderate size of the data sample [closed]

It is often that the authors of papers in AI, ML and Statistics write that their methods are proved to perform good for "moderate sample sizes". I saw this statement in very high ranking ...
  • 376
4 votes
3 answers
186 views

Why does a small $p$-value indicate incompatibility with the null?

Let's take, as a simple example, the two-tailed one-sample hypothesis test on the population mean. Suppose we've determined an $\alpha$-level a priori. Let $X_1, \dots, X_n \overset{\text{iid}}{\sim}\...
  • 4,013
67 votes
32 answers
4k views

What are the worst (commonly adopted) ideas/principles in statistics?

In my statistical teaching, I encounter some stubborn ideas/principles relating to statistics that have become popularised, yet seem to me to be misleading, or in some cases utterly without merit. I ...
2 votes
0 answers
51 views

Interpretation of sampling distribution as the main distinction between Bayesian and classical statistics (Leamer)

In Hendry et al. (1990) p. 187-188, Edward Leamer says: To me the essential difference between the Bayesian and a classical point of view is not that the parameters are treated as random variables, ...
1 vote
1 answer
21 views

Should we try imputation on cases with slightly problematic datasets or prefer ommiting observations

I would like to ask a general question which makes me worry when I try to impute NA values. We know that most of the imputation methods are based on the rest non-NA values. However, if we know that ...
  • 1,823
6 votes
2 answers
271 views

Do variable-selection methods (e.g. Elastic Net; Lasso) invalidate theory-based models in fields where little is known?

I'm caught in a bind about the relationship between theoretical models about how the world works and statistical methods for accurately predicting an outcome in fields where little is known. I ...
  • 659
4 votes
2 answers
4k views

What is the 'true' value of a probability parameter?

We seem to distinguish empirical estimates of parameters from 'true' values, and make comparisons between the two. I can understand what an empirical estimate is. What is a 'true' value? For instance,...
  • 341
1 vote
2 answers
78 views

giving meaning to a prior distribution without invoking "physical probability"

Let's say we have a machine that produces an output which is either defective or it is not. N trial units of output have been produced. Given some prior distribution of the probability of a defective ...
2 votes
0 answers
603 views

Regression: Causation vs Prediction vs Description

In my experience it seems me that the interpretation about regression, its meaning and its scope, are debatable and great confusion exist about those things. It seems me that confusions are not go ...
  • 4,369
5 votes
2 answers
213 views

Asking nature a single question or a carefully thought out questionnaire

In The Seven Pillars of Statistical Wisdom book by Stephen M. Stigler: William Stanley Jevons, writing in his Principles of ...
  • 153
4 votes
1 answer
60 views

Typo in Deepleariningbook.org or am I misunderstanding Bayesian stats?

This is on page 133 of the book: https://www.deeplearningbook.org/contents/ml.html#pf10 In the above, it says that the data set is directly observed and so is not random If that data we observe ...
  • 2,543
7 votes
1 answer
116 views

A reliance on repeated sampling ideas can lead to logical paradoxes that appear in common rather than esoteric procedures?

I am currently studying the textbook In All Likelihood -- Statistical Modelling and Inference Using Likelihood by Yudi Pawitan. Section Repeated sampling principle: the frequentists of chapter 1 says ...
  • 1,146
4 votes
2 answers
438 views

Why would considering $\theta$ to be a random variable not be 'Bayesian'?

I am currently studying the textbook In All Likelihood -- Statistical Modelling and Inference Using Likelihood by Yudi Pawitan. Section Inverse probability: the Bayesians of chapter 1 says the ...
  • 1,146
1 vote
3 answers
192 views

Confusion about Bayesian statistics. Does the probability for heads change from .5 to 1, after observing heads?

P(A): The coin has a 50 percent chance of being Heads. P(A|X): You look at the coin, observe a Heads has landed, denote this information X, and trivially assign probability 1.0 to Heads and 0.0 to ...
  • 111
3 votes
0 answers
40 views

What's the rationale behind a normality test followed by a $t$-test?

Correct me if I'm wrong, but from my understanding, the standard procedure of testing whether data from an unknown source have a specific mean is to (a) perform a normality test to see if the data are ...
  • 1,487