Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [philosophical]

For questions about PHILOSOPHY of statistics or probability: interpretations of probability, foundational issues with frequentist/Bayesian statistics, etc. Do not use this tag for generally speculative (aka "philosophical") questions.

0
votes
0answers
13 views

Verify that data have property $X$ with hypothesis testing

In data analysis, one usually need to verify that data have property $X$ before applying method $Y$, which takes $X$ as a prerequisite. To illustrate, possible values of $(X, Y)$ include $(\text{...
1
vote
2answers
25 views

How is probability distribution type decided before applying maximum likelihood estimation?

What I can't grasp for a long time is how is the assumption about the probability distribution type made? Why do we assume that the human height is distributed exactly normally? Why not any other ...
7
votes
3answers
138 views

What is the $p$ in Bernoulli distribution?

In the Bayesian theory of probability, probability is our expression of knowledge about a certain thing, not a property of that thing. However, I always see people treat $p$ as a parameter that needs ...
0
votes
1answer
47 views

What would be an ignorance prior of AB, given the probabilities of A and B?

Let us have two events, $A$ and $B$ whose probabilities are $P(A)$ and $P(B)$. In the absence of any other information, what would be a reasonable probability to assign to $AB$, that is, $A$ and $B$ ...
0
votes
1answer
48 views

Iterating Bayes rule over time

$\require{cancel}$ In a online bayesian inference procedure one is iteratively changing the prior with a new posterior, calculated given a new set of observations. Does it mean we capture time ...
19
votes
5answers
524 views

An example where the likelihood principle *really* matters?

Is there an example where two different defensible tests with proportional likelihoods would lead one to markedly different (and equally defensible) inferences, for instance, where the p-values are ...
0
votes
1answer
36 views

Frequentist definition of probability and prediction?

The frequentist definition of probability states that: The probability of an event is the ratio of the number of cases favorable to it, to the number of all cases possible when nothing leads us to ...
2
votes
1answer
41 views

Can every parameter $\Theta$ in Bayesian modelling be explained via De Finetti`s representation theorem

My question is the following: I recently got to know (and love) De Finetti`s representation theorem and I now started to read a Book an Bayesian statistics. However this book simply takes as the ...
3
votes
0answers
47 views

Overview articles on Bayesian philosophy and methodology

While we have questions about Bayesian textbooks (1 and several other) and the philosophy behind the Bayesian thinking (2), I am interested papers (or sources of similar length, e.g. blog posts or ...
3
votes
2answers
72 views

Deterministic or stochastic universe in frequentist statistics

Does frequentist statistics take a stand on whether the universe (or at least the processes that are being modeled) is deterministic or stochastic? If so, where in the methodology does that matter?
2
votes
1answer
44 views

Probability based on Observed Data

I'm trying to figure if my reasoning and application of probability is correct in this made up example using R. Example data: ...
1
vote
1answer
136 views

Covariance versus correlation: which is a “deeper” or more “structural” property of the data?

It might seem obvious that the covariance is a "deeper" property of the data generation process (DGP), since normally the specification of a joint distribution is done in terms of its mean vector and ...
1
vote
0answers
56 views

What can we infer about an individual from a distribution over the population?

This is a somewhat philosophical question, or maybe it has to do with the interpretation of statistics. The question is: if we have a distribution of some property of a large population, but then look ...
1
vote
1answer
330 views

How can (L1 / L2) regularization be equivalent to using a prior when priors can't be changed?

I understand the argument for how training with an L1/L2 regularizer is the same thing as finding the MAP estimate when the prior is Gaussian/Laplace. But there's a crucial difference. In Bayes' ...
5
votes
3answers
107 views

Intuition about the deep meaning of Bayesian priors and its influence on posteriors

In estimating posterior distributions, Bayesians rely on the idea of the prior distribution. In many examples, I see this being set fairly arbitrarily, ie ~N(0,1). It's clear that the posterior is ...
0
votes
0answers
18 views

Can we get uniform distributions on infinite spaces by giving up infinite additivity

I am wondering whether it is possible to translate the idea of drawing a number randomly from the set of all natural numbers. If we have infinite additivity as an axiom this obviously does not work. ...
1
vote
0answers
31 views

Assigning percentages to different causes

There was recently a celebrated spat between Cathy Newman, one of the main presenters of Channel 4 News in the UK, and Jordan Peterson, a well-known academic and clinical psychologist, on the causes ...
1
vote
1answer
110 views

How does a Bayesian update his belief when something with probability 0 happened?

Define $X:=$ "coin has probability 1 to land heads" Assume that one has the prior belief: $P(X)= 1$. However after tossing the coin once it lands tails ($E:= $ "coin landed tails"). How should a ...
4
votes
0answers
74 views

Is it a problem that limiting frequencies (can) violate countable additivity?

I`ve stumbled upon the following paper by Alán Hajek https://www.jstor.org/stable/40267419, in which the author states that the Frequentist interpretation of probabilities as limiting frequencies ...
17
votes
4answers
738 views

How do Bayesian Statistics handle the absence of priors?

This question was inspired by two recent interactions I had, one here in CV, the other over at economics.se. There, I had posted an answer to the well-known "Envelope Paradox" (mind you, not as the "...
4
votes
1answer
339 views

What is the difference between classical frequentist methods and likelihood methods?

You may assume that I'm familiar with the material in Casella and Berger. This question is identical to What is the difference between Fisherian vs frequentist statistics?; however, the question was ...
3
votes
2answers
234 views

Is there a reason other than conventions why a CDF must be defined for all real numbers

There are many cases when the sample space is not the entire set of real numbers (for instance a Bernoulli trial or sampling from an interval). On the one hand: for the definition $F_X(x) = P(X \leq ...
3
votes
0answers
273 views

What is the difference between Fisherian vs frequentist statistics? [closed]

I just read a research paper that said implicitly that there was a difference between the two. I thought that Fisherian statistics was another word for frequentist statistics.
1
vote
2answers
54 views

What are some basic principles to handle “none of the above” events in Bayesian statistics?

Suppose you want to assign a noninformative prior to the following event: The next tree that we will encounter is a: Spruce Pine None of the above We don't have any prior information, so we are ...
3
votes
1answer
261 views

Isn't the Solomonoff Universal Prior Biased Like Any Other?

The Solomonoff universal prior is fixed relative to a specific choice of universal Turing machine (UTM). Now, I understand that a UTM can simulate any other UTM, so that they assign a complexity to ...
4
votes
1answer
67 views

Is a frequentist approach to inference appropriate when working with non-repeatable data?

Jackman (2009) writes on p.xxxi-xxxii: Consider researchers analyzing cross-national data in economics, political science, or sociology, say, using national accounts data from the OECD. [...] ...
1
vote
0answers
78 views

Applying machine learning to dynamic complex systems (e.g. weather prediction)

Would it be correct to say that: Physics-based, domain specific models are more widely used and are more practical in (longer term) weather forecasts than pure machine learning approaches The reason ...
7
votes
2answers
173 views

Is testing model assumptions considered p-hacking/fishing?

"P-hacking", "fishing", and "garden of forking paths" as explained here and here describes an exploratory data analysis-like style of doing research that produces biased estimates. Does testing model ...
2
votes
0answers
40 views

Does it ever make sense to talk about the confidence (faith) in a probability value?

Let us suppose that we want to know what weather will be tomorrow. We ask two meteorologists and both give us an identical probabilistic answer: It will rain with probability of 30% It will snow ...
0
votes
1answer
71 views

What are the problems with existence of an “Omniscience Machine” within the constraints of collective human knowledge? [closed]

"If a machine can read and process written texts, manuscripts, journal articles and books, it would be able to predict results of new scientific experiments and construct new scientific theories." ...
1
vote
0answers
46 views

Why can scientists that refuse to bound the prior probability declare discoveries?

Summary: There appears to be scientists that refuse to put prior probabilities on some statements, such as the existence of the Higgs Boson. This is an understandable position. These scientists, ...
19
votes
4answers
1k views

Is everyday probability just a way of dealing with the unknown (not talking quantum physics here)?

It seems like in everyday probability (not quantum physics), probabilities are really just a substitute for an unknown. Take a coin flip for example. We say it's "random," a 50% change of head and a ...
7
votes
3answers
224 views

Is there an example of two causally dependent events being logically (probabilistically) independent?

Two events $A,B$ are independent when $P(A \cap B ) = P(A)P(B)$ I am trying to drill into this definition and to try to reconcile it with our intuitive idea of independence in the real world. I feel ...
9
votes
1answer
115 views

P-values and likelihood principle

This question came up in class: If we use p-values to evaluate hypotheses on an experiment, which part of the Likelihood Principle are we not obeying: Sufficiency or Conditionality? My intuition ...
2
votes
1answer
118 views

How important are interpretations of probability to the practice of statistics?

I know that the frequentist interpretation of probability is associated with classical statistics and maximum likelihood estimation, and that a subjective interpretation of probability is considered ...
65
votes
11answers
7k views

Why should I be Bayesian when my model is wrong?

Edits: I have added a simple example: inference of the mean of the $X_i$. I have also slightly clarified why the credible intervals not matching confidence intervals is bad. I, a fairly devout ...
3
votes
1answer
119 views

HMM: Why are observations conditional on the latent state and not vice versa?

The model of a HMM consists of a latent Markov chain with state $X$ and transition probabilities $P(X^t \mid X^{t-1})$, and observation variables $Y$ that depend on the current latent state via $P(Y^t ...
4
votes
2answers
2k views

Probabilistic vs. other approaches to machine learning

I'm taking a grad course on machine learning in the ECE department of my university. On the first lecture my professor seemed to make it a point to stress the fact that the course would be taking a ...
5
votes
6answers
849 views

What is the rationale behind using the t-distribution?

(Probably a very naive question.) According to this tutorial, Student's t-test deals with the problems associated with inference based on "small" samples: the calculated mean and standard ...
2
votes
3answers
154 views

Do frequentists and Bayesians use different hypothesis tests?

In a basic refresher course on stats, they covered Z-tests, T-tests from a practical perspective: they described the assumptions for each test, and "let the software packages do their job" - as an ...
0
votes
0answers
45 views

Conditionality in p-value definition

Can we interpret p-value: $$P(T(x)>t\mid H_0:\mu=\mu_0)$$ in terms of usual conditional probability, which some random variable (T - test statistics) is conditioned on an event that some population ...
2
votes
1answer
122 views

Philosophy relationship between stochastic model and deterministic mechanism behind the problem

I have a question about the philosophy using probability/statistic theory to solve some real world problems that we don't have a fully understanding of its deterministic mechanism yet. For example, ...
5
votes
0answers
91 views

Does it make sense to speak of probability of a hidden outcome? (context: confidence intervals) [duplicate]

My question is concerning the frequentist approach to probability. Assume that you tossed a coin but don't see the outcome. Does it make sense to say that it shows head with a probability of 0.5? Or ...
8
votes
0answers
168 views

Is probability fundamentally about reference classes (real or imagined)?

Question: It seems that frequentism and Bayesianism may not really be different as far as the the ultimate basis for what a probability is (relative frequency within a reference class) - it's just ...
13
votes
6answers
1k views

Does there exist any univariate distribution that we can't sample from?

We have great variety of methods for random generation from univariate distributions (inverse transform, accept-reject, Metropolis-Hastings etc.) and it seems that we can sample from literally any ...
9
votes
3answers
727 views

When (and why) do Bayesians reject valid Bayesian methods? [closed]

From what I have read and from answers to other questions I have asked here, many so-called frequentist methods correspond mathematically (I don't care if they correspond philosophically, I only care ...
67
votes
10answers
8k views

Is there any *mathematical* basis for the Bayesian vs frequentist debate?

It says on Wikipedia that: the mathematics [of probability] is largely independent of any interpretation of probability. Question: Then if we want to be mathematically correct, shouldn't we ...
0
votes
1answer
76 views

What is Statistics? [closed]

I suggest to pause for a moment, sit back and think about this philosophical question. What are we doing when we are doing statistics? How to embed statistics into the landscape of modern science and ...
53
votes
9answers
3k views

What are some examples of anachronistic practices in statistics?

I am referring to practices that still maintain their presence, even though the problems (usually computational) they were designed to cope with have been mostly solved. For example, Yates' ...
3
votes
1answer
3k views

Intuition about f1 score

Let's say I have a data set where half of the data points are labelled as positive and half are labelled as negative. My task is to create a classifier which recognises when a sample from the dataset ...