Questions tagged [decision-theory]
Decision theory is the science of making optimal decisions in the face of uncertainty. Statistical decision theory is concerned with the making of decisions when in the presence of statistical knowledge (data) which sheds light on some of the uncertainties involved in the decision problem.
45
questions
165
votes
34
answers
34k
views
The Sleeping Beauty Paradox
The situation
Some researchers would like to put you to sleep. Depending on the secret toss of a fair coin, they will briefly awaken you either once (Heads) or twice (Tails). After each waking, they ...
- 309k
12
votes
5
answers
1k
views
Why care so much about expected utility?
I have a naive question about decision theory. We calculate the probabilities of various outcomes assuming particular decisions and assign utilities or costs to each outcome. We find the optimal ...
- 1,334
14
votes
1
answer
12k
views
What are complete sufficient statistics?
I have some trouble understanding complete sufficient statistics?
Let $T=\Sigma x_i$ be a sufficient statistic.
If $E[g(T)]=0$ with probability 1, for some function $g$, then it is a complete ...
- 906
3
votes
1
answer
1k
views
How to make optimal decisions with uncertain outcomes: achieving a "Yahtzee"
The game of Yahtzee is a poker-like game played with dice. Each move consists of three rolls of five (ordinary, fair, six-sided) dice. After each of the first two rolls the player may designate any ...
- 31
21
votes
2
answers
21k
views
Understanding the Bayes risk
When evaluating an estimator, the two probably most common used criteria are the maximum risk and the Bayes risk. My question refers to the latter one:
The bayes risk under the prior $\pi$ is defined ...
- 439
20
votes
2
answers
2k
views
What is the decision-theoretic justification for Bayesian credible interval procedures?
(To see why I wrote this, check the comments below my answer to this question.)
Type III errors and statistical decision theory
Giving the right answer to the wrong question is sometimes called a Type ...
- 2,808
10
votes
1
answer
3k
views
What is a loss function in decision theory?
My notes define a loss function as the 'cost' incurred when the true value of $\theta$ is estimated by $\hat\theta$. What kind of cost is it talking about? monetary cost? or is it something related to ...
- 929
6
votes
2
answers
1k
views
Is a loss function the flip side of a coin to a utility function, or are they not related?
I'm trying to get a grasp on utility and loss functions, and at first I thought that a utility function was the flipside of a loss function and vice versa. Kind of like how if you know the probability ...
5
votes
1
answer
1k
views
A constant as an admissible estimator
This is a homework question so I would appreciate hints. I believe I have the first part correct, but I fail to see how the second part is different.
Assume square error loss, $L(\theta ,a)=(\theta -...
- 1,402
3
votes
1
answer
256
views
Is a constant ever inadmissible?
For now, assume square loss. Let's estimate some parameter $\theta$, such as $\theta = \mu$ in $N(\mu, 1)$.
Is there ever a case where there is no such $c$ to make $\hat{\theta} = c$ an admissible ...
- 51.2k
19
votes
4
answers
1k
views
Under which conditions do Bayesian and frequentist point estimators coincide?
With a flat prior, the ML (frequentist -- maximum likelihood) and the MAP (Bayesian -- maximum a posteriori) estimators coincide.
More generally, however, I'm talking about point estimators derived as ...
- 832
13
votes
3
answers
1k
views
MAP is a solution to $L(\theta) = \mathcal{I}[\theta \ne \theta^{*}]$
I have come across these slides (slide # 16 & #17) in one of the online courses. The instructor was trying to explain how Maximum Posterior Estimate(MAP) is actually the solution $L(\theta) = \...
- 1,542
12
votes
1
answer
4k
views
Different definitions of Bayes risk
I'm having trouble understanding the proper definition of Bayes risk.
Let the data/variate $x \sim P(X|\theta)$, $\theta\in \Theta$, $\pi$ be a distribution on $\Theta$ (prior), $\hat \theta(x)$ be ...
- 395
10
votes
2
answers
4k
views
Aside from Durbin-Watson, what hypothesis tests can produce inconclusive results?
The Durbin-Watson test statistic can lie in an inconclusive region, where it is not possible either to reject or fail to reject the null hypothesis (in this case, of zero autocorrelation).
What other ...
- 22.1k
9
votes
1
answer
221
views
What problem or game are variance and standard deviation optimal solutions for?
For a given random variable (or a population, or a stochastic process), mathematical expectation is the answer to a question What point forecast minimizes the expected square loss?. Also, it is the ...
- 62.3k
8
votes
1
answer
998
views
What is the relation between statistics theory and decision theory?
I was wondering how statistics and decision theory are related?
It looks to me all the statistics problems/tasks can be formulated in decision theory. Also problems in decision theory can be ...
- 18.6k
7
votes
2
answers
867
views
Does a density forecast add value beyond a point forecast when the loss function is given?
Density forecasts are more universal than point forecasts; they provide information on the whole predicted distribution of a random variable rather than on a concrete function thereof (such as ...
- 62.3k
6
votes
2
answers
805
views
Drawing numbered balls from an urn
PROBLEM
There is an urn with a set of balls where each ball is labeled with a different integer. The numbers on the balls are known and are not a range of integers. For example the set of balls could ...
- 521
3
votes
1
answer
2k
views
Bayes estimate with weighted square error loss
First, let $T(x)$ be an estimator of $g(\theta)$ and assume we have a square error loss function defined as
$$L[g(\theta),T(x)]=[g(\theta)-T(x)]^2$$
Then the posterior expected risk of $T$ is
$$\...
- 609
2
votes
1
answer
28k
views
Decision tree model evaluation for "training set " vs "testing set " in R
So I got my training set with 70% of my data called "train" / 30% "test"
I use ctree to get my decision tree model with something like this code below :
...
- 205
14
votes
1
answer
467
views
Is $\frac1{n+1}\sum_{i=1}^n(X_i-\overline X)^2$ an admissible estimator for $\sigma^2$?
Consider a sample $X_1,X_2,\ldots,X_n$ from a univariate $N(\mu,\sigma^2)$ distribution where $\mu,\sigma^2$ are both unknown. Then it is known that under squared error loss, the sample variance $s^2=\...
- 10.3k
11
votes
3
answers
2k
views
How do I choose the best metric to measure my calibration?
I program and do test-driven development. After I made a change in my code I run my tests. Sometimes they succeed and sometimes they fail. Before I run a test I write down a number from 0.01 to 0.99 ...
- 1,009
10
votes
1
answer
402
views
Why is Wald's decision theory not universally recognized as the foundation of statistics?
This is somewhat ill-defined, but: Why is Wald's decision theory not universally recognized as the foundation of statistics? I gather (or maybe I infer) that it was formulated to put frequentist and ...
- 571
9
votes
4
answers
895
views
Loss functions in statistical decision theory vs. machine learning?
I'm quite familiar with loss functions in machine learning, but am struggling to connect them to loss functions in statistical decision theory [1].
In machine learning, a loss function is usually only ...
- 971
9
votes
3
answers
614
views
Does a Bayes estimator require that the true parameter is a possible variate of the prior?
This might be a bit of a philosophical question, but here we go: In decision theory, the risk of a Bayes estimator $\hat\theta(x)$ for $\theta\in\Theta$ is defined with respect to a prior distribution ...
- 395
8
votes
1
answer
4k
views
How do I combine multiple prior components and a likelihood?
Lets imagine I am comparing two groups of animals (treatment/control). There is previous data from cell cultures indicating the treatment should have a positive effect. This gives me "prior component ...
- 1,921
5
votes
1
answer
419
views
Loss function that relates ROPE with HDI?
In Doing Bayesian Data Analysis (link to the book) and Bayesian Estimation Supersedes the t-Test, J. Kruschke proposes using the following criterion to reject or accept the null hypothesis in a ...
- 18.6k
5
votes
0
answers
563
views
Why can't the complete class theorem be easily generalized to all locally-compact spaces?
So I was reading Christian P. Robert's The Bayesian Choice, going through the constellation of results related to complete class theorems, and I don't see why all of them are necessary. In particular, ...
- 283
5
votes
2
answers
2k
views
Does Bayesian Statistics have no concept of statistical hypothesis testing?
I was told that the framework of Bayesian Statistics has no concept of statistical hypothesis testing or confidence intervals.
How does this make sense? Bayesian statistics only says that we ...
user46925
5
votes
1
answer
199
views
What is a robust way to find the max of $n$ independent, non-identical random variates?
Suppose I observe $n$ random variates along with their variance (but not mean) and I'd like to select the one with the largest mean as frequently as possible. The procedure must be memoryless--you ...
- 202
4
votes
1
answer
309
views
James-Stein estimator with multiple samples
Let $X_1, \dots, X_n \in \mathbb{R}^p$ be i.i.d. samples from the $p$-normal distribution $N(\theta, \tau^2 I)$. Suppose we are interested in estimating $\theta$ with known variance $\tau^2$. Take the ...
- 433
4
votes
2
answers
958
views
Minimizing expected brier score and Brier score interpretation
For a probabilistic binary forecast, the BS (Brier score) is given by
$$
\text{BS}=
\begin{cases}
(1-f_i)^2\\
f_i^2\\
\end{cases}
$$
Where $f$ is the forecast. If the event occurs with probability $...
- 1,035
4
votes
1
answer
196
views
Classification optimal decisions considering a loss function
Suppose we're given data from three different classes which are normally distributed with the following means and variances:
$C_1: \mu_1=(1,2)^T, \Sigma_1^{-1}=( \begin{array}{ccc}2 & 1 \\1 &...
- 835
4
votes
1
answer
3k
views
Quadratic loss function implying conditional expectation
I am reading Bishop's pattern recognition book. In the decision theory part he first derives that using a quadratic loss function implies that our estimate $y(x)$ should be the conditional expectation ...
- 165
4
votes
1
answer
988
views
Uniform random variables and optimal strategy
This comes from Fivethirtyeight's riddler weekly challenge...
Toddler poker is played by two players. Each is dealt a “card,” which
is actually a number randomly chosen uniformly from the ...
- 31.3k
3
votes
1
answer
146
views
Machine learning methods for exploring relationships for a continuous response variable
I would like to explore a model to predict the value of a continuous response variable, from a set (around 100) of explanatory variables. I do not want to apply PCA like feature reduction, because I ...
- 2,021
2
votes
2
answers
1k
views
Is summing posterior probabilities valid for classification problems?
A classification for two mutually exclusive problem can be formulated by having a decision hinge on whether $P_0(x) > P_1(x)$ or $P_0(x) < P_1(x)$ where $P_0(x)$ and $P_1(x)$ are posterior ...
- 41
2
votes
1
answer
700
views
Derivation of Bayes classifier in Murphy's book
I am reading Kevin Murphy's Machine Learning book (MLAPP, 1st printing) and want to know how he got the expression for the Bayes classifier using minimization of the posterior expected loss.
He wrote ...
- 926
2
votes
0
answers
71
views
How to Build a Model with Correlation / Statistical Dependency for Bayesian A / B Testing
I use the Beta Binomial model for A/B testing.
I wonder if there a way to build a model in PyMC which models correlation between the conversion rate of group A with ...
- 165
2
votes
1
answer
947
views
admissibility of bayes rule
How to show that for a binomial(n, p) distribution, the MLE X/n is admissible under square error loss?
The Bayes rule undr square error loss with beta($\alpha, \beta$) prior is X+$\alpha$/ (n +$\...
- 131
2
votes
1
answer
328
views
Relationship between "Logistic regression + L1 regularization" and PCA
This is the experiment I have done.
My data contain several hundreds of samples but with over 20k features per sample, so I used logistic regression + L1 regularization (LR+L1) to fit a linear ...
- 1,097
1
vote
0
answers
51
views
A Proper Conjugate Model for A/B Test for Revenue per Click (RPC)
What would be a proper Conjugate Posterior model for Earning / Revenue per Click in A/B test?
The data is the total number of visitors and the total revenue per day per variant (A and B).
What are the ...
- 367
1
vote
0
answers
66
views
Optimal decisions based on frequentist estimators
Consider a decision problem aimed at minimizing the expected loss1 where the argument is a parameter estimate. In a Bayesian setting, given a posterior distribution of the parameter and the loss ...
- 62.3k
1
vote
1
answer
177
views
Optimal classification rule given data, model and loss function
Setup
Suppose I have a data set with a categorical variable $Y$ (with possible values $j=1,\dots,J$) and another variable $X$. I wish to classify $Y$ based on the information in $X$.
For simplicity, ...
- 62.3k
0
votes
0
answers
34
views
Expected utility maximization when beliefs are inaccurate
In the framework of maximization of expected utility (MEU), is it somehow optimal or justifiable to make choices based on the subjective probability distribution when we know it may be inaccurate (...
- 62.3k