Questions tagged [decision-theory]

Decision theory is the science of making optimal decisions in the face of uncertainty. Statistical decision theory is concerned with the making of decisions when in the presence of statistical knowledge (data) which sheds light on some of the uncertainties involved in the decision problem.

Filter by
Sorted by
Tagged with
147
votes
30answers
29k views

The Sleeping Beauty Paradox

The situation Some researchers would like to put you to sleep. Depending on the secret toss of a fair coin, they will briefly awaken you either once (Heads) or twice (Tails). After each waking, they ...
12
votes
2answers
9k views

What are complete sufficient statistics?

I have some trouble understanding complete sufficient statistics? Let $T=\Sigma x_i$ be a sufficient statistic. If $E[g(T)]=0$ with probability 1, for some function $g$, then it is a complete ...
20
votes
2answers
2k views

What is the decision-theoretic justification for Bayesian credible interval procedures?

(To see why I wrote this, check the comments below my answer to this question.) Type III errors and statistical decision theory Giving the right answer to the wrong question is sometimes called a ...
16
votes
2answers
11k views

Understanding the Bayes risk

When evaluating an estimator, the two probably most common used criteria are the maximum risk and the Bayes risk. My question refers to the latter one: The bayes risk under the prior $\pi$ is defined ...
3
votes
1answer
847 views

How to make optimal decisions with uncertain outcomes: achieving a “Yahtzee”

The game of Yahtzee is a poker-like game played with dice. Each move consists of three rolls of five (ordinary, fair, six-sided) dice. After each of the first two rolls the player may designate any ...
2
votes
1answer
27k views

Decision tree model evaluation for “training set ” vs “testing set ” in R

So I got my training set with 70% of my data called "train" / 30% "test" I use ctree to get my decision tree model with something like this code below : ...
8
votes
1answer
2k views

What is a loss function in decision theory?

My notes define a loss function as the 'cost' incurred when the true value of $\theta$ is estimated by $\hat\theta$. What kind of cost is it talking about? monetary cost? or is it something related to ...
18
votes
4answers
751 views

Under which conditions do Bayesian and frequentist point estimators coincide?

With a flat prior, the ML (frequentist -- maximum likelihood) and the MAP (Bayesian -- maximum a posteriori) estimators coincide. More generally, however, I'm talking about point estimators derived as ...
10
votes
3answers
621 views

MAP is a solution to $L(\theta) = \mathcal{I}[\theta \ne \theta^{*}]$

I have come across these slides (slide # 16 & #17) in one of the online courses. The instructor was trying to explain how Maximum Posterior Estimate(MAP) is actually the solution $L(\theta) = \...
8
votes
5answers
538 views

Why care so much about expected utility?

I have a naive question about decision theory. We calculate the probabilities of various outcomes assuming particular decisions and assign utilities or costs to each outcome. We find the optimal ...
5
votes
2answers
565 views

Drawing numbered balls from an urn

PROBLEM There is an urn with a set of balls where each ball is labeled with a different integer. The numbers on the balls are known and are not a range of integers. For example the set of balls could ...
10
votes
2answers
3k views

Aside from Durbin-Watson, what hypothesis tests can produce inconclusive results?

The Durbin-Watson test statistic can lie in an inconclusive region, where it is not possible either to reject or fail to reject the null hypothesis (in this case, of zero autocorrelation). What other ...
6
votes
2answers
839 views

Is a loss function the flip side of a coin to a utility function, or are they not related?

I'm trying to get a grasp on utility and loss functions, and at first I thought that a utility function was the flipside of a loss function and vice versa. Kind of like how if you know the probability ...
3
votes
1answer
681 views

A constant as an admissible estimator

This is a homework question so I would appreciate hints. I believe I have the first part correct, but I fail to see how the second part is different. Assume square error loss, $L(\theta ,a)=(\theta -...
8
votes
1answer
818 views

What is the relation between statistics theory and decision theory?

I was wondering how statistics and decision theory are related? It looks to me all the statistics problems/tasks can be formulated in decision theory. Also problems in decision theory can be ...
10
votes
1answer
2k views

Different definitions of Bayes risk

I'm having trouble understanding the proper definition of Bayes risk. Let the data/variate $x \sim P(X|\theta)$, $\theta\in \Theta$, $\pi$ be a distribution on $\Theta$ (prior), $\hat \theta(x)$ be ...
5
votes
2answers
2k views

Does Bayesian Statistics have no concept of statistical hypothesis testing?

I was told that the framework of Bayesian Statistics has no concept of statistical hypothesis testing or confidence intervals. How does this make sense? Bayesian statistics only says that we ...
9
votes
3answers
409 views

Does a Bayes estimator require that the true parameter is a possible variate of the prior?

This might be a bit of a philosophical question, but here we go: In decision theory, the risk of a Bayes estimator $\hat\theta(x)$ for $\theta\in\Theta$ is defined with respect to a prior distribution ...
4
votes
1answer
2k views

Quadratic loss function implying conditional expectation

I am reading Bishop's pattern recognition book. In the decision theory part he first derives that using a quadratic loss function implies that our estimate $y(x)$ should be the conditional expectation ...
10
votes
3answers
1k views

How do I choose the best metric to measure my calibration?

I program and do test-driven development. After I made a change in my code I run my tests. Sometimes they succeed and sometimes they fail. Before I run a test I write down a number from 0.01 to 0.99 ...
6
votes
1answer
3k views

How do I combine multiple prior components and a likelihood?

Lets imagine I am comparing two groups of animals (treatment/control). There is previous data from cell cultures indicating the treatment should have a positive effect. This gives me "prior component ...
5
votes
1answer
393 views

Loss function that relates ROPE with HDI?

In Doing Bayesian Data Analysis (link to the book) and Bayesian Estimation Supersedes the t-Test, J. Kruschke proposes using the following criterion to reject or accept the null hypothesis in a ...
9
votes
1answer
153 views

What problem or game are variance and standard deviation optimal solutions for?

For a given random variable (or a population, or a stochastic process), mathematical expectation is the answer to a question What point forecast minimizes the expected square loss?. Also, it is the ...
5
votes
0answers
367 views

Why can't the complete class theorem be easily generalized to all locally-compact spaces?

So I was reading Christian P. Robert's The Bayesian Choice, going through the constellation of results related to complete class theorems, and I don't see why all of them are necessary. In particular, ...
7
votes
2answers
503 views

Does a density forecast add value beyond a point forecast when the loss function is given?

Density forecasts are more universal than point forecasts; they provide information on the whole predicted distribution of a random variable rather than on a concrete function thereof (such as ...
4
votes
2answers
617 views

Minimizing expected brier score and Brier score interpretation

For a probabilistic binary forecast, the BS (Brier score) is given by $$ \text{BS}= \begin{cases} (1-f_i)^2\\ f_i^2\\ \end{cases} $$ Where $f$ is the forecast. If the event occurs with probability $...
3
votes
0answers
128 views

What is a robust way to find the max of $n$ independent, non-identical random variates?

Suppose I observe $n$ random variates along with their variance (but not mean) and I'd like to select the one with the largest mean as frequently as possible. The procedure must be memoryless--you ...
3
votes
1answer
175 views

Classification optimal decisions considering a loss function

Suppose we're given data from three different classes which are normally distributed with the following means and variances: $C_1: \mu_1=(1,2)^T, \Sigma_1^{-1}=( \begin{array}{ccc}2 & 1 \\1 &...
2
votes
2answers
656 views

Is summing posterior probabilities valid for classification problems?

A classification for two mutually exclusive problem can be formulated by having a decision hinge on whether $P_0(x) > P_1(x)$ or $P_0(x) < P_1(x)$ where $P_0(x)$ and $P_1(x)$ are posterior ...
2
votes
1answer
716 views

admissibility of bayes rule

How to show that for a binomial(n, p) distribution, the MLE X/n is admissible under square error loss? The Bayes rule undr square error loss with beta($\alpha, \beta$) prior is X+$\alpha$/ (n +$\...
1
vote
1answer
90 views

Optimal classification rule given data, model and loss function

Setup Suppose I have a data set with a categorical variable $Y$ (with possible values $j=1,\dots,J$) and another variable $X$. I wish to classify $Y$ based on the information in $X$. For simplicity, ...
1
vote
0answers
51 views

Optimal decisions based on frequentist estimators

Consider a decision problem aimed at minimizing the expected loss1 where the argument is a parameter estimate. In a Bayesian setting, given a posterior distribution of the parameter and the loss ...
1
vote
1answer
219 views

Derivation of Bayes classifier in Murphy's book

I am reading Kevin Murphy's Machine Learning book (MLAPP, 1st printing) and want to know how he got the expression for the Bayes classifier using minimization of the posterior expected loss. He wrote ...
0
votes
0answers
30 views

Expected utility maximization when beliefs are inaccurate

In the framework of maximization of expected utility (MEU), is it somehow optimal or justifiable to make choices based on the subjective probability distribution when we know it may be inaccurate (...