Skip to main content

Questions tagged [optimal]

For questions about optimality properties of statistical methods, such as optimal parameter estimation or optimal testing. Both for questions about optimality theory in general, and for questions about optimality properties of specific procedures.

Filter by
Sorted by
Tagged with
1 vote
0 answers
48 views

UMP two sided tests for exponential families

Consider a random variable $X$ with density $$f(x : θ) = C(θ)e^{η(θ)T(x)}h(x), θ ∈ Θ$$. Assume that $η(θ)$ is strictly increasing in $θ$ and that the family is full rank. Show that there will not be ...
user671269's user avatar
0 votes
0 answers
23 views

D-Optimal value by hand

I'm trying to compute the D-optimal value by hand (on R) Based on this video I could get the D value by ...
Lefty's user avatar
  • 479
1 vote
1 answer
24 views

How to combine a noisy (but unbiased) estimate with a precise (but possibly biased) estimate in A/B tests?

Suppose I want to estimate some set of unknown quantities $\theta_1$, …, $\theta_N$. For each $i \in \{1, …, N\}$, I have two estimators: $\hat{\theta_i}_A$ and $ \hat{\theta_i}_B$. The goal is to ...
frelk's user avatar
  • 1,357
3 votes
1 answer
190 views

Difference between Parsimonious model vs Optimal model

As per my understanding, parsimonious regression model is the model that has less variables but with those variables I can describe the data best. Is it so? Then ...
user avatar
3 votes
1 answer
93 views

Is there an optimality result for the two-sample Wilcoxon-Mann-Whitney test?

Is there any mathematical result that states that the Wilcoxon-Mann-Whitney (WMW) test is optimal in some sense, for a specific testing problem that is a subproblem of the general problem the WMW test ...
Christian Hennig's user avatar
0 votes
0 answers
76 views

General rule when max likelihood is suboptimal

Inspired by the question regarding Bessel's correction, I wonder whether there is a general rule regarding applicability of maximum likelihood for parameter estimation. My guess is that the parameters ...
Igor F.'s user avatar
  • 9,238
0 votes
0 answers
16 views

Sliding parameters over a ML model

I'm analysing the performance of an algorithm based on neural networks and I have to tune two parameters( number of past days to consider for the turbidity and volume of water flowing) by hand. So ...
X HOxha's user avatar
  • 11
2 votes
1 answer
3k views

What does it mean for the Bayes Classifier to be optimal? [closed]

The Bayes classifier is always called the 'optimal' classifier. What does this actually mean? In particular does optimal mean the Bayes classifier will never make a mistake when predicting the label ...
E. Turok's user avatar
4 votes
2 answers
207 views

What is the theoretical justification for alternatives to MSE minimisation?

I'm trying to wrap my head around the connection between statistical regression and its probability theoretical justification. In many books on statistics/machine learning, one is introduced to the ...
Othman El Hammouchi's user avatar
3 votes
1 answer
549 views

Optimal prediction under squared percentage loss

I have to find an answer on the following question but I am struggling: Consider a leaf of a decision tree that consists of object-label pairs $(x_{1}, y_{1}), \dots, (x_{n}, y_{n})$. The prediction $...
ghxk's user avatar
  • 65
1 vote
0 answers
98 views

Optimal rate of convergence of nonparametric density estimators

Suppose that $X_1, X_2, \dots, X_n$ forms an independent and identically distributed sample from some $d$-dimensional probability distribution with unknown probability density function $f$. Let $x$ be ...
lmaosome's user avatar
  • 140
2 votes
0 answers
101 views

Optimal compressibility and PCA

I have a population $\mathcal{X}$ of $N$ samples extracted from a multivariate gaussian random variable $\mathbf{x} \in \mathbb{R}^d$. Let us define a transformation $f_{d\rightarrow r} (\mathbf{x}) = ...
David Shor's user avatar
2 votes
1 answer
22 views

Optimal Feature Engeneering creation: best optimization method?

basically I would like to solve this problem: (1) say I have N features that I want to transform with a generic f(x, theta) ...
Asher11's user avatar
  • 229
1 vote
1 answer
517 views

Question on Optimal predictors for the 0-1 loss function

The input $X \in \{0, 1\}$ and label $T \in \{0,1\}$ are binary random variables, and the set of predictors that we consider are the functions $y : \{0, 1\} \rightarrow \{0, 1\}$. Recall the $0$-$1$ ...
xxxxxx's user avatar
  • 133
2 votes
1 answer
606 views

UMP test and non-decreasing power function

Let $\phi$ be a UMP test for $H_o: \theta \leq \theta_0$ and $H_1: \theta > \theta_0$. Let its power function, $E_\theta(\phi)$, be differentiable w.r.t. $\theta$. Show the power function is non-...
William Ambrose's user avatar
1 vote
1 answer
126 views

Optimality in Clustering algorithms and an Optimality guaranteed clustering algorithm

So there are lots of clustering algorithms with different characteristics. What I am interested in now is a clustering algorithm which guarantees to find the optimal clustering result (if exists). And ...
Erik's user avatar
  • 21
2 votes
1 answer
234 views

Most powerful test for Hardy–Weinberg proportions

Consider a population with three kinds of individuals labeled $1, 2$, and $3$ occuring in the Hardy–Weinberg proportions $f(1,\theta)=\theta^2,f(2,\theta)=2\theta(1−\theta),f(3,\theta)=(1−\theta)^2$. ...
statwoman's user avatar
  • 663
0 votes
1 answer
1k views

Uniformly most powerful test does not exists

I am having tough time understanding this concept The book says: “We caution the reader that UMP tests for testing H0 : θ1 ≤ θ ≤ θ2 and H0′ : θ = θ0 for the one-parameter exponential family do not ...
User0405's user avatar
0 votes
0 answers
30 views

Optimal cut-off value for continuous covariate with binary outcome [duplicate]

Research question: What is the optimal cut-off value for a continuous covariate (age) with a binary outcome (suitable or not for a certain treatment). I realize that we lose information by ...
Daan's user avatar
  • 51
2 votes
1 answer
52 views

D-Optimality for regression of polynomial models in one variable with missing terms

Let's say I have a model that looks as follows: $$y = x + ax^3 + bx^5 + cx^7 + dx^9$$ Given $n$ free choices for x as input measurements how can I determine which $x$'s I should input to best ...
Sadikov's user avatar
  • 31
3 votes
0 answers
151 views

Least favorable prior - Find the distribution that maximizes the Bayes risk

Suppose I've found that the Bayes risk is of the form $$r(\theta) = \int_{-a}^a \theta^2 \pi(\theta)d\theta $$ I want to show that the following distribution, $\pi(a)=\pi(-a)=0.5$, maximizes this ...
Maverick Meerkat's user avatar
1 vote
0 answers
94 views

D-optimal design with nuisance parameters

I am a mechanical engineer trying to develope an optimal design of experiments in a problem with nuisance parameters. I would like to calculate the parameters $\mathbf{d}$ to optimally estimate ...
Juan Fernandez's user avatar
1 vote
1 answer
900 views

Can we always get an optimal $k$-means cluster arrangement?

I am currently studying $k$-means clustering. An optimal $k$-cluster arrangement is defined as follows: Fix a distance $\Delta$ and $k < n$. Assume $\mathbb{X}$ have been partitioned into $k$ ...
The Pointer's user avatar
  • 1,942
1 vote
1 answer
28 views

Optimal combination of biased samplers

Suppose we are interested in the mean $\mu$ of a random variable $X$ but the only way to sample it is from known biased distributions $p_{\lambda}(x)$, such that $\left<{X}\right>_{\lambda} =\...
Godzilla's user avatar
  • 133
1 vote
0 answers
577 views

Identifying inflexion point in elbow method (cluster analysis)

I am looking for the optimal number of clusters to conduct a cluster analysis and used the following code to determine it: ...
Catarina Toscano's user avatar
0 votes
0 answers
43 views

Optimal point pairs for estimating the distance function in a metric space

I’m looking for the name of the methods or subfield that deals with problems like this: Assume that we have $n \times m$ points over a regular grid.They are embedded into a metric space. We can get ...
rozsasarpi's user avatar
4 votes
2 answers
192 views

Biased estimator obtained by optimal experiment design

I am using a model-based approach to infer the parameters of a given system. Namely, I represent my system by a model $\mathcal{M}$ with parameters $\theta$. To estimate the true value of $\theta$, I ...
Camille Gontier's user avatar
1 vote
0 answers
56 views

Find distribution that minimises a function of its moments [closed]

Imagine a probability density function $f(x)$, defined for positive $x$, and let's note its $n$th non-centred moment $x_{n}$. The mean $x_{1}$ is fixed (and positive). How can I find $f(x)$ that ...
user655870's user avatar
2 votes
1 answer
106 views

How to understand the sufficient condition for global optimum for a constrained optimization probelm

How to understand the sufficient condition for global optimum? From my understanding, the global optimum should be 0 instead of $\geq 0$, where does this come from? Consider the constrained ...
FantasticAI's user avatar
4 votes
0 answers
154 views

Is it possible to show that this estimator has minimum variance?

Doing some exercises I stumbled upon this tricky one: Suppose we have an independent random sample $(X_1, ... , X_n)$ with $X_i \sim Poisson(\lambda)$. Define $\theta = e^{-\lambda}$. Let $$ \...
Simone Vernengo's user avatar
0 votes
0 answers
48 views

Variance estimator that is optimal under absolute loss

Given a random i.i.d. sample from a population with a finite variance $\sigma^2<\infty$, what estimator of $\sigma^2$ is optimal under absolute loss? $$ \arg\min_{\hat\sigma^{2}\in F}\mathbb{E}(|\...
Richard Hardy's user avatar
3 votes
1 answer
95 views

how to understand this math formula for bandwidth calculation?

I am reading a paper that uses the following equation to calculate the optimal bandwidth, however, I am confused about the position of "4" and "3" in the equation. is this a typo? or what does it mean?...
flashing sweep's user avatar
0 votes
0 answers
160 views

Explain asymptotic optimality property of AIC (Akaike information criterion)

I am trying to use AIC in my research, I know how to apply it and how to interpret its value, but I do not understand the asymptotic optimality property in general and why AIC has this property. Can ...
useruser's user avatar
1 vote
0 answers
18 views

How to optimize a number of games with a limit set of second chances at guessing other options given probabilities?

Let's say I have a game with $M$ outcomes $\{O_i\}_{i=1}^{M}$ and I have a predictor $P$ that gives me probabilities for the outcomes of a game. I now have $N$ independent games that I want to ...
Reed Richards's user avatar
1 vote
1 answer
108 views

D-optimal DOE suggest repeated samples

I tried to generate a D optimal design but the design output sounds very weird to me. I have a (real) process and I`d like to explore 3 factors, but the process have a lot of constraints so I provide ...
Rodrigo PG's user avatar
0 votes
0 answers
21 views

Finding the 'optimal distribution' with one unknown variable

I am trying to find the optimal distribution (of ingredients) with one unknown parameter (skillset). Example, when baking a bread the ingredients are flour,salt,yeast and water. When I ask 100 ...
Arjen's user avatar
  • 101
8 votes
3 answers
946 views

Justification for and optimality of $R^2_{adj.}$ as a model selection criterion

In a recent thread, use of adjusted $R^2$ ($R^2_{adj.}$) is mentioned in the context of model selection, e.g. The adjustment was invented as a solution to problems caused by variable selection ...
Richard Hardy's user avatar
1 vote
0 answers
60 views

What is the best algorithm so far for probabilistic adaptive group testing?

I am checking https://en.wikipedia.org/wiki/Group_testing#Classification_of_group-testing_problems but could not figure out the answer. I am focusing on probabilistic adaptive group testing. The ...
mommomonthewind's user avatar
0 votes
1 answer
458 views

Rejection sampling for optimal $\lambda$ and $a$

Suppose $f(x) \propto \exp ({-(x-u)^2\over2\sigma^2}) I_{X>=a}$ and we cannot compute the normalizing constant. Consider rejection sampling using proposal density of a shifted exponential ...
user593721's user avatar
2 votes
0 answers
38 views

Hyper-parameters which minimize the variance of transformed multi-variate Guassian variable

Let $k < p$ be positive integers and $g: \mathbb R^k \rightarrow \mathbb R^p$ be a smooth Lipschitz continuous function. Let $y_1,\ldots, y_N \in \mathbb R^p$ and $a = (a_1,\ldots,a_N) \in \mathbb ...
dohmatob's user avatar
  • 558
7 votes
1 answer
126 views

how to sample data for regression that is the most informative?

Background I have a unknown function $$f(x_1, x_2)$$ But I have access to evaluate this function finite $L$ times, $$y_j = f(x_1^j, x_2^j), j=1,\ldots,L $$ Then I have a model $\hat{f}$ which I ...
ArtificiallyIntelligent's user avatar
3 votes
0 answers
157 views

Is there a UMVUE for arbitrary distribution with density and variance?

Let F be the family of all distributions with probability density and finite variance, and $X_1, ..., X_n$ be random samples from F. Does UMVUE for variance exists for this situation?
456 123's user avatar
  • 457
3 votes
1 answer
942 views

Is there a Fisher Information equivalent in MAP Empirical Bayes estimation?

Background The Fisher information for a linear Gaussian model is $\mathcal{I}_{\theta} = \frac{X X^T}{\sigma^2} $. This is used in optimal experiment design techniques, for example, maximisation of $|...
boomkin's user avatar
  • 855
2 votes
0 answers
54 views

At which rank should I reasonably stop selecting interview candidates?

It seems that my problem could be classified as an "optimal stopping" problem, but I am unable to make much of this information. It is an important problem for my organisation (an NGO), in order to ...
maqueiouseur's user avatar
5 votes
2 answers
1k views

What's the difference between "Optimal linear predictor" and "best unbiased linear estimator"?

Greene (econometric analysis 7th ed. p 53) states that OLS is the "optimal linear predictor": Then on the next page, he states that OLS is also the BLUE estimator (Gauss-Markov Theorem): I ...
user56834's user avatar
  • 2,819
5 votes
1 answer
642 views

Estimator that is optimal under all sensible loss (evaluation) functions

Consider a probability distribution $D$ with a parameter $\theta$ and an i.i.d. sample $S$ from that distribution. I am interested in an estimator $\hat\theta(S)$ of $\theta$ that satisfies the ...
Richard Hardy's user avatar
1 vote
0 answers
44 views

Feature scaling and equivalence of the mean squared loss function optimal solutions

When applying the feature scaling, what is the mathematical proof that the optimum of the mean squared loss function is the same in both without-normalisation case and with-normalisation of the ...
moth's user avatar
  • 189
13 votes
1 answer
1k views

Does there exist an analogous statement to BLUE (Gauss-Markov) for GLMs?

I recall from my graduate school days that the Gauss-Markov (GM) theorem states that the Best Linear Unbiased Estimator (BLUE) in a linear regression is $\vec{\beta}=(X^TX)^{-1}X^T\vec{y}$. An amazing ...
Lucas Roberts's user avatar
1 vote
0 answers
27 views

Relationship between 4SID and 4DVAR

Background: The "4SID" method allows one to back analytic models from data using a form similar to that of a Kalman filter. (1) The 4DVAR is an approach to use data with an analytic model for weather ...
EngrStudent's user avatar
  • 9,415
1 vote
0 answers
99 views

D-optimal mixture design- poor evaluation of design

My problem is related to the design evaluation of my set of experiments. I am trying to design a mixture of 4 materials, each having certain minimum and maximum dosage constraints. To come up with a ...
Nilz's user avatar
  • 11