Questions tagged [optimal]
For questions about optimality properties of statistical methods, such as optimal parameter estimation or optimal testing. Both for questions about optimality theory in general, and for questions about optimality properties of specific procedures.
65 questions
0
votes
0
answers
25
views
Are interior-point methods guranteed to converge to the global optimum of a convex objective function?
I am looking into convex optimization. However, I am not sure if there are interior-point methods that are guaranteed to converge to the globally optimal solution given either a strictly convex or a ...
- 101
0
votes
0
answers
25
views
Unbiased test for homogeneity of means of exponenential samples
Given $K$ independent samples of $Y_{i1},\dots,Y_{in_i} \ \text{i.i.d.} \ \sim Exp(\lambda_i)$ with $i=1,\dots, K$ and $n_i$ the size of the $i$-th sample, is there any statistics with analytically ...
1
vote
0
answers
69
views
UMP two sided tests for exponential families
Consider a random variable $X$ with density $$f(x : θ) = C(θ)e^{η(θ)T(x)}h(x), θ ∈ Θ$$.
Assume that $η(θ)$ is strictly increasing in $θ$ and that the family is full rank. Show that there will not be ...
- 217
1
vote
1
answer
48
views
How to combine a noisy (but unbiased) estimate with a precise (but possibly biased) estimate in A/B tests?
Suppose I want to estimate some set of unknown quantities $\theta_1$, …, $\theta_N$. For each $i \in \{1, …, N\}$, I have two estimators: $\hat{\theta_i}_A$ and $ \hat{\theta_i}_B$. The goal is to ...
- 1,447
3
votes
1
answer
318
views
Difference between Parsimonious model vs Optimal model
As per my understanding, parsimonious regression model is the model that has less variables but with those variables I can describe the data best. Is it so?
Then ...
user386962
3
votes
1
answer
104
views
Is there an optimality result for the two-sample Wilcoxon-Mann-Whitney test?
Is there any mathematical result that states that the Wilcoxon-Mann-Whitney (WMW) test is optimal in some sense, for a specific testing problem that is a subproblem of the general problem the WMW test ...
- 28.3k
0
votes
0
answers
80
views
General rule when max likelihood is suboptimal
Inspired by the question regarding Bessel's correction, I wonder whether there is a general rule regarding applicability of maximum likelihood for parameter estimation.
My guess is that the parameters ...
- 9,658
0
votes
0
answers
16
views
Sliding parameters over a ML model
I'm analysing the performance of an algorithm based on neural networks and I have to tune two parameters( number of past days to consider for the turbidity and volume of water flowing) by hand. So ...
- 11
2
votes
1
answer
4k
views
What does it mean for the Bayes Classifier to be optimal? [closed]
The Bayes classifier is always called the 'optimal' classifier. What does this actually mean? In particular does optimal mean the Bayes classifier will never make a mistake when predicting the label ...
- 21
5
votes
2
answers
258
views
What is the theoretical justification for alternatives to MSE minimisation?
I'm trying to wrap my head around the connection between statistical regression and its probability theoretical justification. In many books on statistics/machine learning, one is introduced to the ...
3
votes
1
answer
631
views
Optimal prediction under squared percentage loss
I have to find an answer on the following question but I am struggling:
Consider a leaf of a decision tree that consists of object-label pairs $(x_{1}, y_{1}), \dots, (x_{n}, y_{n})$.
The prediction $...
- 65
1
vote
0
answers
121
views
Optimal rate of convergence of nonparametric density estimators
Suppose that $X_1, X_2, \dots, X_n$ forms an independent and identically distributed sample from some $d$-dimensional probability distribution with unknown probability density function $f$. Let $x$ be ...
- 170
2
votes
0
answers
104
views
Optimal compressibility and PCA
I have a population $\mathcal{X}$ of $N$ samples extracted from a multivariate gaussian random variable $\mathbf{x} \in \mathbb{R}^d$. Let us define a transformation $f_{d\rightarrow r} (\mathbf{x}) = ...
- 31
2
votes
1
answer
23
views
Optimal Feature Engeneering creation: best optimization method?
basically I would like to solve this problem:
(1) say I have N features that I want to transform with a generic f(x, theta) ...
- 229
1
vote
1
answer
543
views
Question on Optimal predictors for the 0-1 loss function
The input $X \in \{0, 1\}$ and label $T \in \{0,1\}$ are binary random variables, and the set of predictors that we consider are the functions $y : \{0, 1\} \rightarrow \{0, 1\}$. Recall the $0$-$1$ ...
- 133
2
votes
1
answer
700
views
UMP test and non-decreasing power function
Let $\phi$ be a UMP test for $H_o: \theta \leq \theta_0$ and $H_1: \theta > \theta_0$. Let its power function, $E_\theta(\phi)$, be differentiable w.r.t. $\theta$.
Show the power function is non-...
- 121
1
vote
1
answer
135
views
Optimality in Clustering algorithms and an Optimality guaranteed clustering algorithm
So there are lots of clustering algorithms with different characteristics. What I am interested in now is a clustering algorithm which guarantees to find the optimal clustering result (if exists). And ...
- 21
2
votes
1
answer
243
views
Most powerful test for Hardy–Weinberg proportions
Consider a population with three kinds of individuals labeled $1, 2$, and $3$ occuring in the Hardy–Weinberg proportions $f(1,\theta)=\theta^2,f(2,\theta)=2\theta(1−\theta),f(3,\theta)=(1−\theta)^2$. ...
- 703
0
votes
1
answer
1k
views
Uniformly most powerful test does not exists
I am having tough time understanding this concept
The book says:
“We caution the reader that UMP tests for testing H0 : θ1 ≤ θ ≤ θ2 and
H0′ : θ = θ0 for the one-parameter exponential family do not ...
- 13
0
votes
0
answers
30
views
Optimal cut-off value for continuous covariate with binary outcome [duplicate]
Research question: What is the optimal cut-off value for a continuous covariate (age) with a binary outcome (suitable or not for a certain treatment). I realize that we lose information by ...
- 51
2
votes
1
answer
56
views
D-Optimality for regression of polynomial models in one variable with missing terms
Let's say I have a model that looks as follows:
$$y = x + ax^3 + bx^5 + cx^7 + dx^9$$
Given $n$ free choices for x as input measurements how can I determine which $x$'s I should input to best ...
- 31
3
votes
0
answers
161
views
Least favorable prior - Find the distribution that maximizes the Bayes risk
Suppose I've found that the Bayes risk is of the form
$$r(\theta) = \int_{-a}^a \theta^2 \pi(\theta)d\theta
$$
I want to show that the following distribution, $\pi(a)=\pi(-a)=0.5$, maximizes this ...
- 3,670
1
vote
0
answers
95
views
D-optimal design with nuisance parameters
I am a mechanical engineer trying to develope an optimal design of experiments in a problem with nuisance parameters. I would like to calculate the parameters $\mathbf{d}$ to optimally estimate ...
1
vote
1
answer
968
views
Can we always get an optimal $k$-means cluster arrangement?
I am currently studying $k$-means clustering. An optimal $k$-cluster arrangement is defined as follows:
Fix a distance $\Delta$ and $k < n$. Assume $\mathbb{X}$ have been partitioned into $k$ ...
- 2,204
1
vote
1
answer
28
views
Optimal combination of biased samplers
Suppose we are interested in the mean $\mu$ of a random variable $X$ but the only way to sample it is from known biased distributions $p_{\lambda}(x)$, such that $\left<{X}\right>_{\lambda} =\...
- 133
1
vote
0
answers
597
views
Identifying inflexion point in elbow method (cluster analysis)
I am looking for the optimal number of clusters to conduct a cluster analysis and used the following code to determine it:
...
- 377
0
votes
0
answers
44
views
Optimal point pairs for estimating the distance function in a metric space
I’m looking for the name of the methods or subfield that deals with problems like this:
Assume that we have $n \times m$ points over a regular grid.They are embedded into a metric space. We can get ...
- 175
4
votes
2
answers
197
views
Biased estimator obtained by optimal experiment design
I am using a model-based approach to infer the parameters of a given system. Namely, I represent my system by a model $\mathcal{M}$ with parameters $\theta$. To estimate the true value of $\theta$, I ...
- 2,931
1
vote
0
answers
60
views
Find distribution that minimises a function of its moments [closed]
Imagine a probability density function $f(x)$, defined for positive $x$, and let's note its $n$th non-centred moment $x_{n}$. The mean $x_{1}$ is fixed (and positive).
How can I find $f(x)$ that ...
- 111
2
votes
1
answer
112
views
How to understand the sufficient condition for global optimum for a constrained optimization probelm
How to understand the sufficient condition for global optimum? From my understanding, the global optimum should be 0 instead of $\geq 0$, where does this come from?
Consider the constrained ...
- 487
4
votes
0
answers
155
views
Is it possible to show that this estimator has minimum variance?
Doing some exercises I stumbled upon this tricky one:
Suppose we have an independent random sample $(X_1, ... , X_n)$ with $X_i \sim Poisson(\lambda)$. Define $\theta = e^{-\lambda}$.
Let $$ \...
0
votes
0
answers
51
views
Variance estimator that is optimal under absolute loss
Given a random i.i.d. sample from a population with a finite variance $\sigma^2<\infty$, what estimator of $\sigma^2$ is optimal under absolute loss?
$$
\arg\min_{\hat\sigma^{2}\in F}\mathbb{E}(|\...
- 69.5k
3
votes
1
answer
107
views
how to understand this math formula for bandwidth calculation?
I am reading a paper that uses the following equation to calculate the optimal bandwidth, however, I am confused about the position of "4" and "3" in the equation. is this a typo? or what does it mean?...
- 445
0
votes
0
answers
170
views
Explain asymptotic optimality property of AIC (Akaike information criterion)
I am trying to use AIC in my research, I know how to apply it and how to interpret its value, but I do not understand the asymptotic optimality property in general and why AIC has this property. Can ...
- 1
1
vote
0
answers
21
views
How to optimize a number of games with a limit set of second chances at guessing other options given probabilities?
Let's say I have a game with $M$ outcomes $\{O_i\}_{i=1}^{M}$ and I have a predictor $P$ that gives me probabilities for the outcomes of a game.
I now have $N$ independent games that I want to ...
- 123
1
vote
1
answer
115
views
D-optimal DOE suggest repeated samples
I tried to generate a D optimal design but the design output sounds very weird to me. I have a (real) process and I`d like to explore 3 factors, but the process have a lot of constraints so I provide ...
- 11
0
votes
0
answers
21
views
Finding the 'optimal distribution' with one unknown variable
I am trying to find the optimal distribution (of ingredients) with one unknown parameter (skillset).
Example, when baking a bread the ingredients are flour,salt,yeast and water. When I ask 100 ...
- 101
8
votes
3
answers
1k
views
Justification for and optimality of $R^2_{adj.}$ as a model selection criterion
In a recent thread, use of adjusted $R^2$ ($R^2_{adj.}$) is mentioned in the context of model selection, e.g.
The adjustment was invented as a solution to problems caused by variable selection
...
- 69.5k
1
vote
0
answers
62
views
What is the best algorithm so far for probabilistic adaptive group testing?
I am checking https://en.wikipedia.org/wiki/Group_testing#Classification_of_group-testing_problems but could not figure out the answer.
I am focusing on probabilistic adaptive group testing. The ...
- 1,017
0
votes
1
answer
481
views
Rejection sampling for optimal $\lambda$ and $a$
Suppose $f(x) \propto \exp ({-(x-u)^2\over2\sigma^2}) I_{X>=a}$ and we cannot compute the normalizing constant. Consider rejection sampling using proposal density of a shifted exponential ...
- 11
2
votes
0
answers
39
views
Hyper-parameters which minimize the variance of transformed multi-variate Guassian variable
Let $k < p$ be positive integers and $g: \mathbb R^k \rightarrow \mathbb R^p$ be a smooth Lipschitz continuous function. Let $y_1,\ldots, y_N \in \mathbb R^p$ and $a = (a_1,\ldots,a_N) \in \mathbb ...
- 538
7
votes
1
answer
129
views
how to sample data for regression that is the most informative?
Background
I have a unknown function
$$f(x_1, x_2)$$
But I have access to evaluate this function finite $L$ times,
$$y_j = f(x_1^j, x_2^j), j=1,\ldots,L $$
Then I have a model $\hat{f}$ which I ...
3
votes
0
answers
158
views
Is there a UMVUE for arbitrary distribution with density and variance?
Let F be the family of all distributions with probability density and finite variance, and $X_1, ..., X_n$ be random samples from F.
Does UMVUE for variance exists for this situation?
- 467
3
votes
1
answer
982
views
Is there a Fisher Information equivalent in MAP Empirical Bayes estimation?
Background
The Fisher information for a linear Gaussian model is
$\mathcal{I}_{\theta} = \frac{X X^T}{\sigma^2} $. This is used in optimal experiment design techniques, for example, maximisation of $|...
- 865
2
votes
0
answers
59
views
At which rank should I reasonably stop selecting interview candidates?
It seems that my problem could be classified as an "optimal stopping" problem, but I am unable to make much of this information. It is an important problem for my organisation (an NGO), in order to ...
- 21
5
votes
2
answers
1k
views
What's the difference between "Optimal linear predictor" and "best unbiased linear estimator"?
Greene (econometric analysis 7th ed. p 53) states that OLS is the "optimal linear predictor":
Then on the next page, he states that OLS is also the BLUE estimator (Gauss-Markov Theorem):
I ...
- 2,987
5
votes
1
answer
673
views
Estimator that is optimal under all sensible loss (evaluation) functions
Consider a probability distribution $D$ with a parameter $\theta$ and an i.i.d. sample $S$ from that distribution. I am interested in an estimator $\hat\theta(S)$ of $\theta$ that satisfies the ...
- 69.5k
1
vote
0
answers
44
views
Feature scaling and equivalence of the mean squared loss function optimal solutions
When applying the feature scaling, what is the mathematical proof that the optimum of the mean squared loss function is the same in both without-normalisation case and with-normalisation of the ...
- 189
13
votes
1
answer
1k
views
Does there exist an analogous statement to BLUE (Gauss-Markov) for GLMs?
I recall from my graduate school days that the Gauss-Markov (GM) theorem states that the Best Linear Unbiased Estimator (BLUE) in a linear regression is $\vec{\beta}=(X^TX)^{-1}X^T\vec{y}$. An amazing ...
- 4,369
1
vote
0
answers
27
views
Relationship between 4SID and 4DVAR
Background:
The "4SID" method allows one to back analytic models from data using a form similar to that of a Kalman filter. (1)
The 4DVAR is an approach to use data with an analytic model for weather ...
- 9,843