Questions tagged [optimal]
For questions about optimality properties of statistical methods, such as optimal parameter estimation or optimal testing. Both for questions about optimality theory in general, and for questions about optimality properties of specific procedures.
61
questions
1
vote
0
answers
44
views
From inputs, updating $f\left(X_{i},Y_{i}\right)$ and $g\left(X_{i},Y_{i}\right)$ much other than $\overline{X_{i}}$ and $\overline{Y_{i}}$ to compare [closed]
(Premier League $2019$/$20$). The season was affected by the COVID–$19$ Pandemic while each team had a quarter of their schedule left (I call it $1/4$ because $1/2$ teams had $4$ away matches, and $5$ ...
- 113
0
votes
0
answers
14
views
Optimal MSPE predictor, estimating equations
I am reading through Bickels and Doksum's Mathematical Statistics book, and in the second chapter they give justification for the use of a minimum contrast estimator:
However, I do not quite ...
- 159
1
vote
1
answer
54
views
What does it mean for the Bayes Classifier to be optimal? [closed]
The Bayes classifier is always called the 'optimal' classifier. What does this actually mean? In particular does optimal mean the Bayes classifier will never make a mistake when predicting the label ...
- 11
0
votes
0
answers
21
views
Optimality conditions for the LASSO
In this paper, on page 1122, it states that the optimality conditions for the LASSO give
$\hat{\beta} = n_{\lambda}(\hat{\beta} - X^T(X\hat{\beta} - y))$, where $n_\lambda$ is the soft-thresholding ...
- 1
1
vote
0
answers
43
views
What is the theoretical justification for alternatives to MSE minimisation?
I'm trying to wrap my head around the connection between statistical regression and its probability theoretical justification. In many books on statistics/machine learning, one is introduced to the ...
0
votes
0
answers
7
views
Method to find optimal settings given an output metric
I am trying to optimize a screening problem that is usual done through brute force. I am trying to expand my skill set but I don't know where to start.
I have built a visualization tool that helps ...
- 21
3
votes
1
answer
115
views
Optimal prediction under squared percentage loss
I have to find an answer on the following question but I am struggling:
Consider a leaf of a decision tree that consists of object-label pairs $(x_{1}, y_{1}), \dots, (x_{n}, y_{n})$.
The prediction $...
- 65
1
vote
0
answers
28
views
Optimal rate of convergence of nonparametric density estimators
Suppose that $X_1, X_2, \dots, X_n$ forms an independent and identically distributed sample from some $d$-dimensional probability distribution with unknown probability density function $f$. Let $x$ be ...
- 140
2
votes
0
answers
74
views
Optimal compressibility and PCA
I have a population $\mathcal{X}$ of $N$ samples extracted from a multivariate gaussian random variable $\mathbf{x} \in \mathbb{R}^d$. Let us define a transformation $f_{d\rightarrow r} (\mathbf{x}) = ...
- 21
1
vote
1
answer
16
views
Optimal Feature Engeneering creation: best optimization method?
basically I would like to solve this problem:
(1) say I have N features that I want to transform with a generic f(x, theta) ...
- 209
1
vote
1
answer
298
views
Question on Optimal predictors for the 0-1 loss function
The input $X \in \{0, 1\}$ and label $T \in \{0,1\}$ are binary random variables, and the set of predictors that we consider are the functions $y : \{0, 1\} \rightarrow \{0, 1\}$. Recall the $0$-$1$ ...
- 143
2
votes
1
answer
190
views
UMP test and non-decreasing power function
Let $\phi$ be a UMP test for $H_o: \theta \leq \theta_0$ and $H_1: \theta > \theta_0$. Let its power function, $E_\theta(\phi)$, be differentiable w.r.t. $\theta$.
Show the power function is non-...
- 121
0
votes
0
answers
15
views
Help needed in the following Most Powerful Test Problem
I have solved this problem. I am just not sure if it is correct or not.
Let me explain my approach. To compute the size:
$\alpha = E_0(\phi(x)) = P_0(1) + P_0(2) + 0.25 P_0(3) = 0.0225$. Similarly, ...
- 1
1
vote
1
answer
50
views
Optimality in Clustering algorithms and an Optimality guaranteed clustering algorithm
So there are lots of clustering algorithms with different characteristics. What I am interested in now is a clustering algorithm which guarantees to find the optimal clustering result (if exists). And ...
- 21
2
votes
1
answer
92
views
Most powerful test for Hardy–Weinberg proportions
Consider a population with three kinds of individuals labeled $1, 2$, and $3$ occuring in the Hardy–Weinberg proportions $f(1,\theta)=\theta^2,f(2,\theta)=2\theta(1−\theta),f(3,\theta)=(1−\theta)^2$. ...
- 551
0
votes
1
answer
378
views
Uniformly most powerful test does not exists
I am having tough time understanding this concept
The book says:
“We caution the reader that UMP tests for testing H0 : θ1 ≤ θ ≤ θ2 and
H0′ : θ = θ0 for the one-parameter exponential family do not ...
- 13
0
votes
0
answers
27
views
Optimal cut-off value for continuous covariate with binary outcome [duplicate]
Research question: What is the optimal cut-off value for a continuous covariate (age) with a binary outcome (suitable or not for a certain treatment). I realize that we lose information by ...
- 41
3
votes
0
answers
84
views
Least favorable prior - Find the distribution that maximizes the Bayes risk
Suppose I've found that the Bayes risk is of the form
$$r(\theta) = \int_{-a}^a \theta^2 \pi(\theta)d\theta
$$
I want to show that the following distribution, $\pi(a)=\pi(-a)=0.5$, maximizes this ...
- 2,217
1
vote
0
answers
68
views
D-optimal design with nuisance parameters
I am a mechanical engineer trying to develope an optimal design of experiments in a problem with nuisance parameters. I would like to calculate the parameters $\mathbf{d}$ to optimally estimate ...
1
vote
1
answer
484
views
Can we always get an optimal $k$-means cluster arrangement?
I am currently studying $k$-means clustering. An optimal $k$-cluster arrangement is defined as follows:
Fix a distance $\Delta$ and $k < n$. Assume $\mathbb{X}$ have been partitioned into $k$ ...
- 1,096
1
vote
1
answer
27
views
Optimal combination of biased samplers
Suppose we are interested in the mean $\mu$ of a random variable $X$ but the only way to sample it is from known biased distributions $p_{\lambda}(x)$, such that $\left<{X}\right>_{\lambda} =\...
- 133
1
vote
0
answers
307
views
Identifying inflexion point in elbow method (cluster analysis)
I am looking for the optimal number of clusters to conduct a cluster analysis and used the following code to determine it:
...
- 327
0
votes
0
answers
40
views
Optimal point pairs for estimating the distance function in a metric space
I’m looking for the name of the methods or subfield that deals with problems like this:
Assume that we have $n \times m$ points over a regular grid.They are embedded into a metric space. We can get ...
- 175
4
votes
2
answers
139
views
Biased estimator obtained by optimal experiment design
I am using a model-based approach to infer the parameters of a given system. Namely, I represent my system by a model $\mathcal{M}$ with parameters $\theta$. To estimate the true value of $\theta$, I ...
- 1,248
1
vote
0
answers
35
views
Find distribution that minimises a function of its moments [closed]
Imagine a probability density function $f(x)$, defined for positive $x$, and let's note its $n$th non-centred moment $x_{n}$. The mean $x_{1}$ is fixed (and positive).
How can I find $f(x)$ that ...
- 11
2
votes
1
answer
59
views
How to understand the sufficient condition for global optimum for a constrained optimization probelm
How to understand the sufficient condition for global optimum? From my understanding, the global optimum should be 0 instead of $\geq 0$, where does this come from?
Consider the constrained ...
- 427
4
votes
0
answers
106
views
Is it possible to show that this estimator has minimum variance?
Doing some exercises I stumbled upon this tricky one:
Suppose we have an independent random sample $(X_1, ... , X_n)$ with $X_i \sim Poisson(\lambda)$. Define $\theta = e^{-\lambda}$.
Let $$ \...
0
votes
0
answers
29
views
Variance estimator that is optimal under absolute loss
Given a random i.i.d. sample from a population with a finite variance $\sigma^2<\infty$, what estimator of $\sigma^2$ is optimal under absolute loss?
$$
\arg\min_{\hat\sigma^{2}\in F}\mathbb{E}(|\...
- 55.5k
3
votes
1
answer
32
views
how to understand this math formula for bandwidth calculation?
I am reading a paper that uses the following equation to calculate the optimal bandwidth, however, I am confused about the position of "4" and "3" in the equation. is this a typo? or what does it mean?...
- 433
0
votes
0
answers
74
views
Explain asymptotic optimality property of AIC (Akaike information criterion)
I am trying to use AIC in my research, I know how to apply it and how to interpret its value, but I do not understand the asymptotic optimality property in general and why AIC has this property. Can ...
- 1
1
vote
0
answers
14
views
How to optimize a number of games with a limit set of second chances at guessing other options given probabilities?
Let's say I have a game with $M$ outcomes $\{O_i\}_{i=1}^{M}$ and I have a predictor $P$ that gives me probabilities for the outcomes of a game.
I now have $N$ independent games that I want to ...
- 123
1
vote
1
answer
64
views
D-optimal DOE suggest repeated samples
I tried to generate a D optimal design but the design output sounds very weird to me. I have a (real) process and I`d like to explore 3 factors, but the process have a lot of constraints so I provide ...
- 11
0
votes
0
answers
21
views
Finding the 'optimal distribution' with one unknown variable
I am trying to find the optimal distribution (of ingredients) with one unknown parameter (skillset).
Example, when baking a bread the ingredients are flour,salt,yeast and water. When I ask 100 ...
- 101
7
votes
3
answers
376
views
Justification for and optimality of $R^2_{adj.}$ as a model selection criterion
In a recent thread, use of adjusted $R^2$ ($R^2_{adj.}$) is mentioned in the context of model selection, e.g.
The adjustment was invented as a solution to problems caused by variable selection
...
- 55.5k
1
vote
0
answers
53
views
What is the best algorithm so far for probabilistic adaptive group testing?
I am checking https://en.wikipedia.org/wiki/Group_testing#Classification_of_group-testing_problems but could not figure out the answer.
I am focusing on probabilistic adaptive group testing. The ...
- 897
0
votes
1
answer
303
views
Rejection sampling for optimal $\lambda$ and $a$
Suppose $f(x) \propto \exp ({-(x-u)^2\over2\sigma^2}) I_{X>=a}$ and we cannot compute the normalizing constant. Consider rejection sampling using proposal density of a shifted exponential ...
- 11
2
votes
0
answers
36
views
Hyper-parameters which minimize the variance of transformed multi-variate Guassian variable
Let $k < p$ be positive integers and $g: \mathbb R^k \rightarrow \mathbb R^p$ be a smooth Lipschitz continuous function. Let $y_1,\ldots, y_N \in \mathbb R^p$ and $a = (a_1,\ldots,a_N) \in \mathbb ...
- 468
7
votes
1
answer
101
views
how to sample data for regression that is the most informative?
Background
I have a unknown function
$$f(x_1, x_2)$$
But I have access to evaluate this function finite $L$ times,
$$y_j = f(x_1^j, x_2^j), j=1,\ldots,L $$
Then I have a model $\hat{f}$ which I ...
3
votes
0
answers
139
views
Is there a UMVUE for arbitrary distribution with density and variance?
Let F be the family of all distributions with probability density and finite variance, and $X_1, ..., X_n$ be random samples from F.
Does UMVUE for variance exists for this situation?
- 427
3
votes
1
answer
582
views
Is there a Fisher Information equivalent in MAP Empirical Bayes estimation?
Background
The Fisher information for a linear Gaussian model is
$\mathcal{I}_{\theta} = \frac{X X^T}{\sigma^2} $. This is used in optimal experiment design techniques, for example, maximisation of $|...
- 805
2
votes
0
answers
48
views
At which rank should I reasonably stop selecting interview candidates?
It seems that my problem could be classified as an "optimal stopping" problem, but I am unable to make much of this information. It is an important problem for my organisation (an NGO), in order to ...
- 21
4
votes
2
answers
610
views
What's the difference between "Optimal linear predictor" and "best unbiased linear estimator"?
Greene (econometric analysis 7th ed. p 53) states that OLS is the "optimal linear predictor":
Then on the next page, he states that OLS is also the BLUE estimator (Gauss-Markov Theorem):
I ...
- 2,177
5
votes
1
answer
501
views
Estimator that is optimal under all sensible loss (evaluation) functions
Consider a probability distribution $D$ with a parameter $\theta$ and an i.i.d. sample $S$ from that distribution. I am interested in an estimator $\hat\theta(S)$ of $\theta$ that satisfies the ...
- 55.5k
1
vote
0
answers
36
views
Feature scaling and equivalence of the mean squared loss function optimal solutions
When applying the feature scaling, what is the mathematical proof that the optimum of the mean squared loss function is the same in both without-normalisation case and with-normalisation of the ...
- 169
10
votes
0
answers
571
views
Does there exist an analogous statement to BLUE (Gauss-Markov) for GLMs?
I recall from my graduate school days that the Gauss-Markov (GM) theorem states that the Best Linear Unbiased Estimator (BLUE) in a linear regression is $\vec{\beta}=(X^TX)^{-1}X^T\vec{y}$. An amazing ...
- 3,919
1
vote
0
answers
26
views
Relationship between 4SID and 4DVAR
Background:
The "4SID" method allows one to back analytic models from data using a form similar to that of a Kalman filter. (1)
The 4DVAR is an approach to use data with an analytic model for weather ...
- 8,280
1
vote
0
answers
88
views
D-optimal mixture design- poor evaluation of design
My problem is related to the design evaluation of my set of experiments.
I am trying to design a mixture of 4 materials, each having certain minimum and maximum dosage constraints. To come up with a ...
- 11
1
vote
0
answers
31
views
Bayes decision boundary naming [duplicate]
Why the optimal classifier is called "Bayes". I do not see the connection with being Bayesian or so.
Edit: This question asks about naming. Not about math.
Another phrasing of the question: Given ...
- 123
1
vote
0
answers
110
views
What parameter to use when testing a SVM classifier on an independent test dataset?
I am trying to make a classifier that effectively distinguishes between control group and patient group, and then I want to use that classifier to distinguish high-risk patients who convert to patient ...
- 95
1
vote
1
answer
292
views
Questions about gradient descent and local optima
I'm starting out learning about gradient descent, and have a couple of conceptual questions.
I noticed a common pitfall of gradient descent is getting stuck in the local optima. How can this be ...
- 111