Questions tagged [optimal]

For questions about optimality properties of statistical methods, such as optimal parameter estimation or optimal testing. Both for questions about optimality theory in general, and for questions about optimality properties of specific procedures.

Filter by
Sorted by
Tagged with
1 vote
0 answers
44 views

From inputs, updating $f\left(X_{i},Y_{i}\right)$ and $g\left(X_{i},Y_{i}\right)$ much other than $\overline{X_{i}}$ and $\overline{Y_{i}}$ to compare [closed]

(Premier League $2019$/$20$). The season was affected by the COVID–$19$ Pandemic while each team had a quarter of their schedule left (I call it $1/4$ because $1/2$ teams had $4$ away matches, and $5$ ...
user avatar
0 votes
0 answers
14 views

Optimal MSPE predictor, estimating equations

I am reading through Bickels and Doksum's Mathematical Statistics book, and in the second chapter they give justification for the use of a minimum contrast estimator: However, I do not quite ...
user avatar
  • 159
1 vote
1 answer
54 views

What does it mean for the Bayes Classifier to be optimal? [closed]

The Bayes classifier is always called the 'optimal' classifier. What does this actually mean? In particular does optimal mean the Bayes classifier will never make a mistake when predicting the label ...
user avatar
0 votes
0 answers
21 views

Optimality conditions for the LASSO

In this paper, on page 1122, it states that the optimality conditions for the LASSO give $\hat{\beta} = n_{\lambda}(\hat{\beta} - X^T(X\hat{\beta} - y))$, where $n_\lambda$ is the soft-thresholding ...
user avatar
  • 1
1 vote
0 answers
43 views

What is the theoretical justification for alternatives to MSE minimisation?

I'm trying to wrap my head around the connection between statistical regression and its probability theoretical justification. In many books on statistics/machine learning, one is introduced to the ...
user avatar
0 votes
0 answers
7 views

Method to find optimal settings given an output metric

I am trying to optimize a screening problem that is usual done through brute force. I am trying to expand my skill set but I don't know where to start. I have built a visualization tool that helps ...
user avatar
3 votes
1 answer
115 views

Optimal prediction under squared percentage loss

I have to find an answer on the following question but I am struggling: Consider a leaf of a decision tree that consists of object-label pairs $(x_{1}, y_{1}), \dots, (x_{n}, y_{n})$. The prediction $...
user avatar
  • 65
1 vote
0 answers
28 views

Optimal rate of convergence of nonparametric density estimators

Suppose that $X_1, X_2, \dots, X_n$ forms an independent and identically distributed sample from some $d$-dimensional probability distribution with unknown probability density function $f$. Let $x$ be ...
user avatar
  • 140
2 votes
0 answers
74 views

Optimal compressibility and PCA

I have a population $\mathcal{X}$ of $N$ samples extracted from a multivariate gaussian random variable $\mathbf{x} \in \mathbb{R}^d$. Let us define a transformation $f_{d\rightarrow r} (\mathbf{x}) = ...
user avatar
1 vote
1 answer
16 views

Optimal Feature Engeneering creation: best optimization method?

basically I would like to solve this problem: (1) say I have N features that I want to transform with a generic f(x, theta) ...
user avatar
  • 209
1 vote
1 answer
298 views

Question on Optimal predictors for the 0-1 loss function

The input $X \in \{0, 1\}$ and label $T \in \{0,1\}$ are binary random variables, and the set of predictors that we consider are the functions $y : \{0, 1\} \rightarrow \{0, 1\}$. Recall the $0$-$1$ ...
user avatar
  • 143
2 votes
1 answer
190 views

UMP test and non-decreasing power function

Let $\phi$ be a UMP test for $H_o: \theta \leq \theta_0$ and $H_1: \theta > \theta_0$. Let its power function, $E_\theta(\phi)$, be differentiable w.r.t. $\theta$. Show the power function is non-...
user avatar
0 votes
0 answers
15 views

Help needed in the following Most Powerful Test Problem

I have solved this problem. I am just not sure if it is correct or not. Let me explain my approach. To compute the size: $\alpha = E_0(\phi(x)) = P_0(1) + P_0(2) + 0.25 P_0(3) = 0.0225$. Similarly, ...
user avatar
  • 1
1 vote
1 answer
50 views

Optimality in Clustering algorithms and an Optimality guaranteed clustering algorithm

So there are lots of clustering algorithms with different characteristics. What I am interested in now is a clustering algorithm which guarantees to find the optimal clustering result (if exists). And ...
user avatar
  • 21
2 votes
1 answer
92 views

Most powerful test for Hardy–Weinberg proportions

Consider a population with three kinds of individuals labeled $1, 2$, and $3$ occuring in the Hardy–Weinberg proportions $f(1,\theta)=\theta^2,f(2,\theta)=2\theta(1−\theta),f(3,\theta)=(1−\theta)^2$. ...
user avatar
  • 551
0 votes
1 answer
378 views

Uniformly most powerful test does not exists

I am having tough time understanding this concept The book says: “We caution the reader that UMP tests for testing H0 : θ1 ≤ θ ≤ θ2 and H0′ : θ = θ0 for the one-parameter exponential family do not ...
user avatar
0 votes
0 answers
27 views

Optimal cut-off value for continuous covariate with binary outcome [duplicate]

Research question: What is the optimal cut-off value for a continuous covariate (age) with a binary outcome (suitable or not for a certain treatment). I realize that we lose information by ...
user avatar
  • 41
3 votes
0 answers
84 views

Least favorable prior - Find the distribution that maximizes the Bayes risk

Suppose I've found that the Bayes risk is of the form $$r(\theta) = \int_{-a}^a \theta^2 \pi(\theta)d\theta $$ I want to show that the following distribution, $\pi(a)=\pi(-a)=0.5$, maximizes this ...
user avatar
1 vote
0 answers
68 views

D-optimal design with nuisance parameters

I am a mechanical engineer trying to develope an optimal design of experiments in a problem with nuisance parameters. I would like to calculate the parameters $\mathbf{d}$ to optimally estimate ...
user avatar
1 vote
1 answer
484 views

Can we always get an optimal $k$-means cluster arrangement?

I am currently studying $k$-means clustering. An optimal $k$-cluster arrangement is defined as follows: Fix a distance $\Delta$ and $k < n$. Assume $\mathbb{X}$ have been partitioned into $k$ ...
user avatar
  • 1,096
1 vote
1 answer
27 views

Optimal combination of biased samplers

Suppose we are interested in the mean $\mu$ of a random variable $X$ but the only way to sample it is from known biased distributions $p_{\lambda}(x)$, such that $\left<{X}\right>_{\lambda} =\...
user avatar
  • 133
1 vote
0 answers
307 views

Identifying inflexion point in elbow method (cluster analysis)

I am looking for the optimal number of clusters to conduct a cluster analysis and used the following code to determine it: ...
user avatar
0 votes
0 answers
40 views

Optimal point pairs for estimating the distance function in a metric space

I’m looking for the name of the methods or subfield that deals with problems like this: Assume that we have $n \times m$ points over a regular grid.They are embedded into a metric space. We can get ...
user avatar
4 votes
2 answers
139 views

Biased estimator obtained by optimal experiment design

I am using a model-based approach to infer the parameters of a given system. Namely, I represent my system by a model $\mathcal{M}$ with parameters $\theta$. To estimate the true value of $\theta$, I ...
user avatar
1 vote
0 answers
35 views

Find distribution that minimises a function of its moments [closed]

Imagine a probability density function $f(x)$, defined for positive $x$, and let's note its $n$th non-centred moment $x_{n}$. The mean $x_{1}$ is fixed (and positive). How can I find $f(x)$ that ...
user avatar
2 votes
1 answer
59 views

How to understand the sufficient condition for global optimum for a constrained optimization probelm

How to understand the sufficient condition for global optimum? From my understanding, the global optimum should be 0 instead of $\geq 0$, where does this come from? Consider the constrained ...
user avatar
4 votes
0 answers
106 views

Is it possible to show that this estimator has minimum variance?

Doing some exercises I stumbled upon this tricky one: Suppose we have an independent random sample $(X_1, ... , X_n)$ with $X_i \sim Poisson(\lambda)$. Define $\theta = e^{-\lambda}$. Let $$ \...
user avatar
0 votes
0 answers
29 views

Variance estimator that is optimal under absolute loss

Given a random i.i.d. sample from a population with a finite variance $\sigma^2<\infty$, what estimator of $\sigma^2$ is optimal under absolute loss? $$ \arg\min_{\hat\sigma^{2}\in F}\mathbb{E}(|\...
user avatar
3 votes
1 answer
32 views

how to understand this math formula for bandwidth calculation?

I am reading a paper that uses the following equation to calculate the optimal bandwidth, however, I am confused about the position of "4" and "3" in the equation. is this a typo? or what does it mean?...
user avatar
0 votes
0 answers
74 views

Explain asymptotic optimality property of AIC (Akaike information criterion)

I am trying to use AIC in my research, I know how to apply it and how to interpret its value, but I do not understand the asymptotic optimality property in general and why AIC has this property. Can ...
user avatar
1 vote
0 answers
14 views

How to optimize a number of games with a limit set of second chances at guessing other options given probabilities?

Let's say I have a game with $M$ outcomes $\{O_i\}_{i=1}^{M}$ and I have a predictor $P$ that gives me probabilities for the outcomes of a game. I now have $N$ independent games that I want to ...
user avatar
1 vote
1 answer
64 views

D-optimal DOE suggest repeated samples

I tried to generate a D optimal design but the design output sounds very weird to me. I have a (real) process and I`d like to explore 3 factors, but the process have a lot of constraints so I provide ...
user avatar
0 votes
0 answers
21 views

Finding the 'optimal distribution' with one unknown variable

I am trying to find the optimal distribution (of ingredients) with one unknown parameter (skillset). Example, when baking a bread the ingredients are flour,salt,yeast and water. When I ask 100 ...
user avatar
  • 101
7 votes
3 answers
376 views

Justification for and optimality of $R^2_{adj.}$ as a model selection criterion

In a recent thread, use of adjusted $R^2$ ($R^2_{adj.}$) is mentioned in the context of model selection, e.g. The adjustment was invented as a solution to problems caused by variable selection ...
user avatar
1 vote
0 answers
53 views

What is the best algorithm so far for probabilistic adaptive group testing?

I am checking https://en.wikipedia.org/wiki/Group_testing#Classification_of_group-testing_problems but could not figure out the answer. I am focusing on probabilistic adaptive group testing. The ...
user avatar
0 votes
1 answer
303 views

Rejection sampling for optimal $\lambda$ and $a$

Suppose $f(x) \propto \exp ({-(x-u)^2\over2\sigma^2}) I_{X>=a}$ and we cannot compute the normalizing constant. Consider rejection sampling using proposal density of a shifted exponential ...
user avatar
2 votes
0 answers
36 views

Hyper-parameters which minimize the variance of transformed multi-variate Guassian variable

Let $k < p$ be positive integers and $g: \mathbb R^k \rightarrow \mathbb R^p$ be a smooth Lipschitz continuous function. Let $y_1,\ldots, y_N \in \mathbb R^p$ and $a = (a_1,\ldots,a_N) \in \mathbb ...
user avatar
  • 468
7 votes
1 answer
101 views

how to sample data for regression that is the most informative?

Background I have a unknown function $$f(x_1, x_2)$$ But I have access to evaluate this function finite $L$ times, $$y_j = f(x_1^j, x_2^j), j=1,\ldots,L $$ Then I have a model $\hat{f}$ which I ...
user avatar
3 votes
0 answers
139 views

Is there a UMVUE for arbitrary distribution with density and variance?

Let F be the family of all distributions with probability density and finite variance, and $X_1, ..., X_n$ be random samples from F. Does UMVUE for variance exists for this situation?
user avatar
  • 427
3 votes
1 answer
582 views

Is there a Fisher Information equivalent in MAP Empirical Bayes estimation?

Background The Fisher information for a linear Gaussian model is $\mathcal{I}_{\theta} = \frac{X X^T}{\sigma^2} $. This is used in optimal experiment design techniques, for example, maximisation of $|...
user avatar
  • 805
2 votes
0 answers
48 views

At which rank should I reasonably stop selecting interview candidates?

It seems that my problem could be classified as an "optimal stopping" problem, but I am unable to make much of this information. It is an important problem for my organisation (an NGO), in order to ...
user avatar
4 votes
2 answers
610 views

What's the difference between "Optimal linear predictor" and "best unbiased linear estimator"?

Greene (econometric analysis 7th ed. p 53) states that OLS is the "optimal linear predictor": Then on the next page, he states that OLS is also the BLUE estimator (Gauss-Markov Theorem): I ...
user avatar
  • 2,177
5 votes
1 answer
501 views

Estimator that is optimal under all sensible loss (evaluation) functions

Consider a probability distribution $D$ with a parameter $\theta$ and an i.i.d. sample $S$ from that distribution. I am interested in an estimator $\hat\theta(S)$ of $\theta$ that satisfies the ...
user avatar
1 vote
0 answers
36 views

Feature scaling and equivalence of the mean squared loss function optimal solutions

When applying the feature scaling, what is the mathematical proof that the optimum of the mean squared loss function is the same in both without-normalisation case and with-normalisation of the ...
user avatar
  • 169
10 votes
0 answers
571 views

Does there exist an analogous statement to BLUE (Gauss-Markov) for GLMs?

I recall from my graduate school days that the Gauss-Markov (GM) theorem states that the Best Linear Unbiased Estimator (BLUE) in a linear regression is $\vec{\beta}=(X^TX)^{-1}X^T\vec{y}$. An amazing ...
user avatar
1 vote
0 answers
26 views

Relationship between 4SID and 4DVAR

Background: The "4SID" method allows one to back analytic models from data using a form similar to that of a Kalman filter. (1) The 4DVAR is an approach to use data with an analytic model for weather ...
user avatar
  • 8,280
1 vote
0 answers
88 views

D-optimal mixture design- poor evaluation of design

My problem is related to the design evaluation of my set of experiments. I am trying to design a mixture of 4 materials, each having certain minimum and maximum dosage constraints. To come up with a ...
user avatar
  • 11
1 vote
0 answers
31 views

Bayes decision boundary naming [duplicate]

Why the optimal classifier is called "Bayes". I do not see the connection with being Bayesian or so. Edit: This question asks about naming. Not about math. Another phrasing of the question: Given ...
user avatar
  • 123
1 vote
0 answers
110 views

What parameter to use when testing a SVM classifier on an independent test dataset?

I am trying to make a classifier that effectively distinguishes between control group and patient group, and then I want to use that classifier to distinguish high-risk patients who convert to patient ...
user avatar
1 vote
1 answer
292 views

Questions about gradient descent and local optima

I'm starting out learning about gradient descent, and have a couple of conceptual questions. I noticed a common pitfall of gradient descent is getting stuck in the local optima. How can this be ...
user avatar
  • 111