Questions tagged [optimal]
For questions about optimality properties of statistical methods, such as optimal parameter estimation or optimal testing. Both for questions about optimality theory in general, and for questions about optimality properties of specific procedures.
65 questions
0
votes
0
answers
25
views
Are interior-point methods guranteed to converge to the global optimum of a convex objective function?
I am looking into convex optimization. However, I am not sure if there are interior-point methods that are guaranteed to converge to the globally optimal solution given either a strictly convex or a ...
- 101
0
votes
0
answers
25
views
Unbiased test for homogeneity of means of exponenential samples
Given $K$ independent samples of $Y_{i1},\dots,Y_{in_i} \ \text{i.i.d.} \ \sim Exp(\lambda_i)$ with $i=1,\dots, K$ and $n_i$ the size of the $i$-th sample, is there any statistics with analytically ...
- 82.8k
5
votes
2
answers
1k
views
What's the difference between "Optimal linear predictor" and "best unbiased linear estimator"?
Greene (econometric analysis 7th ed. p 53) states that OLS is the "optimal linear predictor":
Then on the next page, he states that OLS is also the BLUE estimator (Gauss-Markov Theorem):
I ...
- 1,411
1
vote
0
answers
69
views
UMP two sided tests for exponential families
Consider a random variable $X$ with density $$f(x : θ) = C(θ)e^{η(θ)T(x)}h(x), θ ∈ Θ$$.
Assume that $η(θ)$ is strictly increasing in $θ$ and that the family is full rank. Show that there will not be ...
- 82.8k
0
votes
0
answers
80
views
General rule when max likelihood is suboptimal
Inspired by the question regarding Bessel's correction, I wonder whether there is a general rule regarding applicability of maximum likelihood for parameter estimation.
My guess is that the parameters ...
- 82.8k
1
vote
1
answer
49
views
How to combine a noisy (but unbiased) estimate with a precise (but possibly biased) estimate in A/B tests?
Suppose I want to estimate some set of unknown quantities $\theta_1$, …, $\theta_N$. For each $i \in \{1, …, N\}$, I have two estimators: $\hat{\theta_i}_A$ and $ \hat{\theta_i}_B$. The goal is to ...
- 35.2k
3
votes
1
answer
318
views
Difference between Parsimonious model vs Optimal model
As per my understanding, parsimonious regression model is the model that has less variables but with those variables I can describe the data best. Is it so?
Then ...
- 128k
3
votes
1
answer
104
views
Is there an optimality result for the two-sample Wilcoxon-Mann-Whitney test?
Is there any mathematical result that states that the Wilcoxon-Mann-Whitney (WMW) test is optimal in some sense, for a specific testing problem that is a subproblem of the general problem the WMW test ...
- 46.8k
5
votes
2
answers
258
views
What is the theoretical justification for alternatives to MSE minimisation?
I'm trying to wrap my head around the connection between statistical regression and its probability theoretical justification. In many books on statistics/machine learning, one is introduced to the ...
2
votes
1
answer
56
views
D-Optimality for regression of polynomial models in one variable with missing terms
Let's say I have a model that looks as follows:
$$y = x + ax^3 + bx^5 + cx^7 + dx^9$$
Given $n$ free choices for x as input measurements how can I determine which $x$'s I should input to best ...
- 82.8k
13
votes
1
answer
1k
views
Does there exist an analogous statement to BLUE (Gauss-Markov) for GLMs?
I recall from my graduate school days that the Gauss-Markov (GM) theorem states that the Best Linear Unbiased Estimator (BLUE) in a linear regression is $\vec{\beta}=(X^TX)^{-1}X^T\vec{y}$. An amazing ...
- 67k
0
votes
0
answers
16
views
Sliding parameters over a ML model
I'm analysing the performance of an algorithm based on neural networks and I have to tune two parameters( number of past days to consider for the turbidity and volume of water flowing) by hand. So ...
- 11
2
votes
1
answer
4k
views
What does it mean for the Bayes Classifier to be optimal? [closed]
The Bayes classifier is always called the 'optimal' classifier. What does this actually mean? In particular does optimal mean the Bayes classifier will never make a mistake when predicting the label ...
- 21
1
vote
1
answer
115
views
D-optimal DOE suggest repeated samples
I tried to generate a D optimal design but the design output sounds very weird to me. I have a (real) process and I`d like to explore 3 factors, but the process have a lot of constraints so I provide ...
- 82.8k
3
votes
1
answer
631
views
Optimal prediction under squared percentage loss
I have to find an answer on the following question but I am struggling:
Consider a leaf of a decision tree that consists of object-label pairs $(x_{1}, y_{1}), \dots, (x_{n}, y_{n})$.
The prediction $...
- 131k
2
votes
0
answers
104
views
Optimal compressibility and PCA
I have a population $\mathcal{X}$ of $N$ samples extracted from a multivariate gaussian random variable $\mathbf{x} \in \mathbb{R}^d$. Let us define a transformation $f_{d\rightarrow r} (\mathbf{x}) = ...
- 31
1
vote
0
answers
121
views
Optimal rate of convergence of nonparametric density estimators
Suppose that $X_1, X_2, \dots, X_n$ forms an independent and identically distributed sample from some $d$-dimensional probability distribution with unknown probability density function $f$. Let $x$ be ...
- 82.8k
8
votes
3
answers
1k
views
Justification for and optimality of $R^2_{adj.}$ as a model selection criterion
In a recent thread, use of adjusted $R^2$ ($R^2_{adj.}$) is mentioned in the context of model selection, e.g.
The adjustment was invented as a solution to problems caused by variable selection
...
- 143
2
votes
1
answer
23
views
Optimal Feature Engeneering creation: best optimization method?
basically I would like to solve this problem:
(1) say I have N features that I want to transform with a generic f(x, theta) ...
- 35.2k
1
vote
1
answer
543
views
Question on Optimal predictors for the 0-1 loss function
The input $X \in \{0, 1\}$ and label $T \in \{0,1\}$ are binary random variables, and the set of predictors that we consider are the functions $y : \{0, 1\} \rightarrow \{0, 1\}$. Recall the $0$-$1$ ...
- 58.2k
3
votes
0
answers
161
views
Least favorable prior - Find the distribution that maximizes the Bayes risk
Suppose I've found that the Bayes risk is of the form
$$r(\theta) = \int_{-a}^a \theta^2 \pi(\theta)d\theta
$$
I want to show that the following distribution, $\pi(a)=\pi(-a)=0.5$, maximizes this ...
- 82.8k
2
votes
1
answer
700
views
UMP test and non-decreasing power function
Let $\phi$ be a UMP test for $H_o: \theta \leq \theta_0$ and $H_1: \theta > \theta_0$. Let its power function, $E_\theta(\phi)$, be differentiable w.r.t. $\theta$.
Show the power function is non-...
- 11.6k
2
votes
1
answer
243
views
Most powerful test for Hardy–Weinberg proportions
Consider a population with three kinds of individuals labeled $1, 2$, and $3$ occuring in the Hardy–Weinberg proportions $f(1,\theta)=\theta^2,f(2,\theta)=2\theta(1−\theta),f(3,\theta)=(1−\theta)^2$. ...
- 9,001
1
vote
1
answer
135
views
Optimality in Clustering algorithms and an Optimality guaranteed clustering algorithm
So there are lots of clustering algorithms with different characteristics. What I am interested in now is a clustering algorithm which guarantees to find the optimal clustering result (if exists). And ...
- 58.2k
4
votes
1
answer
86
views
Verification of an optimal parameter from an empirical CDF
Suppose we have the following model for the variable $V_5$:
$$V_5 = \prod_{k=1}^5(e^{\mu + 0.2X_k}+0.05e^{0.05Y_i - 0.00125}), X_i,Y_i\sim N(0,1)$$
What I wish to do is to solve the problem $\min_{\...
CommunityBot
- 1
3
votes
0
answers
158
views
Is there a UMVUE for arbitrary distribution with density and variance?
Let F be the family of all distributions with probability density and finite variance, and $X_1, ..., X_n$ be random samples from F.
Does UMVUE for variance exists for this situation?
- 82.8k
0
votes
1
answer
1k
views
Uniformly most powerful test does not exists
I am having tough time understanding this concept
The book says:
“We caution the reader that UMP tests for testing H0 : θ1 ≤ θ ≤ θ2 and
H0′ : θ = θ0 for the one-parameter exponential family do not ...
- 46.8k
0
votes
0
answers
30
views
Optimal cut-off value for continuous covariate with binary outcome [duplicate]
Research question: What is the optimal cut-off value for a continuous covariate (age) with a binary outcome (suitable or not for a certain treatment). I realize that we lose information by ...
- 51
4
votes
2
answers
197
views
Biased estimator obtained by optimal experiment design
I am using a model-based approach to infer the parameters of a given system. Namely, I represent my system by a model $\mathcal{M}$ with parameters $\theta$. To estimate the true value of $\theta$, I ...
- 2,931
1
vote
0
answers
95
views
D-optimal design with nuisance parameters
I am a mechanical engineer trying to develope an optimal design of experiments in a problem with nuisance parameters. I would like to calculate the parameters $\mathbf{d}$ to optimally estimate ...
4
votes
0
answers
155
views
Is it possible to show that this estimator has minimum variance?
Doing some exercises I stumbled upon this tricky one:
Suppose we have an independent random sample $(X_1, ... , X_n)$ with $X_i \sim Poisson(\lambda)$. Define $\theta = e^{-\lambda}$.
Let $$ \...
- 82.8k
1
vote
1
answer
968
views
Can we always get an optimal $k$-means cluster arrangement?
I am currently studying $k$-means clustering. An optimal $k$-cluster arrangement is defined as follows:
Fix a distance $\Delta$ and $k < n$. Assume $\mathbb{X}$ have been partitioned into $k$ ...
- 28.3k
1
vote
1
answer
28
views
Optimal combination of biased samplers
Suppose we are interested in the mean $\mu$ of a random variable $X$ but the only way to sample it is from known biased distributions $p_{\lambda}(x)$, such that $\left<{X}\right>_{\lambda} =\...
- 133
1
vote
0
answers
597
views
Identifying inflexion point in elbow method (cluster analysis)
I am looking for the optimal number of clusters to conduct a cluster analysis and used the following code to determine it:
...
0
votes
0
answers
44
views
Optimal point pairs for estimating the distance function in a metric space
I’m looking for the name of the methods or subfield that deals with problems like this:
Assume that we have $n \times m$ points over a regular grid.They are embedded into a metric space. We can get ...
- 175
1
vote
0
answers
60
views
Find distribution that minimises a function of its moments [closed]
Imagine a probability density function $f(x)$, defined for positive $x$, and let's note its $n$th non-centred moment $x_{n}$. The mean $x_{1}$ is fixed (and positive).
How can I find $f(x)$ that ...
- 82.8k
2
votes
1
answer
112
views
How to understand the sufficient condition for global optimum for a constrained optimization probelm
How to understand the sufficient condition for global optimum? From my understanding, the global optimum should be 0 instead of $\geq 0$, where does this come from?
Consider the constrained ...
- 58.2k
0
votes
0
answers
51
views
Variance estimator that is optimal under absolute loss
Given a random i.i.d. sample from a population with a finite variance $\sigma^2<\infty$, what estimator of $\sigma^2$ is optimal under absolute loss?
$$
\arg\min_{\hat\sigma^{2}\in F}\mathbb{E}(|\...
- 69.5k
3
votes
1
answer
107
views
how to understand this math formula for bandwidth calculation?
I am reading a paper that uses the following equation to calculate the optimal bandwidth, however, I am confused about the position of "4" and "3" in the equation. is this a typo? or what does it mean?...
- 445
0
votes
0
answers
170
views
Explain asymptotic optimality property of AIC (Akaike information criterion)
I am trying to use AIC in my research, I know how to apply it and how to interpret its value, but I do not understand the asymptotic optimality property in general and why AIC has this property. Can ...
- 69.5k
1
vote
0
answers
21
views
How to optimize a number of games with a limit set of second chances at guessing other options given probabilities?
Let's say I have a game with $M$ outcomes $\{O_i\}_{i=1}^{M}$ and I have a predictor $P$ that gives me probabilities for the outcomes of a game.
I now have $N$ independent games that I want to ...
- 123
0
votes
0
answers
21
views
Finding the 'optimal distribution' with one unknown variable
I am trying to find the optimal distribution (of ingredients) with one unknown parameter (skillset).
Example, when baking a bread the ingredients are flour,salt,yeast and water. When I ask 100 ...
- 101
5
votes
1
answer
673
views
Estimator that is optimal under all sensible loss (evaluation) functions
Consider a probability distribution $D$ with a parameter $\theta$ and an i.i.d. sample $S$ from that distribution. I am interested in an estimator $\hat\theta(S)$ of $\theta$ that satisfies the ...
CommunityBot
- 1
1
vote
0
answers
62
views
What is the best algorithm so far for probabilistic adaptive group testing?
I am checking https://en.wikipedia.org/wiki/Group_testing#Classification_of_group-testing_problems but could not figure out the answer.
I am focusing on probabilistic adaptive group testing. The ...
- 82.8k
7
votes
1
answer
129
views
how to sample data for regression that is the most informative?
Background
I have a unknown function
$$f(x_1, x_2)$$
But I have access to evaluate this function finite $L$ times,
$$y_j = f(x_1^j, x_2^j), j=1,\ldots,L $$
Then I have a model $\hat{f}$ which I ...
- 316
0
votes
1
answer
481
views
Rejection sampling for optimal $\lambda$ and $a$
Suppose $f(x) \propto \exp ({-(x-u)^2\over2\sigma^2}) I_{X>=a}$ and we cannot compute the normalizing constant. Consider rejection sampling using proposal density of a shifted exponential ...
- 86.4k
1
vote
0
answers
111
views
UMVUE of parameter $(1-\sigma^2)^{-\frac{n}{2}}$
suppose $X_1,X_2,\ldots,X_n$ be random sample of $N(0,\sigma^2)$. how can I calculate UMVUE of parameter $(1-\sigma^2)^{-\frac{n}{2}}$
- 82.8k
1
vote
0
answers
102
views
D-optimal mixture design- poor evaluation of design
My problem is related to the design evaluation of my set of experiments.
I am trying to design a mixture of 4 materials, each having certain minimum and maximum dosage constraints. To come up with a ...
- 82.8k
2
votes
0
answers
104
views
A test for a binomial r.v. vs. sum of two binomial r.v.'s?
Suppose I draw a sample from one of the two possible random variables $X$ or $Y$ where $X\sim\text{Binomial}(p,N)$ and $Y=A+B$ with $A\sim\text{Binomial}(p,M)$ and $B\sim\text{Binomial}(q,N-M)$. ...
- 82.8k
2
votes
0
answers
39
views
Hyper-parameters which minimize the variance of transformed multi-variate Guassian variable
Let $k < p$ be positive integers and $g: \mathbb R^k \rightarrow \mathbb R^p$ be a smooth Lipschitz continuous function. Let $y_1,\ldots, y_N \in \mathbb R^p$ and $a = (a_1,\ldots,a_N) \in \mathbb ...
- 538