I was studying the slides of the course in statistics, but there is a theorem that is not clear for me. This chapter was about finding a complete statistic, and it explains that it can be found with an exponential family. The problem is that it is not enough to find it, it is needed also to check for an "open ball". I did not understand what is that concept, and how to apply it to find the estimator. For example, I didnt know how to check for the open ball when finding the estimators for a two-parameter exponential. It would be nice to show an example as I am new in maths and statistics... I uploaded the example when finding complete estimators for a gamma distribution. Thanks so much
-
$\begingroup$ What means an open set? Teacher said that we had to check always for the range of the parameters or something like this... For example, the statistics in a Gamma distribution belonged to (-1, inf) * (-inf, 0), so he said that it was an open ball in R^2 $\endgroup$– Javier Moreno SepenaCommented Dec 21, 2020 at 17:39
-
2$\begingroup$ An "open ball" $B(\mathbf{x}, r)$ is determined by a point $\mathbf{x}\in\mathbb{R}^n$ and a numerical radius $r\ge 0.$ By definition, it is the set of all points whose distance to $\mathbf{x}$ is less than $r:$ $$B(\mathbf{x},r)=\{\mathbf{y}\in\mathbb{R}^n\mid |\mathbf{y}-\mathbf{x}|\lt r\}.$$ This is a basic concept of metric space theory. $\endgroup$– whuber ♦Commented Dec 21, 2020 at 18:01
-
1$\begingroup$ Please type your question as text, do not just post a photograph or screenshot (see here). When you retype the question, add the self-study tag & read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. Please make these changes as just posting your homework & hoping someone will do it for you is grounds for closing. $\endgroup$– kjetil b halvorsen ♦Commented Dec 22, 2020 at 0:46
-
$\begingroup$ Notice that this expression of the density is not unique: instead of the vector $(q_j(\theta))$ you may use a version that has been shifted by any constant amount and rescaled by any positive amount. Thus, "contains an open ball" really means there is a way to write the density for which $Z$ is "almost all" of $\mathbb{R}^k$ in the sense that it contains all vectors within an arbitrarily large distance from the origin. For conceptual purposes, then, you might begin your study of the theorem by interpreting "almost all" as meaning "all" or, if you like, "all with possibly a few exceptions." $\endgroup$– whuber ♦Commented Dec 22, 2020 at 14:04
2 Answers
When formally defining a probability density as $$f_\theta(x)=c(\theta)h(x)\exp\left\{\sum_{j=1}^k q_j(\theta)t_j(x)\right\}\tag{1}$$ it is always possible to add useless terms in the expression, as for instance in \begin{align}f_\theta(x)&=c(\theta)h(x)\exp\left\{[q_1(\theta)-q_{k+1}(\theta)]t_1(x)+[q_2(\theta)-q_{k+1}(\theta)]t_2(x))\right.\\ &\left.\quad+\sum_{j=3}^k q_j(\theta)t_j(x) + q_{k+1}(\theta)[t_1(x)+t_2(x)]\right\}\\ \end{align} for an arbitrary function $q_{k+1}(\theta)$. It thus also writes as $$f_\theta(x)=c(\theta)h(x)\exp\left\{\sum_{j=1}^{k+1} q'_j(\theta)t'_j(x)\right\}\tag{2}$$ Another example is to write (1) as $$f_\theta(x)=c(\theta)h(x)\exp\left\{\sum_{j=1}^k q_j(\theta)t_j(x)+0. t_{k+1}(x)\right\}\tag{3}$$ and to set $q_{k+1}(\theta)=0$, with $t_{k+1}(x)$ an arbitrary statistic.
This means that the integer $k$ in (1) is not unique unless some constraints are set on the terms in (1). This leads to the notion of a full-rank exponential family whose density is of the type (1) with constraints that
- the functions $q_j(\cdot)$ are linearly independent
- the functions $t_j(\cdot)$ are linearly independent, which rules out the above counterexample
These two conditions are not enough for completeness: if the vector $z(\theta)=(q_1(\theta),\ldots,q_k(\theta))$ is not varying freely enough when $\theta$ varies in $\Theta$, completeness of $\tau(x)=(t_1(x),\ldots,t_k(x))$ does not always hold. See for instance this example of a translated exponential (which is not an exponential family).
The sufficient constraint for ensuring completeness is that the set $\mathcal Z=z(\Theta)$ of $z(\theta)$'s is large enough, which translates as, for every $\zeta\in\mathcal Z$, there exists a ball $\mathcal B$ centred at $\zeta$ and with a small enough positive radius such that $\mathcal B\subset\mathcal Z$.
-
1$\begingroup$ Thank you so much! Completeness i so complex... $\endgroup$ Commented Dec 21, 2020 at 22:26
Your book is presenting a simple concept in very formal mathematics. An "open ball" is a concept in mathematics referring to sets which do not contain their boundary points. This is a very general concept in mathematics, but we will usually work with real number ($\mathbb{R}^n$) and use Euclidean distance in statistics. It is helpful to think in terms of this specific case.
In 1-dimension these are open sets, i.e $(0, 1)$ or ($-\infty, \infty)$ or $(a,b)$. In 2-dimensions these are disks that don't contain their boundary. This also can be the entire Cartesian plane.
Open sets are often useful when maximizing a likelihood function to get a maximum likelihood estimate (MLE). A maximum likelihood estimate will not be asymptotically normal if the derivative of the likelihood function at the MLE is nonzero. This can make it difficult to understand the variance of your test statistic. For example, if $X_i \sim N(\mu, 1)$ and we know $\mu > 0$, you can show the MLE is $$ \hat{\mu} = \left\{\begin{array}{lr} \bar{X} & \text{when } \bar{X} \geq 0 \\ 0 & \text{when } \bar{X} < 0 \end{array}\right\}. $$ This maximum likelihood estimator isn't asymptotically Normal. It can't be; it can't take negative values.
-
1$\begingroup$ Instead of "circle" you mean disk: the circle (by definition) is the boundary. Your statement about MLEs requiring a zero derivative at the minimum is not generally true. $\endgroup$– whuber ♦Commented Dec 21, 2020 at 18:03
-
$\begingroup$ Oh so when a estimator like yours, has an indicator function (0 when sample mean less than 0), the completeness cannot be validated right? So in your case, we cannot say it is complete. $\endgroup$ Commented Dec 21, 2020 at 18:09
-
$\begingroup$ @whuber, I think it is required for an MLE to have zero derivative to be asymptotically normal. It is not a requirement for an unbiased statistic. Do you agree with that statement? I can update my answer. $\endgroup$– EliCommented Dec 21, 2020 at 18:25
-
$\begingroup$ @Xi'an I disagree that there is no connection. The OP asked what an open ball is. I provided the definition and gave an example of why it is important. What are you taking issue with? $\endgroup$– EliCommented Dec 21, 2020 at 18:31
-
2$\begingroup$ I understood a little about derivatives and open balls, but what has to do with completeness? $\endgroup$ Commented Dec 21, 2020 at 19:30