0
$\begingroup$

Let $X_1, \dots, X_n$ denote a random sample from the PDF

$$f_{\varphi}(x)= \begin{cases} \varphi x^{\varphi - 1} &\text{if}\, 0 < x < 1, \varphi > 0\\ 0 &\text{otherwise} \end{cases}$$

I want to show that $f_\varphi(x)$ is a member of the one-parameter exponential family. Furthermore, I want to show that $\sum_{i = 1}^n - \log(X_i)$ is sufficient for $\varphi$.

Chapter 9.13.3 Exponential Families of the textbook All of Statistics: A Concise Course in Statistical Inference by Larry Wasserman says the following:

Most of the parametric models we have studied so far are special cases of a general class of models called exponential families. We say that $\{ f(x; \theta) : \theta \in \Theta \}$ is a one-parameter exponential family if there are functions $\eta(\theta)$, $B(\theta)$, $T(x)$ and $h(x)$ such that $$f(x; \theta) = h(x) e^{\eta(\theta) T(x) - B(\theta)}.$$ It is easy to see that $T(x)$ is sufficient. We call $T$ the natural sufficient statistic.

I calculate the likelihood to be

$$\begin{align} L(\varphi; \mathbf{x}) &= \prod_{i = 1}^n \varphi x_i^{\varphi - 1} \mathbb{1}_{0 < x < 1, \varphi > 0} \\ &= \varphi^n x^{\sum_{i = 1}^n (\varphi - 1)} \prod_{i = 1}^n \mathbb{1}_{0 < x_i < 1, \varphi > 0} \\ &= \varphi^n x^{\sum_{i = 1}^n (\varphi - 1)} \mathbb{1}_{\text{min}(x_i) > 0, \varphi > 0} \end{align}$$

To get this in the appropriate form, I tried

$$\begin{align} \log\left[\varphi^n x^{\sum_{i = 1}^n (\varphi - 1)} \mathbb{1}_{\text{min}(x_i) > 0, \varphi > 0} \right] &= n\log(\varphi) + \sum_{i = 1}^n (\varphi - 1) \log(x_i) + \log \left[ \mathbb{1}_{\text{min}(x_i) > 0, \varphi > 0} \right] \\&= \exp{ \left\{ n\log(\varphi) + (\varphi - 1) \sum_{i = 1}^n \log(x_i) + \log \left[ \mathbb{1}_{\text{min}(x_i) > 0, \varphi > 0} \right] \right\} } \\ &= \exp{ \left\{ n\log(\varphi) + (\varphi - 1) \sum_{i = 1}^n \log(x_i) \right\} } \exp{\left\{ \log \left[ \mathbb{1}_{\text{min}(x_i) > 0, \varphi > 0} \right] \right\}} \\ &= \exp{ \left\{ n\log(\varphi) + (\varphi - 1) \sum_{i = 1}^n \log(x_i) \right\} } \mathbb{1}_{\text{min}(x_i) > 0, \varphi > 0} \end{align}$$

And we then select

$$\begin{align} &\eta(\varphi) = ? \\ &T(\mathbf{x}) = ? \\ &B(\varphi) = ? \\ &h(\mathbf{x}) = ? \end{align}$$

I'm unsure how much of this is correct. The first thing that might be problematic is $\log \left[ \mathbb{1}_{\text{min}(x_i) > 0, \varphi > 0} \right]$. Specifically, I'm not sure that it's valid to log this identity matrix, but I'm not sure what else to do. As you can see, I later use an exponential to cancel out the log, but, as I said, I'm not sure that it was mathematically valid to apply log to it in the first place. The second is that I needed to show that $\sum_{i = 1}^n - \log(X_i)$ is sufficient for $\varphi$, but I'm not sure how to further factorize the final expression in order to get this.

Another issue is that I cannot find any definitions or theorems for this in this chapter. There's usually a theorem that formally and explicitly states what you need to do/show in order show something (in this case, showing that $f_\varphi(x)$ is a member of the one-parameter exponential family and that $\sum_{i = 1}^n - \log(X_i)$ is sufficient for $\varphi$). The excerpt that I posted above was simply part of the normal discussion of the chapter at the very beginning. Based on that, I presumed that, for showing that $f_\varphi(x)$ is a member of the one-parameter family, I just have to show that the likelihood can take the form $f(x; \theta) = h(x) e^{\eta(\theta) T(x) - B(\theta)}$. However, for the sufficient statistic part, it just says that "it is easy to see that $T(x)$ is sufficient", so I'm not exactly sure what I'm supposed to do with this (do I need to use the Fisher-Neyman factorization theorem, or does the exponential family form imply that $T(\mathbf{x})$ is sufficient, and so we don't need to do anything else?).

So, overall, what is the correct way to do this to show that $f_\varphi(x)$ is a member of the one-parameter exponential family and $\sum_{i = 1}^n - \log(X_i)$ is sufficient for $\varphi$?

$\endgroup$
6
  • 1
    $\begingroup$ My answer at stats.stackexchange.com/questions/519578 gives a simple recipe for determining whether a family is Exponential. $\endgroup$ – whuber Apr 20 at 20:04
  • $\begingroup$ @whuber I know, but it is important that I also learn the "conventional" way, right? Your way is valid, but I'm not doing this just to solve the problem and be done; I need to learn concepts for pedagogical reasons as well. $\endgroup$ – The Pointer Apr 20 at 20:05
  • $\begingroup$ I don't know what a "conventional" way might mean. Both for practical and pedagogical purposes, any way that is logical and clearly explained will be satisfactory. The advantage of my approach is that it obviates any need to have an intuitive or deep understanding of the functions used to express the likelihood, thereby eliminating one conceptual obstacle (as well as adding rigor). $\endgroup$ – whuber Apr 20 at 20:07
  • $\begingroup$ @whuber At the moment, I'm trying to learn by studying Wasserman's textbook, and this is the way he introduces the concepts of exponential families, so if I'm going to study the textbook properly, then I need to ensure that I understand all of the concepts. As you said, the method you posted in that answer works too, but I'm trying to learn and apply these concepts by following the structure of the textbook. $\endgroup$ – The Pointer Apr 20 at 20:12
  • $\begingroup$ He's going to assume you have a basic understanding of the properties of exp, log, indicator functions, etc -- that is, the functions that appear in these simple likelihoods -- but your difficulty is that many of your questions about this topic belie a lack of that knowledge. That gives you two routes of study: one is to learn more about these functions and the other is to apply recipes. $\endgroup$ – whuber Apr 20 at 20:15
0
$\begingroup$

Here is my attempt.

1.Show that $f(x; \varphi)$ is in the one-parameter exponential family.

Assume that $\varphi > 0$.

Rewriting $f(x; \varphi)$, we have that

\begin{align*} f(x; \varphi) &= \exp \left\{\log \left[ \varphi x^{\varphi - 1} \cdot \mathbb{I}(0 < x < 1) \right] \right\} \\ &= \exp \left\{ \log \varphi + (\varphi - 1) \log x + \log \mathbb{I}(0< x < 1)\right\} \\ &= \exp \left \{ \log \varphi - (1 - \varphi) \log x \right\} \mathbb{I}(0 < x < 1) \\ \end{align*}

Defining the quantities

\begin{align*} h(x) &= \mathbb{I}(0 < x < 1) \\ T(x) &= -\log x \\ \eta(\varphi) &= (1 - \varphi) \\ B(\varphi) &= - \log \varphi \\ \end{align*}

shows that $f(x; \varphi)$ is of the form $h(x) \exp(\eta(\varphi) T(x) - B(\varphi))$ and hence belongs to the one-parameter exponential family.

2. Show that $T = - \sum^n_{i=1} \log X_i$ is sufficient for $\varphi$.

Assuming the $X_1, \dots X_n$ are i.i.d., and using the above, the joint density $f(x_1, \dots, x_n; \varphi)$ is

\begin{align*} f(x_1, \dots, x_n; \varphi) &= \prod^n_{i=1} \exp \left \{ \log \varphi - (1 - \varphi) \log x_i \right\} \mathbb{I}(0 < x_i < 1) \\ &= \exp \left \{n \log \varphi + (1 - \varphi) \sum^n_{i=1} - \log x_i \right\} \prod^n_{i=1} \mathbb{I}(0 < x_i < 1) \\ \end{align*}

Define

\begin{align*} h(x_1, \dots, x_n) &= \prod^n_{i=1} \mathbb{I}(0 < x_i < 1) \\ g(T(x_1, \dots, x_n), \varphi) &= \exp \left \{n \log \varphi + (1 - \varphi) \sum^n_{i=1} - \log x_i \right\} \\ \end{align*}

Noting that $h$ does not depend on the parameter $\varphi$ and that $g$ depends on the data $X_1, \dots, X_n$ only through the statistic $T(X_1, \cdots, X_n) = \sum^n_{i=1} - \log X_i$, the factorisation theorem implies that $T$ is sufficient for $\varphi$.


Further comment.

In response to

It is easy to see that $T(X)$ is sufficient.

in the extract of Chapter 9.13.3 Exponential Families in All of Statistics: A Concise Course in Statistical Inference you have quoted, use the factorisation theorem to see this.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.