Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

132 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
17
votes
0answers
761 views

Is there a general expression for ancillary statistics in exponential families?

An i.i.d sample $X_1,\dots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistic if $S(X)$ depends on the sample only through $\frac{X_1}{X_n},\cdots,\frac{X_{...
6
votes
0answers
2k views

“weight” input in glm.nb function in R. How exactly does the weight affect the likelihood?

I would like to understand how the weight argument of glm.nb is affecting the likelihood function. I understand that glm.nb find ...
5
votes
0answers
265 views

Let $X_1,X_2,\dots,X_n$ be random sample from Poisson($\theta$). Find MVUE of $e^{-2\theta}$

Question: Let $X_1,X_2,\dots,X_n$ be random sample from Poisson($\theta$). Find MVUE of $e^{-2\theta}$ My attempt has been by modifying the answer from this question: The Poisson distribution is a one-...
5
votes
0answers
202 views

Dispersion parameters in GLM

I'm trying to find the motivation behind the extended form of the exponential family of distributions in the fundamental paper on GLM by Nelder and Wedderburn (Generalized Linear Models, J. R. Statist....
5
votes
0answers
176 views

Exponential family where set of natural parameters has empty interior

In my math-stat class we have a theorem that goes: Let $\{P_\theta : \theta \in \Theta\}$ be a $k$ parameter exponential family (i.e. the density of a member of this family can be written as $f(\...
4
votes
0answers
41 views

Why is the EM algorithm well suited for exponential families?

I've been brushing up on the EM algorithm, and while I feel like I understand the basics, I keep seeing the claim made (e.g. here, here, among several others) that EM works particularly well for ...
4
votes
0answers
271 views

What are necessary & sufficient conditions for exponential family representation to have complete statistic $T(X)$?

My textbook gives the following theorem for exponential families: Let $X_1, \dots, X_n$ be a random sample from an exponential family with pmf/pdf of the form $$f(x|\theta) = h(x) c(\theta) \exp (w(...
4
votes
0answers
131 views

Show that there is no efficient estimator for the variance of a normal distribution using properties of the exponential family

I want to prove the statement in the title using the following statement from Wikipedia: it was proved that efficient estimation is possible only in an exponential family, and only for the natural ...
4
votes
1answer
264 views

Does stable distribution belong to exponential family?

According to Hougaard (1986), positive stable distribution on $\mathbb{R}^+$ belongs to exponential family, how about the case the support of stable distribution being less than zero? The purpose of ...
4
votes
0answers
939 views

Are there efficient estimators for the variance of an exponential family?

Let us consider the Gaussian model $\mathcal{N}(\mu,\sigma^2)$, where both $\mu$ and $\sigma$ are unknown. I have learnt that (for example, from Amari's information geometry book) the exponential ...
4
votes
0answers
119 views

Smooth expectations outside the exponential family

At page 85-86 of Young and Smith "Essentials of Statistical Inference" there is an interesting result. If $X$ is a r.v. distributed according to the exponential family and $\phi(x)$ is a bounded (but ...
3
votes
0answers
34 views

Can the parameter space of a single parameter exponential family in canonical form be a closed interval?

I know that the parameter space has to be convex (that I've proved using Jensen's inequality). Therefore, since it has to be a subset of $\mathbb R$, it follows that it has to be an interval or a ...
3
votes
0answers
112 views

Are there nonparametric generative models for datasets?

Typically when I see generative models, e.g., Latent Dirichlet Allocation (JMLR) or Linear/Quadratic Discriminant Analysis (wikipedia LDA), they are probabilistic models that belong to the exponential ...
3
votes
0answers
61 views

Maximum entropy probability distribution over non-negative support and finite mean?

I'm trying to derive which univariate probability distribution maximizes entropy, assuming finite mean $\mu$ and non-negative support $[0, \infty)$. I know that the answer is the exponential ...
3
votes
0answers
362 views

UMVUE for $g(p) = \mathbb{E}_p[X^2]$, where X follows a geometric distribution

I have a random variable X with pmf $$p_\lambda(x) = (1-p)^{x-1}p, \ \ x = 1,2,3,\ldots, \ \ p \in (0,1)$$ and I am trying to find a UMVUE for $$g(p) = \mathbb{E}_p[X^2]$$. Here is my attempt so ...
3
votes
0answers
282 views

Finding the UMVUE of $\theta^2$ where $f_X(x\mid\theta) =\frac{x}{\theta^2}e^{-x/\theta}I_{(0,\infty)}(x)$

Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\frac{x}{\theta^2}e^{-x/\theta}I_{(0,\infty)}(x)$$ where $\theta >0$. Give the UMVUE of ${\theta^2}$ I ...
3
votes
1answer
707 views

Conway–Maxwell–Poisson (CMP) distribution and exponential family

So I have a question here about the CMP distribution: My understanding is that $b(\theta)$ is only a function of $\theta$ but why is $v$ able to be included in that function, would $v$ not be a ...
3
votes
1answer
53 views

Effect of the measure on exponential families

This might be a very naive question. Wikipedia describes an exponential family as a distribution $$f(x \mid \theta) = h(x) \exp( - \theta x - A(\theta)),$$ where $$A(\theta) = \log\left(\int h(x)...
3
votes
0answers
317 views

Sufficient statistics and UMVUE for joint poisson, bernoulli

Given a pair $(X,Y)$ of r.v.s such that: $$X \sim \text{Poisson}(\lambda)\quad \text{and}\quad Y \sim B(\frac{\lambda}{1+\lambda})$$ with $X,Y$ independent, determine a one-dimensional sufficient ...
3
votes
0answers
317 views

MLE estimate of normal distribution

I am quoting this from Greene's econometrics book: The occasional statement that the properties of the MLE are only optimal in large samples is not true, however. It can be shown that when sampling ...
3
votes
0answers
87 views

Exponential family parameter estimation and fitting, references

First of all, I want to express my apologies if the question is too broad or wrong, but I am in need of references and I have no idea whom I can ask. If you are interested, the question comes from a ...
3
votes
0answers
96 views

The boundary proprerty of conjugate exponential family

Consider a conjugate prior in the k-parameter canonical exponential family: $$p(w|n_0,y_0)=c(n_0,y_0) \exp(n_0(y_0 w'-b(w))) \, ,$$ where $n_0>0, y_0 \in Y$ is the pseudo observation vector, and $...
2
votes
0answers
24 views

What family of full support probability distributions satisfy that the density of any point in the domain vanishes as the variance goes to infinity?

Let $f(x,\sigma^2)$ be a representative element of a family of PDF's with full support over the reals that is indexed by their variance $\sigma^2$. Under what general conditions of the family of ...
2
votes
0answers
60 views

Mean and variance of the Beta distribution using identities of exponential families

I was studying the part of exponential families from Statistical Inference (George Casella, Roger L. Berger) and they give the following definition of an exponential family: $$ f(x|\pmb{\theta}) = h(x)...
2
votes
0answers
34 views

GLMM and Exponential Family

I am studying now GLMMs (Generalized Linear Mixed Models). From my understanding, in order to estimate the parameters of this model, you need to arrive at the marginal probability by integrating a ...
2
votes
1answer
115 views

Is a maximum likelihood estimator in an exponential family always sufficient?

An exponential family (under natural parameterization) is such that $p(X|\eta)=h(X)\exp\{\eta^TT(X)-A(\eta)\}$, where $X$ is the data, $\eta$ is the natural parameter, and $h,T,A$ are some functions (...
2
votes
0answers
33 views

Why does the canonical parameter give a link function? Why does this relate $E[Y]$ to $x^T \beta$?

If I have a pdf in the form $f(y|\theta,\phi)=\text{exp}\bigg(\frac{y\theta-b(\theta)}{a(\phi)}+c(y,\phi)\bigg)$, then $\theta$ is called the canonical parameter. I'm told we can get a link function $...
2
votes
0answers
31 views

Show Moment estimator and MLE are the same only for exponential family

Show Moment estimator and MLE are the same only for exponential family. I know MM is variant while MLE is not under transformation. Will this be the starting point?
2
votes
0answers
177 views

Role of different exponential families in generalized linear models

I've gone through a variety of introductions to generalized linear models (GLM), and there's always a point in the discussion that confuses me. The story often begins saying that $P(y|x)$ belongs to a ...
2
votes
0answers
68 views

Finding the mean and variance for pmf $P(X_i = x)=-\theta^x/x\log(1-\theta)$?

I'd like to verify that my working/thinking is correct. This is a problem from Keener's book, but the answer isn't provided. Let $X$ have distribution $P(X = x)=-\theta^x/x\log(1-\theta)$ for $x=1,2,\...
2
votes
0answers
100 views

Are the cumulants of sufficient statistics finite for the exponential family?

Under what conditions are the cumulants of the sufficient statistic finite for an exponential family? If we have $$ p(x \mid \theta) = \exp(\theta \cdot T(x) - A(\theta)) $$ then the derivatives of $...
2
votes
0answers
37 views

Does any choice of sufficient statistic and natural parameter create valid exponential family pdfs?

So exponential families can be parameterized as $f(x|\theta) = h(x)e^{T(x)n(\theta)-A(n)}$. I'm trying to understand what the conditions are on $h(x)$, $T(x)$, and $n(\theta)$ are for $f(x|\theta)$ ...
2
votes
0answers
86 views

Is my understanding of “family of distributions” correct?

As I was looking to understand the concept of "family of distributions", I stumbled upon this answer. However, I was a bit confused with answer and I'm hoping that someone may be able to clarify for ...
2
votes
0answers
54 views

Tweedie index parameters are restricted: why?

The Tweedie distribution have variance like var(y) = $\phi \mu^p$ for any real p not between 0 and 1. I read in many places (even wikipedia) that p can take any real value except between 0 and 1. ...
2
votes
0answers
70 views

Parametric family problems

I came across such a problem that I cannot solve: Let $\mathcal{P} = \{\mathbb{P}_\theta : \theta \in \mathbb{R}\}$ be a parametric family over $\{0,1\} \times \mathbb{R}$ defined in the following ...
2
votes
0answers
154 views

Is the skew normal distribution a member of the exponential family

I'm trying to proof that the skew normal distribution ist part of the exponential family, but I cannot find a solution. So is it a member of the exponential family or are my assumptions misleading? ...
2
votes
0answers
749 views

Minimal sufficient statistic for two normal distributions

Let $X_1, . . . , X_m, Y_1, . . . , Y_n$ be independent with $X_i ∼ N(ξ, σ^2)$ and $ Y_j ∼ N(η, τ^2).$ What is the minimal sufficient statistic for $(ξ,η,σ^2)$ where $σ^2 = τ^2$? I've seen MSS ...
2
votes
0answers
205 views

Sufficient statistics in multiparameter exponential family

I'm trying to work through a theorem in the Lehmann statistical inference book and I'm confused about a proof. They are proving that a set of tests are UMP unbiased level-alpha tests for a series of ...
2
votes
0answers
75 views

Expected time between two events

I'm having trouble with the following problem: Consider a game between two players A and B. Player A must complete three tasks each of which take an exponentially distributed amount of time with ...
2
votes
0answers
149 views

Variance of Distributions from the Exponential Family

I want to understand how the variance of an exponential family behaves. To take a very concrete example. Let consider the unit ball $B$ in d dimensions. Consider the following distribution over unit ...
2
votes
0answers
249 views

Is the Dirichlet compound multinomial (DCM) distribution in the exponential family?

It occurs to me that the DCM (a.k.a multivariate Pòlya) distribution can be written in the exponential form when the number of draws, $n_1+n_2+\ldots+n_k=N$, is known: $$ p(n|a) = exp \left(tr \left( \...
2
votes
0answers
64 views

Implication of Theorem on 'Empirical Bayes optimality' (Morris 1983)

My question is about Theorem 1 from Morris 1983; Parametric Empirical Bayes Inference (http://www.jstor.org/stable/2287098?seq=1#page_scan_tab_contents). Suppose Y has a univariate natural ...
2
votes
0answers
516 views

KL-divergence as a negative log likelihood for exponential families

I am reading Distributed Estimation, Information Loss and Exponential Families, where the authors consider and compare two estimators for $\theta$ in the parametric model $p(x\mid\theta)$: the ...
2
votes
0answers
647 views

Most Powerful Test; Two-Parameter Normal Distribution

Is it possible to show that the two-parameter Normal distribution has monotone likelihood ratio? EDIT: This is actually part of a larger problem. We have a random sample from $\mathcal N(\mu, \sigma^...
2
votes
0answers
240 views

3 parameter Exponential Family and sufficient statistics

This is a homework problem. I've derived the following distribution from an earlier part in the problem $$ f_{X_1,X_2}(x_1,x_2) = \dfrac{\Gamma(x_1+x_2+r)\alpha_1^{x_1}\alpha_2^{x_2}\theta^r}{\Gamma(r)...
2
votes
0answers
2k views

Hypothesis Testing on Exponential distributions

Let $X_1, \dots, X_n$ be independent exponential $(\theta)$ random variables. Suppose we are interested in testing $H_0: \theta = \theta_0 = 1$ versus $H_A: \theta = \theta_1>1$. Consider two tests ...
2
votes
0answers
198 views

Confidence intervals for log normal responses

I got this assignment from Generalized Linear Model class. At first glace it looked like it is an easy task, but there are a lot of subtle (at least in my opinion) things, which I would like to ...
1
vote
0answers
72 views

UMP test for exponential family when sufficient statistics $T$ is a vector

Assume we have a random sample $X_1,\dots,X_n$ from a distribution of the form $f(x_i;\theta) = h(x)g(\theta)\exp(\eta(\theta) T(x))$ and we wish to test $H_0: \theta \leq \theta_0, H_1: \theta > \...
1
vote
0answers
129 views

PDF of Beta Distribution written as exponential family form

I am trying to write the pdf of a beta random variable in its biparametric canonic form such as: Function 1 $$ f_Y(y; \theta, \phi) = exp \{ \phi[y \theta - b(\theta)] + c(y, \phi) \} \mathbb{1}_A(...
1
vote
0answers
26 views

Show bivariate normal distribution with non-diagonal covariance belongs to curved exponential family?

Question: Suppose that $(X_{i}, Y_{i})$, $i = 1, \dots ,n$ are sampled i.i.d. from the two-dimensional normal distribution $$ \begin{bmatrix} X & Y \end{bmatrix} \sim \mathcal{N}\left( \begin{...