Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
10
votes
1answer
8k views

ML estimate of exponential distribution (with censored data)

In Survival Analysis, you assume the survival time of a r.v. $X_i$ to be exponentially distributed. Considering now that I have $x_1,\dots,x_n$ "outcomes" of i.i.d r.v.'s $X_i$. Only some proportion ...
15
votes
3answers
4k views

Definition of family of a distribution?

Does a family of a distribution have a different definition for statistics than in other disciplines? In general, a family of curves is a set of curves, each of which is given by a function or ...
13
votes
1answer
744 views

Aside from the exponential family, where else can conjugate priors come from?

Do all conjugate priors have to come from the exponential family? If not, what other families are known to have/produce conjugate priors?
19
votes
2answers
4k views

Does log likelihood in GLM have guaranteed convergence to global maxima?

My questions are: Are generalized linear models (GLMs) guaranteed to converge to a global maximum? If so, why? Furthermore, what constraints are there on the link function to insure convexity? My ...
9
votes
2answers
710 views

Find the joint distribution of $X_1$ and $\sum_{i=1}^n X_i$

This question is from Robert Hogg's Introduction to Mathematical Statistics 6th version question 7.6.7. The problem is : Let a random sample of size $n$ be taken from a distribution with the pdf $...
-3
votes
1answer
168 views

Is Poisson–Lindley an exponential family? If not, why? [closed]

$$\begin{aligned}f_Y(y_i)&=\frac{{\theta_i}^2\left(y_i+\theta_i+2\right)}{\left(1+\theta_i\right)^{y_i+3}}\\ &=\exp\ \log\left[\frac{{\theta_i}^2\left(y_i+\theta_i+2\right)}{\left(1+\theta_i\...
8
votes
1answer
3k views

Why do we assume the exponential family in the GLM context?

When I first learned about Generalized Linear Models I thought that the assumption that the dependent variable follows some distribution from the exponential family was made to simplify calculations. ...
4
votes
1answer
407 views

Definition of exponential family with dispersion parameter

I was recently reading a discussion of generalized linear models that considered the response to come from an exponential family with a dispersion parameter so $$ f(y|\theta,\phi) = \exp\left(\frac{y\...
3
votes
1answer
1k views

Is the canonical parameter (and therefore the canonical link function) for a Gamma not unique?

Consider $Y_1, \dots, Y_n$ independent from the Gamma distribution. For $y > 0$: $$\begin{align} f(y \mid \alpha, \beta) &= \dfrac{1}{\beta^{\alpha}\Gamma(\alpha)}y^{\alpha-1}e^{-y/\beta} \\ &...
29
votes
1answer
2k views

When if ever is a median statistic a sufficient statistic?

I came across a casual remark on The Chemical Statistician that a sample median could often be a choice for a sufficient statistic but, besides the obvious case of one or two observations where it ...
2
votes
1answer
6k views

Sufficient statistic for bivariate or multivariate normal

Here on page 7, example 2.7. The claim is that sufficient statistics for $d$ dimensional multivariate normal $\mathbf{x}_i \sim N(\vec{\mu}, \Sigma)$ is $$\left(n^{-1}\sum_{i=1}^n \mathbf{x}_i, \hat{\...
21
votes
3answers
3k views

Why doesn't the exponential family include all distributions?

I am reading the book: Bishop, Pattern Recognition and Machine Learning (2006) which defines the exponential family as distributions of the form (Eq. 2.194): $$ p(\mathbf x|\boldsymbol \eta) = h(\...
5
votes
1answer
3k views

Are complete statistics always sufficient?

I know that a complete sufficient statistic $T$ is such that 1) $T$ is sufficient for $\theta$, unknown parameter and 2) $T$ is complete. So, is it always the case? If the answer is not, what ...
10
votes
1answer
991 views

Find UMVUE of $\frac{1}{\theta}$ where $f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$

Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$$ where $\theta >0$. Give the UMVUE of $\frac{1}{\theta}$ ...
16
votes
2answers
6k views

Kullback–Leibler divergence between two gamma distributions

Choosing to parameterize the gamma distribution $\Gamma(b,c)$ by the pdf $g(x;b,c) = \frac{1}{\Gamma(c)}\frac{x^{c-1}}{b^c}e^{-x/b}$ The Kullback-Leibler divergence between $\Gamma(b_q,c_q)$ and $\...
10
votes
1answer
843 views

Unbiased estimator with minimum variance for $1/\theta$

Let$ X_1, ...,X_n$ be a random sample feom a distribution $Geometric(\theta)$ for $0<\theta<1$. I.e, $$p_{\theta}(x)=\theta(1-\theta)^{x-1} I_{\{1,2,...\}}(x)$$ Find the unbiased estimator ...
7
votes
1answer
4k views

What is exponential family criterion to test the sufficiency and completeness of an estimator?

I am struggling to understand the following result from Casella and Berger about sufficiency and completeness for exponential families: Let $X_{1},X_{2},...,X_{n}$ be iid observations from an ...
6
votes
1answer
1k views

Minimum dimension of sufficient statistics

Suppose that we have a parameter of $k$-dimensions. Say, for example, for $N(u,\theta)$ both unknown then the parameter is of two dimensions, and $n$ i.i.d. observations. Is it possible to find a ...
11
votes
1answer
3k views

How to derive the conjugate prior of an exponential family distribution

I am trying to derive the conjugate prior of the univariate Gaussian distribution over both the mean and the precision. I know that the prior I'm looking for is the normal-gamma distribution, but the ...
3
votes
1answer
116 views

Asymptotic normality of MLE

We know under regularity conditions the MLE is asymptotically normal. Usually, it is said that in practice it's hard to check these assumptions. However, I wondered whether we can claim that these ...
4
votes
1answer
603 views

Can a generalized linear model use shifted exponential as residual distribution?

I am facing a modeling problem: $t_{ij} = D_i + T_j + \epsilon_{ij}, i=0...641, j\in\mathbb{N}$ where $\epsilon_{ij}$ follows exponential distribution, $\epsilon_{ij} \sim \lambda e^{-\lambda t}, \...
3
votes
1answer
584 views

explanation of why an UMVUE doesn't necessarily have to achieve the CRLB?

I'm studying uniformly minimum variance unbiased estimator(UMVUE). I have seen question on this site asking why the UMVUE doesn't achieve the CRLB(Cramer Rao lower bound), and all of the answers have ...
1
vote
1answer
880 views

Zero-truncated Poisson model

In the theory of generalised linear models, you may use the exponential family to find the mean and variance of certain distributions. How would the mean and expectation of the zero-truncated Poisson ...
0
votes
1answer
500 views

Moment generating function of the natural sufficient statistics of Gamma distribution

I have $X_1,X_2 ... X_n$ Gamma-distributed r.v with density: First of all I showed, that Gamma distribution belongs to exponential family and can be represented in form I found, that for Gamma ...
0
votes
0answers
61 views

Why are/aren't these functions members of the exponential family? [duplicate]

I am currently trying to learn about the exponential family of distributions. I am trying to understand this question and this answer from Xi'an. I have the same function: $$f(x; \sigma, \tau)= \begin{...
10
votes
1answer
3k views

Exponential Family: Observed vs. Expected Sufficient Statistics

My question arises from reading reading Minka's "Estimating a Dirichlet Distribution", which states the following without proof in the context of deriving a maximum-likelihood estimator for a ...
21
votes
2answers
2k views

Advantages of the Exponential Family: why should we study it and use it?

So here I am studying inference. I would like that someone could enumerate the advantages of the exponential family. By exponential family, I mean the distributions which are given as \begin{align*} f(...
16
votes
3answers
10k views

Poisson is to exponential as Gamma-Poisson is to what?

A Poisson distribution can measure events per unit time, and the parameter is $\lambda$. The exponential distribution measures the time until next event, with the parameter $\frac{1}{\lambda}$. One ...
9
votes
1answer
1k views

What is meant by the term 'exponential family'? Why it is named so?

I have come across the term exponential family. The Bernoulli, Gaussian and many more distributions come under this exponential family. What would be the commonalities between them?
8
votes
2answers
4k views

Finding UMVUE of $\theta e^{-\theta}$ where $X_i\sim\text{Pois}(\theta)$

Suppose $X_1, X_2, . . . , X_n$ are i.i.d Poisson ($\theta$) random variables, where $\theta\in(0,\infty)$. Give the UMVUE of $\theta e^{-\theta}$ I found a similar problem here. I have that the ...
7
votes
2answers
746 views

Sufficient Statistic for non-exponential family distribution

Question: Let $X_1,X_2,\ldots,X_n$ be an iid sample from $N(\theta , 4 \theta^2 )$. I want to show that this model is not a member of the exponential family and to find a sufficient statistic for $\...
5
votes
0answers
202 views

Dispersion parameters in GLM

I'm trying to find the motivation behind the extended form of the exponential family of distributions in the fundamental paper on GLM by Nelder and Wedderburn (Generalized Linear Models, J. R. Statist....
5
votes
1answer
4k views

Show that Weibull distribution can be transformed to exponential family

How do I show the Weibull distribution $f:(y; \lambda, \rho)$ can be transformed to the exponential family using the transformation $z=y^\lambda$? I know the form I need to express it in is $$\...
5
votes
2answers
7k views

Cramér-Rao Lower Bound for Exponential Families

I am having a problem with applying the Cramér-Rao inequality to identify the lower bound for the variance of an unbiased estimator and hoped that you guys could help me. The problem is the following: ...
5
votes
1answer
853 views

complete sufficient statistic exercise

I have to find complete sufficient statistic of the following pdf $$f(x|\theta)=\frac{\theta}{(1+x)^{(1+\theta)}},\quad 0<x<\infty,\theta>0.$$ My Attempt: The joint density $$f(\mathbf x|\...
3
votes
0answers
111 views

Are there nonparametric generative models for datasets?

Typically when I see generative models, e.g., Latent Dirichlet Allocation (JMLR) or Linear/Quadratic Discriminant Analysis (wikipedia LDA), they are probabilistic models that belong to the exponential ...
7
votes
1answer
391 views

Is the negative exponential distribution a member of the exponential family?

Please correct me if I am wrong. The general form of $k$-parameter exponential family is $$f(x;\boldsymbol{\theta}) = a(\boldsymbol{\theta})g(x) \exp\{\sum_{i=1}^{k}b(\boldsymbol{\theta}) R_i(x)\}$$ ...
7
votes
2answers
2k views

Role of base measure in exponential family

An exponential family distribution $p$ in the canonical form can be written as $p(x|\theta) = h(x)\exp(\theta^\top T(x) - A(\theta))$ where $A(\theta)$ is the log partition function, $T(x)$ is the ...
6
votes
1answer
676 views

Skewness of Tweedie distribution

Tweedie distributions are a family of distributions from the exponential dispersion family that have power-law mean-variance relationship: \begin{align} \mathbb E[X] &= \mu \\ \operatorname{Var}[...
3
votes
1answer
707 views

Conway–Maxwell–Poisson (CMP) distribution and exponential family

So I have a question here about the CMP distribution: My understanding is that $b(\theta)$ is only a function of $\theta$ but why is $v$ able to be included in that function, would $v$ not be a ...
3
votes
1answer
728 views

GLM (conditional/unconditional) distribution

Based on my readings about GLM, I am pretty sure that when we say the distribution of the response variable $y$ is a member of exponential family of distribution, what we really mean is that ...
2
votes
2answers
7k views

Best Fit for Exponential Data

I'm trying to better understand some of the theory behind fitting models that have a nonlinear link between the response and the predictors. ...
1
vote
1answer
89 views

Exponential family definition appears vacuous

I am going through Michael Jordan's notes on exponential families and an exponential family is defined w.r.t. functions $h(\cdot), T(\cdot)$, and parameter $\eta$ such that $$ p(x | \eta) = h(x) \exp\...
1
vote
1answer
75 views

Derivation of the Objective Function for Expectation Propagation

I was reading Expectation Propagation As A Way Of Life and the original paper by Minka Expectation Propagation for Approximate Bayesian Inference and they both say that a fixed point of the EP ...
6
votes
1answer
591 views

Is the t distribution a member of the exponential family?

From what I understand, the exponential family is defined as $$f(y;\theta,\phi) = \exp\left(\frac{y\theta - b(\theta)}{a(\phi)}+c(y,\phi)\right) $$ I've read (but not seen shown anywhere), that the ...
6
votes
1answer
2k views

Deriving the canonical link for a binomial distribution

I define an exponential dispersion family as any distribution whose PMF/PDF is $$f(y \mid \boldsymbol\theta) = \exp\left\{\phi[y\theta - b(\theta)] + c(y, \phi) \right\}\text{, } y \in \Omega$$ where ...
6
votes
1answer
684 views

Are the Boltzmann distributions an exponential family?

Background: When introducing exponential families in class today, my professor said that the cumulant generating function in the definition is essentially a normalizing constant, or a renormalization ...
5
votes
1answer
733 views

Does exponential family of distributions have finite expected value?

I am curious about this question, because in definitions I have never seen this property. Is it true? If yes, why?
4
votes
1answer
2k views

Proving a distribution is a member of the simple exponential family

Does anyone have any tips/ideas/method for proving that a distribution is a member of the simple exponential family (SEF)? Or is the process unique to each distribution? For example, I am trying to ...
4
votes
1answer
413 views

What the dimension of an exponential family tell us about that family?

In Wikipedia it is stated that: A vector exponential family is said to be curved if the dimension of $$ {\boldsymbol \theta} = \left (\theta_1, \theta_2, \ldots, \theta_d \right )^T$$ is ...