Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

0
votes
2answers
39 views

Is the zero truncated Poisson Distribution part of the Exponential Family? [duplicate]

This is the density of a truncated Poisson: $$P(X = x \mid X > 0) = \frac{\lambda ^ x e^{- \lambda} }{x ! \left ( 1 - e^{- \lambda} \right )}$$ To show that it's member of the Exponential ...
0
votes
0answers
6 views

How do we get the canonical parameter $\theta$ from the exponential famility through the variance function?

The exponential family can be described by \begin{align*} f(y|\theta,\phi) = \exp\left\{\phi^{-1}(y\theta - b(\theta) + c(y,\phi)\right\} \end{align*} It also can be shown that $\textbf{E}(Y) = \mu = ...
-1
votes
0answers
22 views

How do we prove the bivariate normal distribution belongs to the exponential family?

Let $\textbf{X} = ((X_{1},Y_{1}),(X_{2},Y_{2}),\ldots,(X_{n},Y_{n}))$ be a sample from a bivariate normal population. Show that the distributions of $\textbf{X}$ form a five-parameter exponential ...
0
votes
0answers
26 views

Does the family of binomial distributions conditioned on $X > 0$ belong to the exponential family?

Does the family of distributions where $p(x,\theta)$ is the conditional frequency function of a binomial $\mathcal{B}(n,\theta)$, variable $X$, given that $X > 0$, belong to the exponencial family? ...
0
votes
0answers
29 views

Exponential family and geometric distribution: how do we prove the sum of independent geometric random variables has negative binomial distribution?

Let $X$ be the number of failures before the first success in a sequence of Bernoulli trials with probability of success $\theta$. Then $P_{\theta}[X = k] = (1-\theta)^{k}\theta$, $k = 1,2,\ldots$ ...
2
votes
0answers
20 views

Maximum entropy probability distribution over non-negative support and finite mean?

I'm trying to derive which univariate probability distribution maximizes entropy, assuming finite mean $\mu$ and non-negative support $[0, \infty)$. I know that the answer is the exponential ...
0
votes
0answers
9 views

Does conjugate prior for natural exponential family needs jacobian to transform natural parameter back to original parameter?

From bayesian theory, we have that if $f(x|\eta) \propto \exp(\eta \cdot T(x)- A(\eta))$ - a natural exponential family, then the prior conjugate of $\eta$ is $\pi^*(\eta | \mu, \lambda) \propto \exp(\...
1
vote
0answers
26 views

If $X$ is from (canonical) exponential family, do we always know the distribution of $T(X)$?

Assume $X$ are generated by a distribution from exponential family, $$ f(X; \theta) = h(X)\exp\{\eta(\theta)T(X) - b(\theta)\}$$ After solving several exercises with various distribution functions, ...
0
votes
1answer
17 views

Reagarding the base measure h(x) in the exponential family

most authors define the canonical form of the exponential family as $$ p(\mathbf{x} | \boldsymbol{\theta})=h(\mathbf{x}) \exp (\boldsymbol{\eta}(\boldsymbol{\theta}) \cdot \mathbf{T}(\mathbf{x})-A(\...
0
votes
0answers
30 views

Proof that one distribution is a GLM (general linear model)

Given $Y_1, Y_2,...,Y_n$ i.i.d random variables, where $Y_i|x_i \sim N(\mu_i, \sigma^2)$ and $\mu_i = \beta_0 + \log(\beta_1 + \beta_2x_i)$. How do I proof that the distribution is a GLM (general ...
0
votes
0answers
23 views

Mixed parameterization of sample from normal distribution

I am studying exponential families and mixed parameterizations. Now, I am told that $$ \mathbf{\theta} = \begin{bmatrix}\mu\\ -\frac{1}{2\sigma^2}\end{bmatrix} $$ is the parameter in a variation-...
0
votes
0answers
31 views

obtain (minimal) sufficient statistic for $\gamma$ knowing the canonical statistic $\theta(\gamma)$

A statistical model for a data set y is an exponential family , with canonical parameter vector $\theta= (\theta_1,\theta_2,..\theta_k)$ and canonical statistic $t(y) = (t_1(y),t_2(y),..t_k(y))$ if ...
3
votes
0answers
102 views

UMVUE for $g(p) = \mathbb{E}_p[X^2]$, where X follows a geometric distribution

I have a random variable X with pmf $$p_\lambda(x) = (1-p)^{x-1}p, \ \ x = 1,2,3,\ldots, \ \ p \in (0,1)$$ and I am trying to find a UMVUE for $$g(p) = \mathbb{E}_p[X^2]$$. Here is my attempt so ...
1
vote
1answer
40 views

Is the marginal distribution of a maximum entropy distribution also a maximum entropy distribution?

I am considering under what circumstances maximum entropy distributions are closed under marginalization. The main case I am interested is the following setting: a finite set of discrete random ...
0
votes
0answers
99 views

Sufficient Statistic and MLE

Suppose $X_1, \dots, X_n \sim B(1,p)$. Show that a sufficient statistic for $\theta = (1-p)^2$ is $T(x) = \sum X_i$ and that the MLE for $\theta$ is $(1-\frac{1}{n}T)^2$. I am having a lot of ...
0
votes
1answer
45 views

What is matrix norm in (Collins, 2001)

I am studying "A generalization of PCA to the exponential family" (Collins et al., 2001) and I don't understand some notations. What is the meaning of the matrix squared norm on page 6 ? Is it a ...
7
votes
1answer
522 views

Distribution of sum of independent exponentials with random number of summands

Let $\tau_i\sim\exp\left(\lambda\right)$ be independent and identically distributed exponentials with parameter $\lambda$. Then, for given $n$, the sum of these values $$T_n := \sum_{i=0}^n \tau_i$$ ...
2
votes
1answer
57 views

Expectation of Sufficient Statistic

Consider $X \sim B(n,p)$ with pmf $P(X=x) = {{n}\choose{x}} p^x (1-p)^{n-x}$. The general exponential form of an exponential family distribution is $p(x|\theta) = f(x) g(\theta) e^{\phi(\theta)^T T(...
1
vote
1answer
126 views

Gamma Distribution Sufficient Statistics

I've been asked to show the gamma distribution can be written in the form $p(x|\alpha, \beta) = f(x) g(\alpha, \beta) e^{h(\alpha,\beta)^T T(x)}$ where $T(x)$ is a sufficient statistic. .... I have ...
3
votes
0answers
34 views

Which properties yield the exponential family of distributions?

It seems like every resource that discusses exponential families simply defines the family of distributions, explains why it's useful and then derives some of its properties. I have only seen one ...
1
vote
1answer
38 views

Relationship between exponential families and moment generating functions

I have recently been playing around with some change of measure arguments for shifting the mean of a sub-gaussian distribution. It occurred to me however, that sub-gaussianity might not be the natural ...
0
votes
2answers
94 views

Independence of Beta ratios of Gamma variates

If $X= x_1/(x_1+x_2+x_3)$ and $Y= x_2/(x_1+x_2+x_3)$ where $x_1, x_2, x_3$ are independent $\chi^2$-distributed random variables with d.f. $-n_1,n_2, n_3$ respectively. Are $X$ and $Y$ independent? I ...
2
votes
0answers
49 views

Are the cumulants of sufficient statistics finite for the exponential family?

Under what conditions are the cumulants of the sufficient statistic finite for an exponential family? If we have $$ p(x \mid \theta) = \exp(\theta \cdot T(x) - A(\theta)) $$ then the derivatives of $...
0
votes
0answers
480 views

Gamma distribution as a member of exponential family

In my lecture notes I have that the distribution of a random variable $Y$ is said to be in the exponential family if it can be written as $f(y;\theta)=exp(a(y)b(\theta)+c(\theta)+d(y))$, where $a,b,c$ ...
3
votes
0answers
108 views

Finding the UMVUE of $\theta^2$ where $f_X(x\mid\theta) =\frac{x}{\theta^2}e^{-x/\theta}I_{(0,\infty)}(x)$

Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\frac{x}{\theta^2}e^{-x/\theta}I_{(0,\infty)}(x)$$ where $\theta >0$. Give the UMVUE of ${\theta^2}$ I ...
2
votes
1answer
163 views

Some questions about exponential families

Regarding the book The Bayesian Choice I understand most of chapter three on exponential families, but there are two parts I have trouble understanding. The first is Consider$$f(x|\theta)=h(x)\...
0
votes
0answers
19 views

Predict node attribute from network variables

ERGM models the probability of a tie forming in a network. Is there a way of using ERGM, or an equivalent model, where the response variable is an attribute of the node, not a tie? Basically, turning ...
1
vote
1answer
82 views

UMP Test of $H_0 : \theta \leq 0.1$ vs. $H_1 : \theta \gt 0.1$ for iid Geometric Random Variables

Consider $90$ iid geometric random variables having parameter $\theta$ (and thus mean $1/\theta$), and a UMP test of the null hypothesis that $\theta \leq 0.1$ against the alternative that $\theta ...
0
votes
1answer
480 views

Optimization using the optim function in R with a two parameter exponential distribution

I'm having trouble trying to optimize a two-parameter exponential distribution, by finding the maximum likelihood function and then using the function optim() in R ...
1
vote
1answer
47 views

Is this a member of an exponential family?

I strongly believe that this distribution does not belong to the exponential family: $f(x;\theta) = \frac{\theta}{2}^{|x|}(1 - \theta)^{1-|x|}I_{\{-1, 0, 1\}}(x)$. I have to write $f(x;\theta)$ as $...
2
votes
0answers
60 views

What are necessary & sufficient conditions for exponential family representation to have complete statistic $T(X)$?

My textbook gives the following theorem for exponential families: Let $X_1, \dots, X_n$ be a random sample from an exponential family with pmf/pdf of the form $$f(x|\theta) = h(x) c(\theta) \exp (w(...
5
votes
1answer
558 views

Find UMVUE of $\theta$ where $f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$

As a slight modification of my previous problem: Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$$ where $\...
6
votes
2answers
542 views

Finding UMVUE of $\theta e^{-\theta}$ where $X_i\sim\text{Pois}(\theta)$

Suppose $X_1, X_2, . . . , X_n$ are i.i.d Poisson ($\theta$) random variables, where $\theta\in(0,\infty)$. Give the UMVUE of $\theta e^{-\theta}$ I found a similar problem here. I have that the ...
9
votes
1answer
235 views

Find UMVUE of $\frac{1}{\theta}$ where $f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$

Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$$ where $\theta >0$. Give the UMVUE of $\frac{1}{\theta}$ ...
2
votes
1answer
191 views

Is the canonical parameter (and therefore the canonical link function) for a Gamma not unique?

Consider $Y_1, \dots, Y_n$ independent from the Gamma distribution. For $y > 0$: $$\begin{align} f(y \mid \alpha, \beta) &= \dfrac{1}{\beta^{\alpha}\Gamma(\alpha)}y^{\alpha-1}e^{-y/\beta} \\ &...
4
votes
1answer
410 views

Deriving the canonical link for a binomial distribution

I define an exponential dispersion family as any distribution whose PMF/PDF is $$f(y \mid \boldsymbol\theta) = \exp\left\{\phi[y\theta - b(\theta)] + c(y, \phi) \right\}\text{, } y \in \Omega$$ where ...
2
votes
0answers
24 views

Does any choice of sufficient statistic and natural parameter create valid exponential family pdfs?

So exponential families can be parameterized as $f(x|\theta) = h(x)e^{T(x)n(\theta)-A(n)}$. I'm trying to understand what the conditions are on $h(x)$, $T(x)$, and $n(\theta)$ are for $f(x|\theta)$ ...
0
votes
0answers
81 views

Support of an exponential family in canonical form

Suppose $Y$ is a random variable in the exponential family, with pmf/pdf $$f(y) = \exp\left[\sum_{j=1}^{s}\theta_jT_j(y)-B(\theta)+c(y) \right]$$ for $y \in \Omega$ (the support of $Y$), and where $$...
5
votes
1answer
93 views

Jointly sufficient statistics of a multi-parameter exponential family

Let $f_X$ be a joint density function that comes from an $s$-parameter exponential family with sufficient statistics $(T_1, T_2, \dots, T_s)$ so that the density $f_X$ can be expressed as $$f_{X|\...
4
votes
1answer
106 views

How many natural parameters are really in the exponential family conjugate prior?

The exponential family with natural parameter $\theta$ can be written $$ p(x|\theta)=h_\ell(x)\exp(\theta^Tt(x)-a_\ell(\theta)) $$ with conjugate prior $$ p(\theta|\lambda)=h_c(\theta)\exp(\lambda_1^T\...
1
vote
0answers
134 views

Weibull distribution is not a glm, why? [duplicate]

It is a generalized linear model with a linear combination of covariates related to the response via a canonical link function. Why weibull distribution is not a glm?
0
votes
0answers
34 views

Unsure about the distribution of the canonical statistic [duplicate]

Iam a bit unsure if the distribution of the canonical statistic below is correct ? I suspect i would need idenpendent variables to show the correct answer. Let $Y_1,Y_2,...Y_n$ be a sample form a ...
5
votes
1answer
162 views

Is the negative exponential distribution a member of the exponential family?

Please correct me if I am wrong. The general form of k-parameter exponential family is $f(x;\boldsymbol{\theta}) = a(\boldsymbol{\theta})g(x) \exp\{\sum_{i=1}^{k}b_(\boldsymbol{\theta}) R_i(x)\}$ ...
3
votes
1answer
62 views

Sampling parameters from exponential family

So suppose PDF $f_{X|\theta}(x_1,...,x_n;\theta_1,...,\theta_m)$ is from the exponential family. Is there any theory or general guidelines for sampling parameters from this PDF? This question is not ...
0
votes
0answers
465 views

Show the lognormal distribution belongs in the exponential family

I just wanted to verify that my attempt is correct. Thank you in advance for reading this. A distribution is said to belong in the exponential family if its probability density function can be ...
0
votes
0answers
123 views

Convert exponential to Bernoulli

If I have a binary variable x, with distribution p(x) = exp{Cx}, how do I put this into the canonical Bernoulli form so as to get the probability p that x=1 that I ...
3
votes
2answers
138 views

intuitive interpretation of canonical parameterization of beta distribution

For exponential family, e.g. Beta distirbution, someone argues that the canonical parameterization is better than the traditional $Beta(\alpha,\beta)$ way. The canonical parameters are defined as $n^...
1
vote
0answers
28 views

Why do we care about maximum entropy? [duplicate]

One justification for the ubiquity of the (multivariate) normal distribution in statistical/machine learning modeling is that it maximizes entropy among distributions with mean $\mu$ and variance ...
1
vote
0answers
17 views

Tools for self-study: Constructing and understanding systems of distributional families

I am looking for a resource that probably does not exist, but, well, hope springs eternal. I have become increasingly interested in the process by which distributions are discovered or invented. ...
0
votes
0answers
181 views

Minimal sufficient statistic for multivariate normal

I have the following iid. variables $X_1,..,X_n,Y_1,..,Y_m$ with distribution $X_i\sim N(\mu_1,\sigma_1^2), Y_j\sim N(\mu_2,\sigma_2^2)$. How do I find the minimal sufficient statistic for $(\mu_1,\...