Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

How do I find the $Var(t(x))$ of an exponential family?

Suppose $X$ is a random variable whose distributions form an exponential family, with pmf or pdf given by $f(x|\theta)=h(x)c(\theta)exp\{w(\theta)t(x)\}.$ How can I find the $Var(t(x))$? I have seen ...
0
votes
0answers
23 views

expectation of conditional categorical distribution

I have the following network with $x$ and $y$ both binary: I've written $p(y|x)$ in exponential family form as: Now I need to get the expectation of this distribution, but I'm not sure how to ...
2
votes
1answer
83 views

Ambiguity with math notation

I was just solving several problems and get stuck into the one with some weird notation. I wasn't able to understand it, though I know the meaning behind the character (angle bracket) in more broader ...
0
votes
0answers
14 views

Moment parametrization for exponential families where the observation is not a sufficient statistics

Consider a probability distribution belonging to an exponential family $$ f(y; \theta) = c(y) \exp ( \eta(\theta)^\top T(y) - \kappa(\theta)) $$ In the case where $\eta(\theta) = \theta$ (always ...
0
votes
0answers
36 views

Expectation of exponential family distributions

Is there a closed form of the following marginal (one dimensional data) $\pi(\theta|y) = \mathbb{E}_{x \sim \pi_R(x|y)} \pi(\theta|x)$, where both $\pi, \pi_R$ are exponential family distributions?
0
votes
0answers
44 views

If $f(x|\theta)$ is conjugate to $p(\theta)$ then is $f(x|r\theta)$ conjugate to $p(\theta)$?

If exponential family $f(x|\theta)$ is conjugate to $p(\theta)$ then is $f(x|r\theta)$ for $r>0$ conjugate to $p(\theta)$? If not, what can we do about it in terms of sampling to make use of ...
0
votes
1answer
31 views

Percentages in utilization of different distributions

I'm sure that this question has been asked before on CV but, in drilling through many pages of previous CV questions, no matches surfaced. Regardless, I'm confident some observant participant will be ...
1
vote
0answers
36 views

Role of different exponential families in generalized linear models

I've gone through a variety of introductions to generalized linear models (GLM), and there's always a point in the discussion that confuses me. The story often begins saying that $P(y|x)$ belongs to a ...
1
vote
1answer
40 views

Definition of exponential family with dispersion parameter

I was recently reading a discussion of generalized linear models that considered the response to come from an exponential family with a dispersion parameter so $$ f(y|\theta,\phi) = \exp\left(\frac{y\...
19
votes
2answers
1k views

Advantages of the Exponential Family: why should we study it and use it?

So here I am studying inference. I would like that someone could enumerate the advantages of the exponential family. By exponential family, I mean the distributions which are given as \begin{align*} f(...
5
votes
1answer
59 views

Name and interpretation of “$h(x)$” in exponential family

The exponential family is defined (in many sources) as: $$p(x | \theta) = h(x) \exp\{\theta^TT(x) - A(\theta)\}$$ where: $T(x)$ is a sufficient statistic, $\theta$ is a canonical parameter, and $A(...
0
votes
1answer
55 views

Let $X\sim\text{Rayleigh}(\theta^{2})$. Prove that $T_{n}$ is consistent, given that $T_{n}(\textbf{X}) = \frac{1}{2n}\sum_{i=1}^{n}x^{2}_{i}$

Let $X\sim\text{Rayleigh}(\theta^{2})$. Prove that $T_{n}$ is consistent, given that $$T_{n}(\textbf{X}) = \frac{1}{2n}\sum_{i=1}^{n}x^{2}_{i}$$ MY ATTEMPT To begin with, let us notice that \begin{...
2
votes
0answers
54 views

Finding the mean and variance for pmf $P(X_i = x)=-\theta^x/x\log(1-\theta)$?

I'd like to verify that my working/thinking is correct. This is a problem from Keener's book, but the answer isn't provided. Let $X$ have distribution $P(X = x)=-\theta^x/x\log(1-\theta)$ for $x=1,2,\...
0
votes
0answers
27 views

How do we obtain the canonical parameter from the variance function in the context of exponential family distributions?

I am assuming the following definition of the exponential family: \begin{align*} f(y,\theta,\phi) = \exp\left\{\phi[y\theta - b(\theta)] + c(y,\phi)\right\} \end{align*} In such context, the variance ...
0
votes
2answers
70 views

Is the zero truncated Poisson Distribution part of the Exponential Family? [duplicate]

This is the density of a truncated Poisson: $$P(X = x \mid X > 0) = \frac{\lambda ^ x e^{- \lambda} }{x ! \left ( 1 - e^{- \lambda} \right )}$$ To show that it's member of the Exponential ...
0
votes
0answers
9 views

How do we get the canonical parameter $\theta$ from the exponential famility through the variance function?

The exponential family can be described by \begin{align*} f(y|\theta,\phi) = \exp\left\{\phi^{-1}(y\theta - b(\theta) + c(y,\phi)\right\} \end{align*} It also can be shown that $\textbf{E}(Y) = \mu = ...
0
votes
0answers
46 views

Does the family of binomial distributions conditioned on $X > 0$ belong to the exponential family?

Does the family of distributions where $p(x,\theta)$ is the conditional frequency function of a binomial $\mathcal{B}(n,\theta)$, variable $X$, given that $X > 0$, belong to the exponencial family? ...
0
votes
0answers
257 views

Exponential family and geometric distribution: how do we prove the sum of independent geometric random variables has negative binomial distribution?

Let $X$ be the number of failures before the first success in a sequence of Bernoulli trials with probability of success $\theta$. Then $P_{\theta}[X = k] = (1-\theta)^{k}\theta$, $k = 1,2,\ldots$ ...
2
votes
0answers
31 views

Maximum entropy probability distribution over non-negative support and finite mean?

I'm trying to derive which univariate probability distribution maximizes entropy, assuming finite mean $\mu$ and non-negative support $[0, \infty)$. I know that the answer is the exponential ...
0
votes
0answers
22 views

Does conjugate prior for natural exponential family needs jacobian to transform natural parameter back to original parameter?

From bayesian theory, we have that if $f(x|\eta) \propto \exp(\eta \cdot T(x)- A(\eta))$ - a natural exponential family, then the prior conjugate of $\eta$ is $\pi^*(\eta | \mu, \lambda) \propto \exp(\...
1
vote
0answers
35 views

If $X$ is from (canonical) exponential family, do we always know the distribution of $T(X)$?

Assume $X$ are generated by a distribution from exponential family, $$ f(X; \theta) = h(X)\exp\{\eta(\theta)T(X) - b(\theta)\}$$ After solving several exercises with various distribution functions, ...
0
votes
1answer
25 views

Reagarding the base measure h(x) in the exponential family

most authors define the canonical form of the exponential family as $$ p(\mathbf{x} | \boldsymbol{\theta})=h(\mathbf{x}) \exp (\boldsymbol{\eta}(\boldsymbol{\theta}) \cdot \mathbf{T}(\mathbf{x})-A(\...
0
votes
0answers
39 views

Proof that one distribution is a GLM (general linear model)

Given $Y_1, Y_2,...,Y_n$ i.i.d random variables, where $Y_i|x_i \sim N(\mu_i, \sigma^2)$ and $\mu_i = \beta_0 + \log(\beta_1 + \beta_2x_i)$. How do I proof that the distribution is a GLM (general ...
0
votes
0answers
25 views

Mixed parameterization of sample from normal distribution

I am studying exponential families and mixed parameterizations. Now, I am told that $$ \mathbf{\theta} = \begin{bmatrix}\mu\\ -\frac{1}{2\sigma^2}\end{bmatrix} $$ is the parameter in a variation-...
0
votes
0answers
51 views

obtain (minimal) sufficient statistic for $\gamma$ knowing the canonical statistic $\theta(\gamma)$

A statistical model for a data set y is an exponential family , with canonical parameter vector $\theta= (\theta_1,\theta_2,..\theta_k)$ and canonical statistic $t(y) = (t_1(y),t_2(y),..t_k(y))$ if ...
3
votes
0answers
136 views

UMVUE for $g(p) = \mathbb{E}_p[X^2]$, where X follows a geometric distribution

I have a random variable X with pmf $$p_\lambda(x) = (1-p)^{x-1}p, \ \ x = 1,2,3,\ldots, \ \ p \in (0,1)$$ and I am trying to find a UMVUE for $$g(p) = \mathbb{E}_p[X^2]$$. Here is my attempt so ...
1
vote
0answers
59 views

Find a UMP test in exponential family - first stage

Consider the exponential family $f(x;\theta) = c(\theta)\exp\{b(\theta) T(x))\}h(x)$ and I would like to find a UMP test for $$H_0: \theta \leq 0\quad \hbox{vs}\quad H_1: \theta > 0.$$ It is ...
1
vote
1answer
55 views

Is the marginal distribution of a maximum entropy distribution also a maximum entropy distribution?

I am considering under what circumstances maximum entropy distributions are closed under marginalization. The main case I am interested is the following setting: a finite set of discrete random ...
0
votes
0answers
120 views

Sufficient Statistic and MLE

Suppose $X_1, \dots, X_n \sim B(1,p)$. Show that a sufficient statistic for $\theta = (1-p)^2$ is $T(x) = \sum X_i$ and that the MLE for $\theta$ is $(1-\frac{1}{n}T)^2$. I am having a lot of ...
0
votes
1answer
45 views

What is matrix norm in (Collins, 2001)

I am studying "A generalization of PCA to the exponential family" (Collins et al., 2001) and I don't understand some notations. What is the meaning of the matrix squared norm on page 6 ? Is it a ...
7
votes
1answer
859 views

Distribution of sum of independent exponentials with random number of summands

Let $\tau_i\sim\exp\left(\lambda\right)$ be independent and identically distributed exponentials with parameter $\lambda$. Then, for given $n$, the sum of these values $$T_n := \sum_{i=0}^n \tau_i$$ ...
2
votes
1answer
121 views

Expectation of Sufficient Statistic

Consider $X \sim B(n,p)$ with pmf $P(X=x) = {{n}\choose{x}} p^x (1-p)^{n-x}$. The general exponential form of an exponential family distribution is $p(x|\theta) = f(x) g(\theta) e^{\phi(\theta)^T T(...
1
vote
1answer
195 views

Gamma Distribution Sufficient Statistics

I've been asked to show the gamma distribution can be written in the form $p(x|\alpha, \beta) = f(x) g(\alpha, \beta) e^{h(\alpha,\beta)^T T(x)}$ where $T(x)$ is a sufficient statistic. .... I have ...
3
votes
0answers
39 views

Which properties yield the exponential family of distributions?

It seems like every resource that discusses exponential families simply defines the family of distributions, explains why it's useful and then derives some of its properties. I have only seen one ...
1
vote
1answer
51 views

Relationship between exponential families and moment generating functions

I have recently been playing around with some change of measure arguments for shifting the mean of a sub-gaussian distribution. It occurred to me however, that sub-gaussianity might not be the natural ...
0
votes
2answers
99 views

Independence of Beta ratios of Gamma variates

If $X= x_1/(x_1+x_2+x_3)$ and $Y= x_2/(x_1+x_2+x_3)$ where $x_1, x_2, x_3$ are independent $\chi^2$-distributed random variables with d.f. $-n_1,n_2, n_3$ respectively. Are $X$ and $Y$ independent? I ...
2
votes
0answers
58 views

Are the cumulants of sufficient statistics finite for the exponential family?

Under what conditions are the cumulants of the sufficient statistic finite for an exponential family? If we have $$ p(x \mid \theta) = \exp(\theta \cdot T(x) - A(\theta)) $$ then the derivatives of $...
0
votes
0answers
1k views

Gamma distribution as a member of exponential family

In my lecture notes I have that the distribution of a random variable $Y$ is said to be in the exponential family if it can be written as $f(y;\theta)=exp(a(y)b(\theta)+c(\theta)+d(y))$, where $a,b,c$ ...
3
votes
0answers
123 views

Finding the UMVUE of $\theta^2$ where $f_X(x\mid\theta) =\frac{x}{\theta^2}e^{-x/\theta}I_{(0,\infty)}(x)$

Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\frac{x}{\theta^2}e^{-x/\theta}I_{(0,\infty)}(x)$$ where $\theta >0$. Give the UMVUE of ${\theta^2}$ I ...
2
votes
1answer
167 views

Some questions about exponential families

Regarding the book The Bayesian Choice I understand most of chapter three on exponential families, but there are two parts I have trouble understanding. The first is Consider$$f(x|\theta)=h(x)\...
0
votes
0answers
22 views

Predict node attribute from network variables

ERGM models the probability of a tie forming in a network. Is there a way of using ERGM, or an equivalent model, where the response variable is an attribute of the node, not a tie? Basically, turning ...
1
vote
1answer
106 views

UMP Test of $H_0 : \theta \leq 0.1$ vs. $H_1 : \theta \gt 0.1$ for iid Geometric Random Variables

Consider $90$ iid geometric random variables having parameter $\theta$ (and thus mean $1/\theta$), and a UMP test of the null hypothesis that $\theta \leq 0.1$ against the alternative that $\theta ...
0
votes
1answer
693 views

Optimization using the optim function in R with a two parameter exponential distribution

I'm having trouble trying to optimize a two-parameter exponential distribution, by finding the maximum likelihood function and then using the function optim() in R ...
1
vote
1answer
48 views

Is this a member of an exponential family?

I strongly believe that this distribution does not belong to the exponential family: $f(x;\theta) = \frac{\theta}{2}^{|x|}(1 - \theta)^{1-|x|}I_{\{-1, 0, 1\}}(x)$. I have to write $f(x;\theta)$ as $...
3
votes
0answers
74 views

What are necessary & sufficient conditions for exponential family representation to have complete statistic $T(X)$?

My textbook gives the following theorem for exponential families: Let $X_1, \dots, X_n$ be a random sample from an exponential family with pmf/pdf of the form $$f(x|\theta) = h(x) c(\theta) \exp (w(...
5
votes
1answer
730 views

Find UMVUE of $\theta$ where $f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$

As a slight modification of my previous problem: Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$$ where $\...
6
votes
2answers
820 views

Finding UMVUE of $\theta e^{-\theta}$ where $X_i\sim\text{Pois}(\theta)$

Suppose $X_1, X_2, . . . , X_n$ are i.i.d Poisson ($\theta$) random variables, where $\theta\in(0,\infty)$. Give the UMVUE of $\theta e^{-\theta}$ I found a similar problem here. I have that the ...
9
votes
1answer
276 views

Find UMVUE of $\frac{1}{\theta}$ where $f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$

Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$$ where $\theta >0$. Give the UMVUE of $\frac{1}{\theta}$ ...
2
votes
1answer
348 views

Is the canonical parameter (and therefore the canonical link function) for a Gamma not unique?

Consider $Y_1, \dots, Y_n$ independent from the Gamma distribution. For $y > 0$: $$\begin{align} f(y \mid \alpha, \beta) &= \dfrac{1}{\beta^{\alpha}\Gamma(\alpha)}y^{\alpha-1}e^{-y/\beta} \\ &...
4
votes
1answer
635 views

Deriving the canonical link for a binomial distribution

I define an exponential dispersion family as any distribution whose PMF/PDF is $$f(y \mid \boldsymbol\theta) = \exp\left\{\phi[y\theta - b(\theta)] + c(y, \phi) \right\}\text{, } y \in \Omega$$ where ...