Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
0
votes
1answer
19 views

Sufficient Statistic for Absolutely Continuous Distribution [duplicate]

The following is a homework problem. Please tell me if my solution is correct and if not please point out my mistakes. Let $x_{1}, x_{2},...,x_{M}$ be i.i.d. samples from the absolute continuous ...
2
votes
1answer
23 views

Is a parameter-independent support sufficient to belong to an exponential family?

From Wikipedia (emphasis mine): As a counterexample if these conditions are relaxed, the family of uniform distributions (either discrete or continuous, with either or both bounds unknown) has a ...
0
votes
0answers
15 views

What are the natural parameters of a matrix-normal distribution?

Assuming that the matrix-normal distribution is in the exponential family (as is a multivariate normal), what are its natural parameters?
3
votes
0answers
23 views

Why is the canonical parameter linearly related to the input x in GLMs and why does it give the link function?

In Andrew Ng's CS229 notes, one of the three assumptions he makes for constructing GLM models is: The natural parameter $\eta$ and the inputs x are related linearly: $\eta=\theta^Tx$ He goes on to ...
3
votes
1answer
48 views

How to determine sample space, $\sigma$-algebra and probability measure from the exponential family?

The sample space of binomial distribution is the set $\{0,1\}$ and its $\sigma$-algebra is the power set of $\{0,1\}$ while the sample space of normal distribution is $\mathbb R$ and its $\sigma$-...
5
votes
2answers
81 views

Are These Conjectures Regarding Sufficient Statistics True?

I have these conjectures that I cannot quite prove (unless I impose another regularity condition of parameter-independent support for distribution, in which case, the conjectures are trivially true ---...
0
votes
1answer
43 views

Is This A Counter-Example To The Theorem by Barndorff-Nielsen-Pedersen (1968)?

In the textbook "Theory of Point Estimation" 2nd Ed. by Lehmann and Casella, Theorem 6.18 states: Suppose $X_1, ..., X_n$ are real-valued IID according to a distribution with density $f_\...
0
votes
0answers
22 views

Understanding natural parameterization of exponential family

I'm going through section 3.4 on exponential families in Statistical Inference by Casella and Berger. They first cite the following general form of an exponential family: $$f(x|\mathbf{\theta})=h(x)c(\...
0
votes
1answer
37 views

Exponential family admissibility of base measure, sufficient statistic and log partition function

Let $$ f(y | \eta) = h(y) \exp\left( \eta^\top T(y) + A(\eta) \right)$$ be the exponential family with base density/pmf $h$, sufficient statistic $T$, log partition function $A$ and natural parameter $...
2
votes
1answer
60 views

What's the intuition behind the canonical link function in GLM?

I have already read the answer from What is the difference between a “link function” and a “canonical link function” for GLM but I think my question is different from this one. I am watching the MIT ...
0
votes
1answer
24 views

Sufficient statistics for a non exponential

I think this is not an exponential family but does it mean that we can't find a sufficient statistic for $\theta$ if $X_1, X_2,..., X_n$ are a random sample from this density? $$ f_{\theta} (x) = \...
2
votes
1answer
30 views

exponential sufficient statistics [duplicate]

A family of pdfs is called an exponential family if $$f(x|\theta) = h(x)c(\theta) \exp \left(\sum_{i=1}^{k} w_{i}(\theta) t_{i}(x) \right)$$ and the statistic $T$ is sufficient iff $f(x;\theta) = h(x)...
0
votes
0answers
27 views

Technical term for natural statistics comined with log-partition

One way to find the posterior of an exponential family distribution with a conjugate prior is to use the natural reparametrization of the likelihood and prior and combining the sufficient statistics ...
4
votes
1answer
115 views

Likelihood function as number of observations increases

If we have $n$ iid observations from some $X \sim p(\cdot|\theta)$, what happens to the likelihood function $p(x_1,\dots,x_n|\theta)$ as $n\rightarrow \infty$? I plotted the product of several $\...
1
vote
1answer
39 views

Showing that $E[X^2] = E[Y^2]$ for two RV following Frechet distributions with different location parameters only [closed]

If random variable X follows a Fréchet distribution (https://en.wikipedia.org/wiki/Fr%C3%A9chet_distribution) with shape parameter $\alpha$, scale parameter $s$, and location parameter $m$, that is $X ...
0
votes
0answers
33 views

Laplace distribution as an Exponential Distribution and Minimizitaion of KL Divergence

In the context of Expectation Propagation [Minka's thesis-2001], I would like to approximate an unknown distribution with a Laplace distribution. This can be solved by minimizing KL-Divergence. In ...
0
votes
0answers
18 views

Standard Error of ERGM Coefficients

I am trying to calculate the standard error of ERGM coefficients, which is estimated by MCMC sample. For an ERGM $P(y;\eta) = \exp[\eta^\top g(y) - \psi(\eta)]$, denote $\eta$ as the true parameter, $\...
3
votes
1answer
129 views

Asymptotic normality of MLE

We know under regularity conditions the MLE is asymptotically normal. Usually, it is said that in practice it's hard to check these assumptions. However, I wondered whether we can claim that these ...
0
votes
0answers
49 views

How to prove that von mises distribution belongs to exponential family?

Can anyone help me prove this, I'm not able to simplify the distribution to find the sufficient statistics, log normalizer, etc.
3
votes
1answer
95 views

Finding UMVUE of difference of exponentals

Let $X_1, \ldots, X_n$ be a sample from an exponential distribution with p.d.f. $f(x; \theta) = \theta e^{-\theta x}$ for $x > 0$ where $\theta > 0$ is an unknown parameter. I would like to find ...
0
votes
0answers
35 views

How can write the probability density function of generalized exponential distribution as exponential family?

I want to use GAM method and generalized exponential distribution for response variable. I know GAM method is a generalized GLM method and the distribution of response variable must be in exponential ...
5
votes
1answer
176 views

How do these results show that $T(\mathbf{X})$ is an unbiased estimator of $E_\varphi[T(\mathbf{X})]$ that achieves the Cramer-Rao lower bound?

Let's say that $X_1, \dots, X_n$ has the joint distribution $f_\varphi(\mathbf{x})$ that belongs to the one-parameter exponential family $$f_\varphi(\mathbf{x}) = \exp{\left\{ c(\varphi) T(\mathbf{x}) ...
0
votes
1answer
251 views

The exponential distribution belongs to the exponential family [closed]

I'm new here. I'm trying to proof that the exponential distribution belongs to the exponential family, but I don't know how to do that. Can you help me? Thanks a lot.
0
votes
1answer
81 views

Showing that $f_\varphi(x)$ is a member of the one-parameter exponential family and $\sum_{i = 1}^n - \log(X_i)$ is sufficient for $\varphi$

Let $X_1, \dots, X_n$ denote a random sample from the PDF $$f_{\varphi}(x)= \begin{cases} \varphi x^{\varphi - 1} &\text{if}\, 0 < x < 1, \varphi > 0\\ 0 &\text{otherwise} \end{...
0
votes
0answers
62 views

Why are/aren't these functions members of the exponential family? [duplicate]

I am currently trying to learn about the exponential family of distributions. I am trying to understand this question and this answer from Xi'an. I have the same function: $$f(x; \sigma, \tau)= \begin{...
-3
votes
1answer
182 views

Is Poisson–Lindley an exponential family? If not, why? [closed]

$$\begin{aligned}f_Y(y_i)&=\frac{{\theta_i}^2\left(y_i+\theta_i+2\right)}{\left(1+\theta_i\right)^{y_i+3}}\\ &=\exp\ \log\left[\frac{{\theta_i}^2\left(y_i+\theta_i+2\right)}{\left(1+\theta_i\...
0
votes
0answers
51 views

How is this implied by the properties of the exponential, gamma, and $\chi^2$ distributions?

Let's say we have the random variables $X_1, \dots, X_p$. Furthermore, say that these random variables are a random sample from a PDF of the form $$f_\tau (x) = \begin{cases} \tau x^{\tau-1}, & 0 ...
2
votes
1answer
72 views

What is the MLE of the Continuous Bernoulli distribution?

The continuous Bernoulli is a distribution I recently discovered. What the maximum likelihood estimate of the distribution's parameter? I'm struggling with the normalizing constant.
0
votes
0answers
38 views

Name for $\eta(\theta) \cdot T(x)$ in exponential family distributions

This is a terminology question. Distributions in the exponential family take the form $$ f(x \mid \theta) = h(x)g(\theta) \exp(\eta(\theta) \cdot T(x)) \text{.} $$ ($\eta$ is the natural parameter, ...
0
votes
0answers
31 views

How to prove that the t distribution doesn't belong to the exponential family?

Or in other words, is there anyway prove that the t distribution doesn't belong to the exponential family without going through all that calculation? Since the density has the gamma function in it ...
0
votes
0answers
26 views

Sufficient statistic for a given distribution from exponential form

Given a particular form, i can verify whether it is sufficient statistic or not using $\frac{p_\theta(x_1,x_2...x_n)}{p_\theta(T(x_1,x_2...x_n))}$ is independendent of $\theta$ then i can say $T(\bar ...
1
vote
1answer
28 views

Log normalizer for Multivariate Gaussian (Exponential Family Representation)

I am searching for the log normalizer based on the natural parameters for the multivariate gaussian in the exponential family representation. For the univariate gaussian, it is given by $$ a(\eta) = \...
8
votes
1answer
297 views

Does a sufficient statistic imply the existence of a conjugate prior?

In the comments on this answer, user Scortchi asks: So iff there's a sufficient statistic of constant dimension, there's a conjugate prior? As far as I know this didn't get a complete answer, so I'm ...
1
vote
1answer
82 views

moment generating function of gamma distribution through log-partition function

How to drive the moment generating function of Gamma distribution using log-partition function? Suppose $X\sim\Gamma(\alpha,\beta)$, gamma distribution with parameter $(\alpha, \beta)$. Then $X$ has ...
0
votes
1answer
49 views

Finding UMVUE of function of poisson parameter

I am to estimate $\exp(-\lambda)\lambda^2/2$ from the distribution $Exp(\lambda) \sim \frac{e^{-\lambda}\lambda^x}{x!}$ I used the indicator function $W=\mathbb I_{2}(X_1)$ as an initial unbiased ...
1
vote
1answer
59 views

Gamma family as conjugate prior of Inverse Gaussian with known $\mu$

I want to show that, when $\mu=\mu_0$, then gamma family $\Gamma(a,b)$ is a conjugate prior to inverse Gaussian with density $f(x,\mu,\lambda)=\sqrt{\frac{\lambda}{2\pi x^2}}exp[-\frac{\lambda(x-\mu)^...
4
votes
1answer
59 views

Practical method to do MLE for natural parameters in exponential family

I encountered the following question in my research and I hope this is the correct place to post it. I'm following the notation in this lecture note by Michael I. Jordan. Assume random vector $X$ ...
4
votes
0answers
44 views

Why is the EM algorithm well suited for exponential families?

I've been brushing up on the EM algorithm, and while I feel like I understand the basics, I keep seeing the claim made (e.g. here, here, among several others) that EM works particularly well for ...
0
votes
0answers
31 views

Multivariate canonical exponential family

Consider the canonical d-dimensional exponential family with densities $$p(x)=exp\left(\langle\theta,T(x)\rangle-A(\theta)\right)h(x),\theta\in\Omega$$ with $\Omega\subset\Omega_0=\{\theta:A(\theta)&...
0
votes
0answers
12 views

What are the hyperparameters and base measure in the conjugate prior for the exponential family?

Setup Suppose we have an exponential family model $\{P_{\theta} : \theta \in \Theta\}$. Let the density function of a random variable $X$ and the prior on $\theta$ have following forms: $$ \begin{...
0
votes
1answer
109 views

Jointly complete and sufficient statistics for multivariate normal distribution

Consider the random sample X from the multivariate normal distribution where xi are i.i.d as N(µ,Σ). *Show that the sample mean x̄ and Sample covariance matrix S are jointly complete and sufficient ...
0
votes
1answer
215 views

Canonical form and exponential family

Suppose you have a random variable X, who's distribution depends on $\theta$. If X is a part of the exponential family of distributions, X can be written in a certain form, namely: $$f_\theta(x)=h(x)*...
0
votes
1answer
32 views

How to prove part of exponential family [closed]

We saw that in order to prove a distribution is a part of a exponential family we should be able to write it in this form: f(y;θ)=exp(a(y)b(θ)+c(θ)+d(y) And to prove that: All statistics T are ...
0
votes
1answer
62 views

Does density belong to exponential family?

$$f(x;\theta) = 2x\theta\exp({-x^2})\left( \frac{\exp({-x^2})}{1-\exp({-x^2})}\right)^{\theta\ - 1}\mathbb I_{(\mathbb R_{++})}(x) $$ with $\theta \in \mathbb R_{++} $ does $f(x;\theta)$ belong to ...
2
votes
0answers
27 views

What family of full support probability distributions satisfy that the density of any point in the domain vanishes as the variance goes to infinity?

Let $f(x,\sigma^2)$ be a representative element of a family of PDF's with full support over the reals that is indexed by their variance $\sigma^2$. Under what general conditions of the family of ...
3
votes
1answer
50 views

determine the minimal exponential family form

Suppose we have a random variable $X$ having p.d.f of the form $$f(x|\theta)=\exp\{c(\theta)'T(x)-B(\theta)\}h(x),$$ then we say $X$ is from exponential family. Further, we say that an exponential ...
1
vote
0answers
22 views

Exponential Family distribution with non-open natural parameter space? [duplicate]

Is there any example of canonical exponential family distribution with a natural parameter space, that is not open? An $k$ dimensional canonical exponential family means having a p.d.f (w.r.t base ...
4
votes
1answer
76 views

$\mathbb{E}(X^2+Y^2)=-2\rho$? $(X,Y)$ is from standard bivariate normal distribution and $Cov(X,Y)=\rho$

How to use log partition function to derive $\mathbb{E}(X^2+Y^2)$, where $(X,Y)$ is from standard bivariate normal distribution? By standard bivariate normal I mean $\mu_x=\mu_y=0$ and $\sigma^2_X=\...
3
votes
1answer
97 views

Example of curved exponential family with $T$ being a complete statistic?

Is there any example of curved exponential family with $T$ being a complete statistic? Here $T$ is the sufficient statistic.
1
vote
0answers
75 views

UMP test for exponential family when sufficient statistics $T$ is a vector

Assume we have a random sample $X_1,\dots,X_n$ from a distribution of the form $f(x_i;\theta) = h(x)g(\theta)\exp(\eta(\theta) T(x))$ and we wish to test $H_0: \theta \leq \theta_0, H_1: \theta > \...

1
2 3 4 5
8