Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
3
votes
1answer
46 views

How to determine sample space, $\sigma$-algebra and probability measure from the exponential family?

The sample space of binomial distribution is the set $\{0,1\}$ and its $\sigma$-algebra is the power set of $\{0,1\}$ while the sample space of normal distribution is $\mathbb R$ and its $\sigma$-...
4
votes
2answers
71 views

Are These Conjectures Regarding Sufficient Statistics True?

I have these conjectures that I cannot quite prove (unless I impose another regularity condition of parameter-independent support for distribution, in which case, the conjectures are trivially true ---...
0
votes
1answer
42 views

Is This A Counter-Example To The Theorem by Barndorff-Nielsen-Pedersen (1968)?

In the textbook "Theory of Point Estimation" 2nd Ed. by Lehmann and Casella, Theorem 6.18 states: Suppose $X_1, ..., X_n$ are real-valued IID according to a distribution with density $f_\...
0
votes
0answers
9 views

How to get value of an exponential with immaginary part? [migrated]

A quaternion is written as: where a,b,c,d are all real numbers. However, a identifies the real part of the quaterion, while the ...
0
votes
0answers
15 views

Understanding natural parameterization of exponential family

I'm going through section 3.4 on exponential families in Statistical Inference by Casella and Berger. They first cite the following general form of an exponential family: $$f(x|\mathbf{\theta})=h(x)c(\...
7
votes
2answers
746 views

Sufficient Statistic for non-exponential family distribution

Question: Let $X_1,X_2,\ldots,X_n$ be an iid sample from $N(\theta , 4 \theta^2 )$. I want to show that this model is not a member of the exponential family and to find a sufficient statistic for $\...
0
votes
1answer
32 views

Exponential family admissibility of base measure, sufficient statistic and log partition function

Let $$ f(y | \eta) = h(y) \exp\left( \eta^\top T(y) + A(\eta) \right)$$ be the exponential family with base density/pmf $h$, sufficient statistic $T$, log partition function $A$ and natural parameter $...
1
vote
1answer
47 views

What's the intuition behind the canonical link function in GLM?

I have already read the answer from What is the difference between a “link function” and a “canonical link function” for GLM but I think my question is different from this one. I am watching the MIT ...
0
votes
1answer
23 views

Sufficient statistics for a non exponential

I think this is not an exponential family but does it mean that we can't find a sufficient statistic for $\theta$ if $X_1, X_2,..., X_n$ are a random sample from this density? $$ f_{\theta} (x) = \...
2
votes
1answer
29 views

exponential sufficient statistics [duplicate]

A family of pdfs is called an exponential family if $$f(x|\theta) = h(x)c(\theta) \exp \left(\sum_{i=1}^{k} w_{i}(\theta) t_{i}(x) \right)$$ and the statistic $T$ is sufficient iff $f(x;\theta) = h(x)...
0
votes
0answers
27 views

Technical term for natural statistics comined with log-partition

One way to find the posterior of an exponential family distribution with a conjugate prior is to use the natural reparametrization of the likelihood and prior and combining the sufficient statistics ...
4
votes
1answer
109 views

Likelihood function as number of observations increases

If we have $n$ iid observations from some $X \sim p(\cdot|\theta)$, what happens to the likelihood function $p(x_1,\dots,x_n|\theta)$ as $n\rightarrow \infty$? I plotted the product of several $\...
1
vote
1answer
28 views

Log normalizer for Multivariate Gaussian (Exponential Family Representation)

I am searching for the log normalizer based on the natural parameters for the multivariate gaussian in the exponential family representation. For the univariate gaussian, it is given by $$ a(\eta) = \...
1
vote
1answer
38 views

Showing that $E[X^2] = E[Y^2]$ for two RV following Frechet distributions with different location parameters only [closed]

If random variable X follows a Fréchet distribution (https://en.wikipedia.org/wiki/Fr%C3%A9chet_distribution) with shape parameter $\alpha$, scale parameter $s$, and location parameter $m$, that is $X ...
7
votes
2answers
2k views

Role of base measure in exponential family

An exponential family distribution $p$ in the canonical form can be written as $p(x|\theta) = h(x)\exp(\theta^\top T(x) - A(\theta))$ where $A(\theta)$ is the log partition function, $T(x)$ is the ...
4
votes
1answer
264 views

Does stable distribution belong to exponential family?

According to Hougaard (1986), positive stable distribution on $\mathbb{R}^+$ belongs to exponential family, how about the case the support of stable distribution being less than zero? The purpose of ...
0
votes
0answers
22 views

Laplace distribution as an Exponential Distribution and Minimizitaion of KL Divergence

In the context of Expectation Propagation [Minka's thesis-2001], I would like to approximate an unknown distribution with a Laplace distribution. This can be solved by minimizing KL-Divergence. In ...
1
vote
1answer
77 views

Definition of $k$-parameter exponential family

I am currently studying the concept of sufficient statistics in mathematical statistics. The following definition is presented: Definition: $k$-parameter exponential family Let $\mathbf{Y} \sim f_\...
2
votes
0answers
37 views

Does any choice of sufficient statistic and natural parameter create valid exponential family pdfs?

So exponential families can be parameterized as $f(x|\theta) = h(x)e^{T(x)n(\theta)-A(n)}$. I'm trying to understand what the conditions are on $h(x)$, $T(x)$, and $n(\theta)$ are for $f(x|\theta)$ ...
9
votes
2answers
2k views

Why is the mixtures of conjugate priors important?

I have questions about the mixture of conjugate priors. I learned and saw the mixture of conjugate priors a couple of times when I am learning bayesian. I am wondering why this theorem is such ...
0
votes
0answers
15 views

Standard Error of ERGM Coefficients

I am trying to calculate the standard error of ERGM coefficients, which is estimated by MCMC sample. For an ERGM $P(y;\eta) = \exp[\eta^\top g(y) - \psi(\eta)]$, denote $\eta$ as the true parameter, $\...
3
votes
1answer
116 views

Asymptotic normality of MLE

We know under regularity conditions the MLE is asymptotically normal. Usually, it is said that in practice it's hard to check these assumptions. However, I wondered whether we can claim that these ...
3
votes
1answer
83 views

Finding UMVUE of difference of exponentals

Let $X_1, \ldots, X_n$ be a sample from an exponential distribution with p.d.f. $f(x; \theta) = \theta e^{-\theta x}$ for $x > 0$ where $\theta > 0$ is an unknown parameter. I would like to find ...
0
votes
0answers
43 views

How to prove that von mises distribution belongs to exponential family?

Can anyone help me prove this, I'm not able to simplify the distribution to find the sufficient statistics, log normalizer, etc.
3
votes
1answer
707 views

Conway–Maxwell–Poisson (CMP) distribution and exponential family

So I have a question here about the CMP distribution: My understanding is that $b(\theta)$ is only a function of $\theta$ but why is $v$ able to be included in that function, would $v$ not be a ...
5
votes
1answer
175 views

How do these results show that $T(\mathbf{X})$ is an unbiased estimator of $E_\varphi[T(\mathbf{X})]$ that achieves the Cramer-Rao lower bound?

Let's say that $X_1, \dots, X_n$ has the joint distribution $f_\varphi(\mathbf{x})$ that belongs to the one-parameter exponential family $$f_\varphi(\mathbf{x}) = \exp{\left\{ c(\varphi) T(\mathbf{x}) ...
19
votes
2answers
4k views

Does log likelihood in GLM have guaranteed convergence to global maxima?

My questions are: Are generalized linear models (GLMs) guaranteed to converge to a global maximum? If so, why? Furthermore, what constraints are there on the link function to insure convexity? My ...
0
votes
0answers
32 views

How can write the probability density function of generalized exponential distribution as exponential family?

I want to use GAM method and generalized exponential distribution for response variable. I know GAM method is a generalized GLM method and the distribution of response variable must be in exponential ...
5
votes
2answers
7k views

Cramér-Rao Lower Bound for Exponential Families

I am having a problem with applying the Cramér-Rao inequality to identify the lower bound for the variance of an unbiased estimator and hoped that you guys could help me. The problem is the following: ...
1
vote
2answers
295 views

Connection between subgaussian/subexponential and exponential family

I am wondering if there is any relationship between subgaussian/subexponential with (one parameter) exponential family. In particular, is there any sub-family density that belongs to both ...
0
votes
1answer
87 views

The exponential distribution belongs to the exponential family [closed]

I'm new here. I'm trying to proof that the exponential distribution belongs to the exponential family, but I don't know how to do that. Can you help me? Thanks a lot.
-3
votes
1answer
168 views

Is Poisson–Lindley an exponential family? If not, why? [closed]

$$\begin{aligned}f_Y(y_i)&=\frac{{\theta_i}^2\left(y_i+\theta_i+2\right)}{\left(1+\theta_i\right)^{y_i+3}}\\ &=\exp\ \log\left[\frac{{\theta_i}^2\left(y_i+\theta_i+2\right)}{\left(1+\theta_i\...
0
votes
1answer
66 views

Showing that $f_\varphi(x)$ is a member of the one-parameter exponential family and $\sum_{i = 1}^n - \log(X_i)$ is sufficient for $\varphi$

Let $X_1, \dots, X_n$ denote a random sample from the PDF $$f_{\varphi}(x)= \begin{cases} \varphi x^{\varphi - 1} &\text{if}\, 0 < x < 1, \varphi > 0\\ 0 &\text{otherwise} \end{...
7
votes
1answer
391 views

Is the negative exponential distribution a member of the exponential family?

Please correct me if I am wrong. The general form of $k$-parameter exponential family is $$f(x;\boldsymbol{\theta}) = a(\boldsymbol{\theta})g(x) \exp\{\sum_{i=1}^{k}b(\boldsymbol{\theta}) R_i(x)\}$$ ...
0
votes
0answers
61 views

Why are/aren't these functions members of the exponential family? [duplicate]

I am currently trying to learn about the exponential family of distributions. I am trying to understand this question and this answer from Xi'an. I have the same function: $$f(x; \sigma, \tau)= \begin{...
3
votes
2answers
4k views

Is Weibull distribution a exponential family?

I'm wondering is Weibull distribution a exponential family?
0
votes
0answers
51 views

How is this implied by the properties of the exponential, gamma, and $\chi^2$ distributions?

Let's say we have the random variables $X_1, \dots, X_p$. Furthermore, say that these random variables are a random sample from a PDF of the form $$f_\tau (x) = \begin{cases} \tau x^{\tau-1}, & 0 ...
2
votes
1answer
56 views

What is the MLE of the Continuous Bernoulli distribution?

The continuous Bernoulli is a distribution I recently discovered. What the maximum likelihood estimate of the distribution's parameter? I'm struggling with the normalizing constant.
0
votes
0answers
37 views

Name for $\eta(\theta) \cdot T(x)$ in exponential family distributions

This is a terminology question. Distributions in the exponential family take the form $$ f(x \mid \theta) = h(x)g(\theta) \exp(\eta(\theta) \cdot T(x)) \text{.} $$ ($\eta$ is the natural parameter, ...
3
votes
2answers
130 views

If Y has an exponential family distribution show that $E(\frac{\partial L}{\partial \theta}) = 0$

I'm working in a self study fashion preparing for a course I'm going to take this semester in generalised linear models. The question is, given that the Y random variable belongs to the exponential ...
0
votes
0answers
29 views

How to prove that the t distribution doesn't belong to the exponential family?

Or in other words, is there anyway prove that the t distribution doesn't belong to the exponential family without going through all that calculation? Since the density has the gamma function in it ...
0
votes
0answers
26 views

Sufficient statistic for a given distribution from exponential form

Given a particular form, i can verify whether it is sufficient statistic or not using $\frac{p_\theta(x_1,x_2...x_n)}{p_\theta(T(x_1,x_2...x_n))}$ is independendent of $\theta$ then i can say $T(\bar ...
1
vote
1answer
55 views

Gamma family as conjugate prior of Inverse Gaussian with known $\mu$

I want to show that, when $\mu=\mu_0$, then gamma family $\Gamma(a,b)$ is a conjugate prior to inverse Gaussian with density $f(x,\mu,\lambda)=\sqrt{\frac{\lambda}{2\pi x^2}}exp[-\frac{\lambda(x-\mu)^...
8
votes
1answer
279 views

Does a sufficient statistic imply the existence of a conjugate prior?

In the comments on this answer, user Scortchi asks: So iff there's a sufficient statistic of constant dimension, there's a conjugate prior? As far as I know this didn't get a complete answer, so I'm ...
0
votes
1answer
46 views

moment generating function of gamma distribution through log-partition function

How to drive the moment generating function of Gamma distribution using log-partition function? Suppose $X\sim\Gamma(\alpha,\beta)$, gamma distribution with parameter $(\alpha, \beta)$. Then $X$ has ...
0
votes
1answer
44 views

Finding UMVUE of function of poisson parameter

I am to estimate $\exp(-\lambda)\lambda^2/2$ from the distribution $Exp(\lambda) \sim \frac{e^{-\lambda}\lambda^x}{x!}$ I used the indicator function $W=\mathbb I_{2}(X_1)$ as an initial unbiased ...
0
votes
1answer
77 views

Jointly complete and sufficient statistics for multivariate normal distribution

Consider the random sample X from the multivariate normal distribution where xi are i.i.d as N(µ,Σ). *Show that the sample mean x̄ and Sample covariance matrix S are jointly complete and sufficient ...
4
votes
0answers
131 views

Show that there is no efficient estimator for the variance of a normal distribution using properties of the exponential family

I want to prove the statement in the title using the following statement from Wikipedia: it was proved that efficient estimation is possible only in an exponential family, and only for the natural ...
4
votes
1answer
56 views

Practical method to do MLE for natural parameters in exponential family

I encountered the following question in my research and I hope this is the correct place to post it. I'm following the notation in this lecture note by Michael I. Jordan. Assume random vector $X$ ...
4
votes
0answers
41 views

Why is the EM algorithm well suited for exponential families?

I've been brushing up on the EM algorithm, and while I feel like I understand the basics, I keep seeing the claim made (e.g. here, here, among several others) that EM works particularly well for ...

1
2 3 4 5
7