Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
2 votes
1 answer
112 views

What is the name of the functions in exponential dispersion family?

If an exponential family is given by: $g(y|\theta) = exp\{\theta^TT(y)-A(\theta)\}h(y)$ then the functions $h(y)$, $A(\theta)$ and $T(y)$ are defined by names: $T(y)$ is a sufficient statistic $A(\...
user avatar
  • 97
1 vote
0 answers
48 views

Simplifying the Kullback-Leibler divergence for a sum of distributions

I want to find an approximation of a mixture of probability distributions that minimises the Kullback-Leibler divergence (KLD). I need to verify my result, as it seems suspect. We have a joint ...
user avatar
  • 161
2 votes
1 answer
36 views

Conditional exponential family implies joint exponential family

Suppose $p(X_j | X_1,\ldots,X_{j-1})$ comes from some known exponential family for every $j=1,\ldots,k$. Does it follow that the joint distribution comes from some (possibly different) exponential ...
user avatar
  • 21
0 votes
0 answers
16 views

Visualizing relationships in log-link/exponential distribution models by placing the linear predictor on the Y axis?

I'm visualizing results from a negative binomial regression. I don't want to the graph of Y vs X to look exponential, I want it to look linear. In SPSS, the value provided for the linear predictor is ...
user avatar
2 votes
1 answer
150 views

Distribution of sum of $n$ random variables with mixture of two exponential distributions

Suppose that the random variable $Y$ follows a mixture of two exponential distributions, that is \begin{equation} f_Y(y) = \sum_{i=1}^{2}\pi_i f(y| \lambda_i) \end{equation} where $\pi$ stands for ...
user avatar
0 votes
1 answer
58 views

What distributions are in the exponential family?

Are there any exponential family distributions other than wishart distribution, multivariate normal distribution, Dirichlet distribution, multinomial(or categorical) distribution, Conway-maxwel ...
user avatar
  • 11
0 votes
0 answers
28 views

Antiderivative of log-partition function of exponential family

Let $Y$ be a random variable with distribution belonging to a minimal regular exponential family. Let $\eta $ denote the scalar-valued canonical parameter of the exponential family and let $A$ be the ...
user avatar
  • 212
1 vote
1 answer
37 views

For the family, $f_\theta(x)=\frac{e^{-(x-\theta)}}{1+e^{-(x-\theta)})^2}$, compute the fisher information, is it an exponential family?

For the family, $f_\theta(x)=\frac{e^{-(x-\theta)}}{1+e^{-(x-\theta)})^2}$, compute the fisher information, is it an exponential family, $x\in \mathbb{R},\theta \in \mathbb{R}$? I computed the fisher ...
user avatar
1 vote
0 answers
34 views

Does this distribution belong to the exponential family? [duplicate]

I was looking at a problem in the book of "Statistical Inference" second edition by George Casella and Roger L. Berger from chapter 6 that deals with sufficient statistics, minimal ...
user avatar
1 vote
2 answers
82 views

Exponential family and conjugate priors

Is a distribution that belongs to the exponential family necessarily conjugate prior?
user avatar
  • 113
0 votes
0 answers
26 views

How to find $\gamma_i$ and $c_i$ for an UMPU Test for exponential family, hypothesis over an interval

From my textbook: For a one-parametric exponential family $$p_\vartheta(x)=c(\vartheta)h(x)\exp(Q(\vartheta)T(x)),$$ where $Q$ is increasing, an UMPU test of the hypothesis $$H: \vartheta \in [\...
user avatar
0 votes
0 answers
38 views

Prove sum of T(Xi) also belong to exponential family

In mathematical statistics, suppose we have an independent and identically distributed sample X = ($X_1$, $X_2$, …, $X_n$) from a distribution that belongs to the exponential family. Say we can write ...
user avatar
  • 1
0 votes
0 answers
14 views

Inhowfar does Variational Inference work better with members of the exponential family?

I am reading Variational Inference: A Review for Statisticians. Working in [the exponential] family simplifies variational inference: it is easier to derive the corresponding CAVI algorithm, and it ...
user avatar
3 votes
1 answer
267 views

What is "natural" about the natural parameterization of an exponential family and the natural parameter space?

In Section 3.4 of Casella and Berger's Statistical Inference, an exponential family is defined to be a set of pdfs or pmfs such that for each member $f(x | \boldsymbol{\theta})$ of the set, $$ f(x | \...
user avatar
  • 251
0 votes
0 answers
15 views

Gibbs distribution and MCMC

In the paper of An Introduction to HMC for Sampling, I once saw the following statement. My question is that does generic MCMC only apply to Gibbs distribution? I also noticed that many MCMC related ...
user avatar
  • 4,672
2 votes
0 answers
27 views

Origins of Dispersion Models in Statistics [closed]

Recently, I have been reading about Dispersion Models in Statistics. For example, here is an example of the general form of a Dispersion Model: Additive exponential dispersion model In the univariate ...
user avatar
  • 5,688
34 votes
4 answers
3k views

Why is the exponential family so important in statistics?

Why is the exponential family so important in statistics? I was recently reading about the exponential family within statistics. As far as I understand, the exponential family refers to any ...
user avatar
  • 5,688
0 votes
0 answers
31 views

Definition of exponential family and entropy

I'm reading Graphical Models, Exponential Families, and Variational Inference [pdf] by Wainwright and Jordan. They define (p. 39) an exponential family by its derivative $$ p_\theta(x) = \exp \{ \...
user avatar
1 vote
1 answer
218 views

What is the dispersion parameter of binomial distribution?

Binomial distribution is a member of exponential dispersion models, but I can not find the dispersion parameter of it. Could anyone help me find it out? IMO the Binomial distribution only has an ...
user avatar
2 votes
1 answer
172 views

Is there a special case of the EM algorithm for exponential family distributions?

According to Wikipedia, the formal definition of the EM algorithm is The EM algorithm seeks to find the MLE of the marginal likelihood by iteratively applying these two steps: Expectation step (E ...
user avatar
  • 2,916
0 votes
1 answer
82 views

Fit exponential decay upwards model - start values give 'convergence failure'

I have some data that when plotted looks similar to this: Then eyeballing the charts on this page, it looks like I might have an exponential decay (Increasing form) relationship? I searched for how ...
user avatar
  • 1,228
0 votes
0 answers
38 views

Entropy of Poisson random variable via exponential family identity

Exercise 8.1 of Probabilistic Graphical Models asks the reader to use the identity $$H_{P_{\theta}}(X)= \ln Z(\theta) - \langle E_{P_{\theta}}[\tau(X)], t(\theta)\rangle$$ to compute the entropy $H$ ...
user avatar
0 votes
0 answers
39 views

Show that this probability distribution is an exponential family

We define a probability distribution on the non-negative integers 0,1,2,...,as having point probabilities $$P_\eta(k)=G(k,\rho)\left(\frac{1}{\rho+\eta}\right)^{\rho}\left(\frac{\eta}{\rho+ \eta}\...
user avatar
  • 253
0 votes
1 answer
25 views

Sufficient Statistic for Absolutely Continuous Distribution [duplicate]

The following is a homework problem. Please tell me if my solution is correct and if not please point out my mistakes. Let $x_{1}, x_{2},...,x_{M}$ be i.i.d. samples from the absolute continuous ...
user avatar
3 votes
1 answer
37 views

Is a parameter-independent support sufficient to belong to an exponential family?

From Wikipedia (emphasis mine): As a counterexample if these conditions are relaxed, the family of uniform distributions (either discrete or continuous, with either or both bounds unknown) has a ...
user avatar
  • 2,916
0 votes
0 answers
20 views

What are the natural parameters of a matrix-normal distribution?

Assuming that the matrix-normal distribution is in the exponential family (as is a multivariate normal), what are its natural parameters?
user avatar
  • 163
3 votes
0 answers
39 views

Why is the canonical parameter linearly related to the input x in GLMs and why does it give the link function?

In Andrew Ng's CS229 notes, one of the three assumptions he makes for constructing GLM models is: The natural parameter $\eta$ and the inputs x are related linearly: $\eta=\theta^Tx$ He goes on to ...
user avatar
  • 131
3 votes
1 answer
78 views

How to determine sample space, $\sigma$-algebra and probability measure from the exponential family?

The sample space of binomial distribution is the set $\{0,1\}$ and its $\sigma$-algebra is the power set of $\{0,1\}$ while the sample space of normal distribution is $\mathbb R$ and its $\sigma$-...
user avatar
5 votes
2 answers
90 views

Are These Conjectures Regarding Sufficient Statistics True?

I have these conjectures that I cannot quite prove (unless I impose another regularity condition of parameter-independent support for distribution, in which case, the conjectures are trivially true ---...
user avatar
0 votes
1 answer
49 views

Is This A Counter-Example To The Theorem by Barndorff-Nielsen-Pedersen (1968)?

In the textbook "Theory of Point Estimation" 2nd Ed. by Lehmann and Casella, Theorem 6.18 states: Suppose $X_1, ..., X_n$ are real-valued IID according to a distribution with density $f_\...
user avatar
1 vote
0 answers
54 views

Understanding natural parameterization of exponential family

I'm going through section 3.4 on exponential families in Statistical Inference by Casella and Berger. They first cite the following general form of an exponential family: $$f(x|\mathbf{\theta})=h(x)c(\...
user avatar
0 votes
1 answer
71 views

Exponential family admissibility of base measure, sufficient statistic and log partition function

Let $$ f(y | \eta) = h(y) \exp\left( \eta^\top T(y) + A(\eta) \right)$$ be the exponential family with base density/pmf $h$, sufficient statistic $T$, log partition function $A$ and natural parameter $...
user avatar
  • 212
3 votes
1 answer
163 views

What's the intuition behind the canonical link function in GLM?

I have already read the answer from What is the difference between a “link function” and a “canonical link function” for GLM but I think my question is different from this one. I am watching the MIT ...
user avatar
  • 599
0 votes
1 answer
41 views

Sufficient statistics for a non exponential

I think this is not an exponential family but does it mean that we can't find a sufficient statistic for $\theta$ if $X_1, X_2,..., X_n$ are a random sample from this density? $$ f_{\theta} (x) = \...
user avatar
1 vote
1 answer
44 views

exponential sufficient statistics [duplicate]

A family of pdfs is called an exponential family if $$f(x|\theta) = h(x)c(\theta) \exp \left(\sum_{i=1}^{k} w_{i}(\theta) t_{i}(x) \right)$$ and the statistic $T$ is sufficient iff $f(x;\theta) = h(x)...
user avatar
0 votes
0 answers
28 views

Technical term for natural statistics comined with log-partition

One way to find the posterior of an exponential family distribution with a conjugate prior is to use the natural reparametrization of the likelihood and prior and combining the sufficient statistics ...
user avatar
  • 11
5 votes
1 answer
164 views

Likelihood function as number of observations increases

If we have $n$ iid observations from some $X \sim p(\cdot|\theta)$, what happens to the likelihood function $p(x_1,\dots,x_n|\theta)$ as $n\rightarrow \infty$? I plotted the product of several $\...
user avatar
  • 349
1 vote
1 answer
45 views

Showing that $E[X^2] = E[Y^2]$ for two RV following Frechet distributions with different location parameters only [closed]

If random variable X follows a Fréchet distribution (https://en.wikipedia.org/wiki/Fr%C3%A9chet_distribution) with shape parameter $\alpha$, scale parameter $s$, and location parameter $m$, that is $X ...
user avatar
  • 107
0 votes
0 answers
80 views

Laplace distribution as an Exponential Distribution and Minimizitaion of KL Divergence

In the context of Expectation Propagation [Minka's thesis-2001], I would like to approximate an unknown distribution with a Laplace distribution. This can be solved by minimizing KL-Divergence. In ...
user avatar
0 votes
0 answers
41 views

Standard Error of ERGM Coefficients

I am trying to calculate the standard error of ERGM coefficients, which is estimated by MCMC sample. For an ERGM $P(y;\eta) = \exp[\eta^\top g(y) - \psi(\eta)]$, denote $\eta$ as the true parameter, $\...
user avatar
3 votes
1 answer
286 views

Asymptotic normality of MLE

We know under regularity conditions the MLE is asymptotically normal. Usually, it is said that in practice it's hard to check these assumptions. However, I wondered whether we can claim that these ...
user avatar
  • 140
1 vote
2 answers
135 views

How to prove that von mises distribution belongs to exponential family?

Can anyone help me prove this, I'm not able to simplify the distribution to find the sufficient statistics, log normalizer, etc.
user avatar
  • 11
2 votes
1 answer
211 views

Finding UMVUE of difference of exponentals

Let $X_1, \ldots, X_n$ be a sample from an exponential distribution with p.d.f. $f(x; \theta) = \theta e^{-\theta x}$ for $x > 0$ where $\theta > 0$ is an unknown parameter. I would like to find ...
user avatar
0 votes
0 answers
40 views

How can write the probability density function of generalized exponential distribution as exponential family?

I want to use GAM method and generalized exponential distribution for response variable. I know GAM method is a generalized GLM method and the distribution of response variable must be in exponential ...
user avatar
5 votes
1 answer
207 views

How do these results show that $T(\mathbf{X})$ is an unbiased estimator of $E_\varphi[T(\mathbf{X})]$ that achieves the Cramer-Rao lower bound?

Let's say that $X_1, \dots, X_n$ has the joint distribution $f_\varphi(\mathbf{x})$ that belongs to the one-parameter exponential family $$f_\varphi(\mathbf{x}) = \exp{\left\{ c(\varphi) T(\mathbf{x}) ...
user avatar
  • 1,096
0 votes
1 answer
1k views

The exponential distribution belongs to the exponential family [closed]

I'm new here. I'm trying to proof that the exponential distribution belongs to the exponential family, but I don't know how to do that. Can you help me? Thanks a lot.
user avatar
0 votes
1 answer
154 views

Showing that $f_\varphi(x)$ is a member of the one-parameter exponential family and $\sum_{i = 1}^n - \log(X_i)$ is sufficient for $\varphi$

Let $X_1, \dots, X_n$ denote a random sample from the PDF $$f_{\varphi}(x)= \begin{cases} \varphi x^{\varphi - 1} &\text{if}\, 0 < x < 1, \varphi > 0\\ 0 &\text{otherwise} \end{...
user avatar
  • 1,096
0 votes
0 answers
69 views

Why are/aren't these functions members of the exponential family? [duplicate]

I am currently trying to learn about the exponential family of distributions. I am trying to understand this question and this answer from Xi'an. I have the same function: $$f(x; \sigma, \tau)= \begin{...
user avatar
  • 1,096
-3 votes
1 answer
259 views

Is Poisson–Lindley an exponential family? If not, why? [closed]

$$\begin{aligned}f_Y(y_i)&=\frac{{\theta_i}^2\left(y_i+\theta_i+2\right)}{\left(1+\theta_i\right)^{y_i+3}}\\ &=\exp\ \log\left[\frac{{\theta_i}^2\left(y_i+\theta_i+2\right)}{\left(1+\theta_i\...
user avatar
0 votes
0 answers
52 views

How is this implied by the properties of the exponential, gamma, and $\chi^2$ distributions?

Let's say we have the random variables $X_1, \dots, X_p$. Furthermore, say that these random variables are a random sample from a PDF of the form $$f_\tau (x) = \begin{cases} \tau x^{\tau-1}, & 0 ...
user avatar
  • 1,096

1
2 3 4 5
8