Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
2 votes
1 answer
66 views

Does this distribution belong to exponential family and does its support depend on $p?$

I have this bivariate distribution and I would like to tell if it is in the exponential family: $$f(y_{i},d_{i}|\theta_{1}\,\theta_{2}\,p)=\left(\left(\frac{1}{\theta_{1}}\right)\exp\left(\frac{-y_{i}}...
1 vote
0 answers
18 views

Is the Hydrogen wave (probability density) function from physics define a probability density function in the exponential family?

Is the wave function from physics define a probability density function in the exponential family? Hydrogen atom [edit] The wave functions of an electron in a Hydrogen atom are expressed in terms of ...
0 votes
0 answers
22 views

If $(y\theta-a(\theta)+b(y))/\phi$ is the log partition-function, then what is the base-measure, sufficient statistic, and natural parameter?

The log partition function for an observation is $$\log f= (y\theta-a(\theta)+b(y))/\phi$$ Differentiate with respect to $\theta$ to get $$(y - a'(\theta))/\phi.$$ Taking expectations $$E[Y]-a'(\theta)...
3 votes
1 answer
57 views

Can the Beta-regression be written in the GLM form?

The Beta distribution is: $$p(y)=\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}y^{\alpha-1}(1-y)^{\beta-1} $$ It's part of the exponential family. We can reparametrize this with using mean ...
0 votes
0 answers
9 views

Calculating the exponential growth rate against the standard deviation of the year coefficient

I have time-series abundance data for various locations. I would like to calculate the exponential growth rate for each location against the standard deviation of the year coefficient. My dataframe ...
-1 votes
1 answer
52 views

Why does $E\left(y_{i t}\right)=a^{\prime}\left(\theta_{i t}\right)$? in the context of assuming some GEE marginal density?

In generalized estimating equations we have a glm-response variable. To establish notation, we let $Y_{i}=\left(y_{i 1}, \ldots, y_{i n_{i}}\right)^{\text {T }}$ be the $n_{i} \times 1$ vector of ...
1 vote
0 answers
23 views

Where is the h function of the exponential family gone?

The problem comes from equation (4.86) of text "Pattern Recognition and Machine Learning" written by Christopher M. Bishop $$a_k({\bf x})={\bf\lambda}_k^T{\bf x}+\ln g({\bf\lambda}_k)+\ln p(\...
  • 67
2 votes
1 answer
29 views

GAM with opposite outcomes with different families

I'm building GAMs and I have some doubts regarding the family to use. I'm fitting GAMs because I expect some non-linear relationships between the response variable and some covariates. I've checked a ...
  • 513
0 votes
0 answers
26 views

UMP Test and UMVUE when there are nuisance parameters

Consider $X_1,...,X_n \sim Weibull(\theta, c)$ where $c>0$ is unknown. Several textbook examples consider when $c$ is known, but here, we consider when $c$ is unknown. Suppose now we wanted to find ...
  • 89
0 votes
1 answer
34 views

Is the Generalized Dirichlet Distribution an Exponential Family?

Is the Generalized Dirichlet distribution an exponential family? If so, what is its log-normalizer, sufficient statistics, and carrier measure?
  • 14k
1 vote
1 answer
27 views

Impact of sample size when using exponential random graph modeling

I am working on revising a manuscript centered on identifying the drivers behind certain types of student interaction. One critique that I'm having trouble addressing is that they are worried about my ...
2 votes
1 answer
134 views

What is the name of the functions in exponential dispersion family?

If an exponential family is given by: $g(y|\theta) = exp\{\theta^TT(y)-A(\theta)\}h(y)$ then the functions $h(y)$, $A(\theta)$ and $T(y)$ are defined by names: $T(y)$ is a sufficient statistic $A(\...
  • 107
1 vote
0 answers
96 views

Simplifying the Kullback-Leibler divergence for a sum of distributions

I want to find an approximation of a mixture of probability distributions that minimises the Kullback-Leibler divergence (KLD). I need to verify my result, as it seems suspect. We have a joint ...
  • 161
2 votes
1 answer
46 views

Conditional exponential family implies joint exponential family

Suppose $p(X_j | X_1,\ldots,X_{j-1})$ comes from some known exponential family for every $j=1,\ldots,k$. Does it follow that the joint distribution comes from some (possibly different) exponential ...
  • 21
0 votes
0 answers
17 views

Visualizing relationships in log-link/exponential distribution models by placing the linear predictor on the Y axis?

I'm visualizing results from a negative binomial regression. I don't want to the graph of Y vs X to look exponential, I want it to look linear. In SPSS, the value provided for the linear predictor is ...
2 votes
1 answer
206 views

Distribution of sum of $n$ random variables with mixture of two exponential distributions

Suppose that the random variable $Y$ follows a mixture of two exponential distributions, that is \begin{equation} f_Y(y) = \sum_{i=1}^{2}\pi_i f(y| \lambda_i) \end{equation} where $\pi$ stands for ...
0 votes
1 answer
71 views

What distributions are in the exponential family?

Are there any exponential family distributions other than wishart distribution, multivariate normal distribution, Dirichlet distribution, multinomial(or categorical) distribution, Conway-maxwel ...
  • 31
0 votes
0 answers
51 views

Antiderivative of log-partition function of exponential family

Let $Y$ be a random variable with distribution belonging to a minimal regular exponential family. Let $\eta $ denote the scalar-valued canonical parameter of the exponential family and let $A$ be the ...
  • 211
1 vote
1 answer
42 views

For the family, $f_\theta(x)=\frac{e^{-(x-\theta)}}{1+e^{-(x-\theta)})^2}$, compute the fisher information, is it an exponential family?

For the family, $f_\theta(x)=\frac{e^{-(x-\theta)}}{1+e^{-(x-\theta)})^2}$, compute the fisher information, is it an exponential family, $x\in \mathbb{R},\theta \in \mathbb{R}$? I computed the fisher ...
1 vote
0 answers
37 views

Does this distribution belong to the exponential family? [duplicate]

I was looking at a problem in the book of "Statistical Inference" second edition by George Casella and Roger L. Berger from chapter 6 that deals with sufficient statistics, minimal ...
1 vote
2 answers
139 views

Exponential family and conjugate priors

Is a distribution that belongs to the exponential family necessarily conjugate prior?
  • 113
0 votes
0 answers
35 views

How to find $\gamma_i$ and $c_i$ for an UMPU Test for exponential family, hypothesis over an interval

From my textbook: For a one-parametric exponential family $$p_\vartheta(x)=c(\vartheta)h(x)\exp(Q(\vartheta)T(x)),$$ where $Q$ is increasing, an UMPU test of the hypothesis $$H: \vartheta \in [\...
0 votes
0 answers
62 views

Prove sum of T(Xi) also belong to exponential family

In mathematical statistics, suppose we have an independent and identically distributed sample X = ($X_1$, $X_2$, …, $X_n$) from a distribution that belongs to the exponential family. Say we can write ...
  • 1
0 votes
0 answers
22 views

Inhowfar does Variational Inference work better with members of the exponential family?

I am reading Variational Inference: A Review for Statisticians. Working in [the exponential] family simplifies variational inference: it is easier to derive the corresponding CAVI algorithm, and it ...
3 votes
1 answer
775 views

What is "natural" about the natural parameterization of an exponential family and the natural parameter space?

In Section 3.4 of Casella and Berger's Statistical Inference, an exponential family is defined to be a set of pdfs or pmfs such that for each member $f(x | \boldsymbol{\theta})$ of the set, $$ f(x | \...
  • 251
0 votes
0 answers
21 views

Gibbs distribution and MCMC

In the paper of An Introduction to HMC for Sampling, I once saw the following statement. My question is that does generic MCMC only apply to Gibbs distribution? I also noticed that many MCMC related ...
  • 4,812
2 votes
0 answers
37 views

Origins of Dispersion Models in Statistics [closed]

Recently, I have been reading about Dispersion Models in Statistics. For example, here is an example of the general form of a Dispersion Model: Additive exponential dispersion model In the univariate ...
  • 6,078
36 votes
4 answers
4k views

Why is the exponential family so important in statistics?

Why is the exponential family so important in statistics? I was recently reading about the exponential family within statistics. As far as I understand, the exponential family refers to any ...
  • 6,078
0 votes
0 answers
36 views

Definition of exponential family and entropy

I'm reading Graphical Models, Exponential Families, and Variational Inference [pdf] by Wainwright and Jordan. They define (p. 39) an exponential family by its derivative $$ p_\theta(x) = \exp \{ \...
1 vote
1 answer
369 views

What is the dispersion parameter of binomial distribution?

Binomial distribution is a member of exponential dispersion models, but I can not find the dispersion parameter of it. Could anyone help me find it out? IMO the Binomial distribution only has an ...
3 votes
1 answer
197 views

Is there a special case of the EM algorithm for exponential family distributions?

According to Wikipedia, the formal definition of the EM algorithm is The EM algorithm seeks to find the MLE of the marginal likelihood by iteratively applying these two steps: Expectation step (E ...
  • 3,179
0 votes
1 answer
142 views

Fit exponential decay upwards model - start values give 'convergence failure'

I have some data that when plotted looks similar to this: Then eyeballing the charts on this page, it looks like I might have an exponential decay (Increasing form) relationship? I searched for how ...
  • 1,258
0 votes
0 answers
62 views

Entropy of Poisson random variable via exponential family identity

Exercise 8.1 of Probabilistic Graphical Models asks the reader to use the identity $$H_{P_{\theta}}(X)= \ln Z(\theta) - \langle E_{P_{\theta}}[\tau(X)], t(\theta)\rangle$$ to compute the entropy $H$ ...
0 votes
1 answer
89 views

Show that this probability distribution is an exponential family

We define a probability distribution on the non-negative integers 0,1,2,...,as having point probabilities $$P_\eta(k)=G(k,\rho)\left(\frac{1}{\rho+\eta}\right)^{\rho}\left(\frac{\eta}{\rho+ \eta}\...
  • 273
0 votes
1 answer
36 views

Sufficient Statistic for Absolutely Continuous Distribution [duplicate]

The following is a homework problem. Please tell me if my solution is correct and if not please point out my mistakes. Let $x_{1}, x_{2},...,x_{M}$ be i.i.d. samples from the absolute continuous ...
3 votes
1 answer
43 views

Is a parameter-independent support sufficient to belong to an exponential family?

From Wikipedia (emphasis mine): As a counterexample if these conditions are relaxed, the family of uniform distributions (either discrete or continuous, with either or both bounds unknown) has a ...
  • 3,179
3 votes
0 answers
54 views

Why is the canonical parameter linearly related to the input x in GLMs and why does it give the link function?

In Andrew Ng's CS229 notes, one of the three assumptions he makes for constructing GLM models is: The natural parameter $\eta$ and the inputs x are related linearly: $\eta=\theta^Tx$ He goes on to ...
  • 131
3 votes
1 answer
91 views

How to determine sample space, $\sigma$-algebra and probability measure from the exponential family?

The sample space of binomial distribution is the set $\{0,1\}$ and its $\sigma$-algebra is the power set of $\{0,1\}$ while the sample space of normal distribution is $\mathbb R$ and its $\sigma$-...
  • 113
6 votes
2 answers
95 views

Are These Conjectures Regarding Sufficient Statistics True?

I have these conjectures that I cannot quite prove (unless I impose another regularity condition of parameter-independent support for distribution, in which case, the conjectures are trivially true ---...
1 vote
1 answer
52 views

Is This A Counter-Example To The Theorem by Barndorff-Nielsen-Pedersen (1968)?

In the textbook "Theory of Point Estimation" 2nd Ed. by Lehmann and Casella, Theorem 6.18 states: Suppose $X_1, ..., X_n$ are real-valued IID according to a distribution with density $f_\...
1 vote
0 answers
120 views

Understanding natural parameterization of exponential family

I'm going through section 3.4 on exponential families in Statistical Inference by Casella and Berger. They first cite the following general form of an exponential family: $$f(x|\mathbf{\theta})=h(x)c(\...
0 votes
1 answer
91 views

Exponential family admissibility of base measure, sufficient statistic and log partition function

Let $$ f(y | \eta) = h(y) \exp\left( \eta^\top T(y) + A(\eta) \right)$$ be the exponential family with base density/pmf $h$, sufficient statistic $T$, log partition function $A$ and natural parameter $...
  • 211
3 votes
1 answer
271 views

What's the intuition behind the canonical link function in GLM?

I have already read the answer from What is the difference between a “link function” and a “canonical link function” for GLM but I think my question is different from this one. I am watching the MIT ...
  • 609
0 votes
1 answer
53 views

Sufficient statistics for a non exponential

I think this is not an exponential family but does it mean that we can't find a sufficient statistic for $\theta$ if $X_1, X_2,..., X_n$ are a random sample from this density? $$ f_{\theta} (x) = \...
1 vote
1 answer
53 views

exponential sufficient statistics [duplicate]

A family of pdfs is called an exponential family if $$f(x|\theta) = h(x)c(\theta) \exp \left(\sum_{i=1}^{k} w_{i}(\theta) t_{i}(x) \right)$$ and the statistic $T$ is sufficient iff $f(x;\theta) = h(x)...
5 votes
1 answer
227 views

Likelihood function as number of observations increases

If we have $n$ iid observations from some $X \sim p(\cdot|\theta)$, what happens to the likelihood function $p(x_1,\dots,x_n|\theta)$ as $n\rightarrow \infty$? I plotted the product of several $\...
  • 349
1 vote
1 answer
51 views

Showing that $E[X^2] = E[Y^2]$ for two RV following Frechet distributions with different location parameters only [closed]

If random variable X follows a Fréchet distribution (https://en.wikipedia.org/wiki/Fr%C3%A9chet_distribution) with shape parameter $\alpha$, scale parameter $s$, and location parameter $m$, that is $X ...
  • 107
3 votes
1 answer
409 views

Asymptotic normality of MLE

We know under regularity conditions the MLE is asymptotically normal. Usually, it is said that in practice it's hard to check these assumptions. However, I wondered whether we can claim that these ...
  • 139
1 vote
2 answers
203 views

How to prove that von mises distribution belongs to exponential family?

Can anyone help me prove this, I'm not able to simplify the distribution to find the sufficient statistics, log normalizer, etc.
  • 11
2 votes
1 answer
229 views

Finding UMVUE of difference of exponentals

Let $X_1, \ldots, X_n$ be a sample from an exponential distribution with p.d.f. $f(x; \theta) = \theta e^{-\theta x}$ for $x > 0$ where $\theta > 0$ is an unknown parameter. I would like to find ...

1
2 3 4 5
8