Questions tagged [exponential-family]
A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.
386
questions
2
votes
1
answer
54
views
For some $\tau=\tau(\theta)$, there exists an unbiased estimator (UMVUE), then the distribution belongs to an exponential family
I read the textbook in Cramer-Rao lower bound (CRLB). Here is a theroem
For some $\tau=\tau(\theta)$, there exists an unbiased estimator $\hat{\tau}$ of $\tau$ such that $Var(\hat{\tau})$ attains the ...
- 611
0
votes
0
answers
24
views
Why do we want to constrain E[ln(x)] in some maximum entropy models?
If we look at the table of distributions in the exponential family, we will see some sufficient statistics have $\log(x)$, which means we have put constraints on $\mathbb{E}[\log(X)]$ when formulating ...
- 1,706
1
vote
1
answer
41
views
Find out which exponential distribution the data belongs to
I'm doing a GLM homework, and I'm stuck with the following problem:
Suppose that data ($Y_i$; $\mathbf{X}_i$); $i = 1, . . . , n$ are observed, where $\mathbf{X}_i$ is a p-dimensional vector for ...
- 63
0
votes
1
answer
26
views
How to prove the Poisson link function is a canonical link function?
So I'm a 3rd year undergraduate doing my thesisin football score models right now. In my thesis I want to include a proof of what the link function for the Poisson distribution is and why it relates ...
1
vote
0
answers
34
views
Expectation of Fisher score not equal to 0 when parametrize Categorical distribution differently
Expectation of Fisher score should equal to zero. The prove can be found in many palces, such as wikipedia.
But I tried a categorical distribution that is not parameterizatized minimally, the expected ...
- 763
1
vote
0
answers
32
views
Fenchel conjugate of the cumulant function (Exponential Family)
We have a minimal exponential family $f(x) = h(x) \exp(\langle\theta, t(x)\rangle - A(\theta)).$ The canonical parameter space is $\Theta = \{ \theta \in \mathbb{R}^d \colon \int h(x) \exp(\langle\...
- 514
0
votes
0
answers
37
views
Mean Parametrization of Bernoulli Distribution(Exponential Family)
In class, we defined an exponential family to be of distributions with density of form
$$h(x) \exp(\langle\theta, T(x)\rangle-A(\theta))$$
$\langle~,~\rangle$ is the inner product and $\theta \in \...
- 514
0
votes
1
answer
60
views
How to check if an exponential family is regular?
A strictly $k$-parameter exponential family $$f=\exp\left(\sum \eta_i(\theta)T_i(x)-B(\theta)\right)h(x)$$ is regular if the natural parameter space $\eta(\theta)$ contains a $k$-dimensional open set.
...
- 43
2
votes
1
answer
115
views
Writing exponential family in canonical form
I have the following pdf with support $x>0$:
$$f_{\mu}(x)=\frac{1}{\sqrt{2\pi x^3}}\textrm{exp}\left(-\frac{(x-\mu)^2}{2\mu^2x}\right)$$
This belongs to the exponential family, and I write this in ...
- 107
0
votes
0
answers
59
views
Smallest threshold for hypothesis test with asymptotic level alpha
Consider a distribution with parameter $\lambda$ that has density
$$f_\lambda(x)=\frac{x^4}{24\lambda^5}e^{\frac{-x}{\lambda}},x>0$$
Let $X_1,...,X_n$ be $n$ independent random variables drawn from ...
- 107
1
vote
0
answers
48
views
Conditions on the log partition of exponential family generating cumulants
Is any minimality condition of the parameterization required such that the derivative of the log partition function is equal to the mean?
The pdf of an exponential family distribution is given by $$p(...
- 21
0
votes
0
answers
20
views
Why not characterise quasi-Poissons by an actual density?
In the discussions I have seen about quasi-Poisson regressin methods, I glean that the method is to merely assume (in the standard notation) $E(Y_i) = \phi \text{Var}(Y_i)$ given the explanatory ...
- 101
1
vote
3
answers
95
views
Why is $\theta\mapsto p_{\theta}$ one-to-one $\iff$ $(n+1)$ functions $\{F_1,\ldots,F_n,1\}$ are linearly independent?
If an n-dimensional model $S=\{p_{\theta}| \theta\in\Theta\}$ can be expressed in terms of the functions $\{C,F_1,\ldots,F_n\}$ on a sample space $X$ and a function $\psi\in \Theta$(parameter space) ...
- 133
0
votes
0
answers
30
views
Is it possible to derive the probit model by writing it in exponential family form?
We always use latent variable approach to derive probit model,
is there any way to derive probit model from the exponential family form (by using link function)?
Also, does logit-normal distribution ...
- 137
0
votes
0
answers
33
views
What probability distribution matches $1/(e^x - 1)?$
I've been looking at the proof of Planck's Law.
This is the first law that triggered the science of quantum theory by introducing the idea of quanta (which come in the form of packets of energy of ...
- 1
2
votes
1
answer
77
views
Does this distribution belong to exponential family and does its support depend on $p?$
I have this bivariate distribution and I would like to tell if it is in the exponential family:
$$f(y_{i},d_{i}|\theta_{1}\,\theta_{2}\,p)=\left(\left(\frac{1}{\theta_{1}}\right)\exp\left(\frac{-y_{i}}...
1
vote
0
answers
19
views
Is the Hydrogen wave (probability density) function from physics define a probability density function in the exponential family?
Is the wave function from physics define a probability density function in the exponential family?
Hydrogen atom [edit]
The wave functions of an electron in a Hydrogen atom are expressed in terms of ...
user318514
0
votes
0
answers
27
views
If $(y\theta-a(\theta)+b(y))/\phi$ is the log partition-function, then what is the base-measure, sufficient statistic, and natural parameter?
The log partition function for an observation is
$$\log f= (y\theta-a(\theta)+b(y))/\phi$$
Differentiate with respect to $\theta$ to get
$$(y - a'(\theta))/\phi.$$
Taking expectations
$$E[Y]-a'(\theta)...
user318514
3
votes
1
answer
156
views
Can the Beta-regression be written in the GLM form?
The Beta distribution is:
$$p(y)=\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}y^{\alpha-1}(1-y)^{\beta-1}
$$
It's part of the exponential family.
We can reparametrize this with using mean ...
- 2,869
0
votes
0
answers
15
views
Calculating the exponential growth rate against the standard deviation of the year coefficient
I have time-series abundance data for various locations. I would like to calculate the exponential growth rate for each location against the standard deviation of the year coefficient. My dataframe ...
- 33
-1
votes
1
answer
56
views
Why does $E\left(y_{i t}\right)=a^{\prime}\left(\theta_{i t}\right)$? in the context of assuming some GEE marginal density?
In generalized estimating equations we have a glm-response variable.
To establish notation, we let $Y_{i}=\left(y_{i 1}, \ldots, y_{i n_{i}}\right)^{\text {T }}$ be the $n_{i} \times 1$ vector of ...
user318514
1
vote
0
answers
28
views
Where is the h function of the exponential family gone?
The problem comes from equation (4.86) of text "Pattern Recognition and Machine Learning" written by Christopher M. Bishop $$a_k({\bf x})={\bf\lambda}_k^T{\bf x}+\ln g({\bf\lambda}_k)+\ln p(\...
- 183
2
votes
1
answer
68
views
GAM with opposite outcomes with different families
I'm building GAMs and I have some doubts regarding the family to use. I'm fitting GAMs because I expect some non-linear relationships between the response variable and some covariates. I've checked a ...
- 553
0
votes
0
answers
45
views
UMP Test and UMVUE when there are nuisance parameters
Consider $X_1,...,X_n \sim Weibull(\theta, c)$ where $c>0$ is unknown. Several textbook examples consider when $c$ is known, but here, we consider when $c$ is unknown. Suppose now we wanted to find ...
- 89
0
votes
1
answer
145
views
Is the Generalized Dirichlet Distribution an Exponential Family?
Is the Generalized Dirichlet distribution an exponential family? If so, what is its log-normalizer, sufficient statistics, and carrier measure?
- 14.4k
1
vote
1
answer
37
views
Impact of sample size when using exponential random graph modeling
I am working on revising a manuscript centered on identifying the drivers behind certain types of student interaction. One critique that I'm having trouble addressing is that they are worried about my ...
- 11
2
votes
1
answer
187
views
What is the name of the functions in exponential dispersion family?
If an exponential family is given by:
$g(y|\theta) = exp\{\theta^TT(y)-A(\theta)\}h(y)$
then the functions $h(y)$, $A(\theta)$ and $T(y)$ are defined by names:
$T(y)$ is a sufficient statistic
$A(\...
- 107
1
vote
0
answers
231
views
Simplifying the Kullback-Leibler divergence for a sum of distributions
I want to find an approximation of a mixture of probability distributions that minimises the Kullback-Leibler divergence (KLD).
I need to verify my result, as it seems suspect.
We have a joint ...
- 81
2
votes
1
answer
81
views
Conditional exponential family implies joint exponential family
Suppose $p(X_j | X_1,\ldots,X_{j-1})$ comes from some known exponential family for every $j=1,\ldots,k$. Does it follow that the joint distribution comes from some (possibly different) exponential ...
- 21
0
votes
0
answers
22
views
Visualizing relationships in log-link/exponential distribution models by placing the linear predictor on the Y axis?
I'm visualizing results from a negative binomial regression. I don't want to the graph of Y vs X to look exponential, I want it to look linear. In SPSS, the value provided for the linear predictor is ...
2
votes
1
answer
342
views
Distribution of sum of $n$ random variables with mixture of two exponential distributions
Suppose that the random variable $Y$ follows a mixture of two exponential distributions, that is
\begin{equation}
f_Y(y) = \sum_{i=1}^{2}\pi_i f(y| \lambda_i)
\end{equation}
where $\pi$ stands for ...
- 133
0
votes
1
answer
92
views
What distributions are in the exponential family?
Are there any exponential family distributions other than wishart distribution, multivariate normal distribution, Dirichlet distribution, multinomial(or categorical) distribution, Conway-maxwel ...
- 31
0
votes
0
answers
90
views
Antiderivative of log-partition function of exponential family
Let $Y$ be a random variable with distribution belonging to a minimal regular exponential family. Let $\eta $ denote the scalar-valued canonical parameter of the exponential family and let $A$ be the ...
- 221
1
vote
1
answer
74
views
For the family, $f_\theta(x)=\frac{e^{-(x-\theta)}}{1+e^{-(x-\theta)})^2}$, compute the fisher information, is it an exponential family?
For the family, $f_\theta(x)=\frac{e^{-(x-\theta)}}{1+e^{-(x-\theta)})^2}$, compute the fisher information, is it an exponential family, $x\in \mathbb{R},\theta \in \mathbb{R}$?
I computed the fisher ...
- 191
1
vote
0
answers
43
views
Does this distribution belong to the exponential family? [duplicate]
I was looking at a problem in the book of "Statistical Inference" second edition by George Casella and Roger L. Berger from chapter 6 that deals with sufficient statistics, minimal ...
1
vote
2
answers
290
views
Exponential family and conjugate priors
Is a distribution that belongs to the exponential family necessarily conjugate prior?
- 123
0
votes
0
answers
99
views
Prove sum of T(Xi) also belong to exponential family
In mathematical statistics, suppose we have an independent and identically distributed sample X = ($X_1$, $X_2$, …, $X_n$) from a distribution that belongs to the exponential family. Say we can write ...
- 1
6
votes
1
answer
2k
views
What is "natural" about the natural parameterization of an exponential family and the natural parameter space?
In Section 3.4 of Casella and Berger's Statistical Inference, an exponential family is defined to be a set of pdfs or pmfs such that for each member $f(x | \boldsymbol{\theta})$ of the set,
$$
f(x | \...
- 281
0
votes
0
answers
33
views
Gibbs distribution and MCMC
In the paper of An Introduction to HMC for Sampling, I once saw the following statement. My question is that does generic MCMC only apply to Gibbs distribution? I also noticed that many MCMC related ...
- 4,994
2
votes
0
answers
44
views
Origins of Dispersion Models in Statistics [closed]
Recently, I have been reading about Dispersion Models in Statistics. For example, here is an example of the general form of a Dispersion Model:
Additive exponential dispersion model
In the univariate ...
- 7,060
39
votes
5
answers
5k
views
Why is the exponential family so important in statistics?
Why is the exponential family so important in statistics?
I was recently reading about the exponential family within statistics. As far as I understand, the exponential family refers to any ...
- 7,060
0
votes
0
answers
54
views
Definition of exponential family and entropy
I'm reading Graphical Models, Exponential Families, and Variational Inference [pdf] by Wainwright and Jordan.
They define (p. 39) an exponential family by its derivative
$$ p_\theta(x) = \exp \{ \...
- 413
1
vote
1
answer
624
views
What is the dispersion parameter of binomial distribution?
Binomial distribution is a member of exponential dispersion models, but I can not find the dispersion parameter of it.
Could anyone help me find it out?
IMO the Binomial distribution only has an ...
3
votes
1
answer
230
views
Is there a special case of the EM algorithm for exponential family distributions?
According to Wikipedia, the formal definition of the EM algorithm is
The EM algorithm seeks to find the MLE of the marginal likelihood by iteratively applying these two steps:
Expectation step (E ...
- 3,698
1
vote
1
answer
231
views
Fit exponential decay upwards model - start values give 'convergence failure'
I have some data that when plotted looks similar to this:
Then eyeballing the charts on this page, it looks like I might have an exponential decay (Increasing form) relationship?
I searched for how ...
- 1,528
0
votes
0
answers
100
views
Entropy of Poisson random variable via exponential family identity
Exercise 8.1 of Probabilistic Graphical Models asks the reader to use the identity
$$H_{P_{\theta}}(X)= \ln Z(\theta) - \langle E_{P_{\theta}}[\tau(X)], t(\theta)\rangle$$
to compute the entropy $H$ ...
1
vote
1
answer
310
views
Show that this probability distribution is an exponential family
We define a probability distribution on the non-negative integers 0,1,2,...,as having point probabilities $$P_\eta(k)=G(k,\rho)\left(\frac{1}{\rho+\eta}\right)^{\rho}\left(\frac{\eta}{\rho+ \eta}\...
- 305
0
votes
1
answer
50
views
Sufficient Statistic for Absolutely Continuous Distribution [duplicate]
The following is a homework problem. Please tell me if my solution is correct and if not please point out my mistakes.
Let $x_{1}, x_{2},...,x_{M}$ be i.i.d. samples from the absolute continuous ...
- 101
3
votes
1
answer
51
views
Is a parameter-independent support sufficient to belong to an exponential family?
From Wikipedia (emphasis mine):
As a counterexample if these conditions are relaxed, the family of uniform distributions (either discrete or continuous, with either or both bounds unknown) has a ...
- 3,698
3
votes
0
answers
79
views
Why is the canonical parameter linearly related to the input x in GLMs and why does it give the link function?
In Andrew Ng's CS229 notes, one of the three assumptions he makes for constructing GLM models is:
The natural parameter $\eta$ and the inputs x are related linearly: $\eta=\theta^Tx$
He goes on to ...
- 131