Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
2 votes
0 answers
81 views

Order of an exponential family possibly less than the dimension of the convex support and parameter space

I am going though Brown's book on the Fundamentals of Exponential Families. I am puzzled by the statement that he makes on page 16 where it is possible for $\text{Order of the exponential family} <...
amc's user avatar
  • 121
0 votes
1 answer
74 views

In a GLM, how do the dimensions of the linear predictor and the range of the link function always align?

Let $\mathbf{\vec y}$ be the response vector. Then, we can write the exponential family as : $$ \large p(y;\boldsymbol{\eta})=h(y) \exp \left(\boldsymbol{\eta} \cdot \mathbf{T}(y)-A(\boldsymbol{\eta})\...
Sagnik Taraphdar's user avatar
3 votes
1 answer
112 views

Why is this derivation of the mean of the gamma distribution using the log-partition function incorrect?

I am using this formulation of the exponential family : $$ \large f_{X}(x;\boldsymbol{\eta})=h(x) \exp \left(\boldsymbol{\eta} \cdot \mathbf{T}(x)-A(\boldsymbol{\eta})\right) $$ The gamma distribution ...
Sagnik Taraphdar's user avatar
1 vote
1 answer
42 views

How to prove that of set of Exponential Family Distribution functions are linearly independent?

Is it possible to prove that a set of exponential family distributions are linearly independent? I want to show that $\sum_k c_{k} F_{k}(\pmb{x},\pmb{\theta})=0$ has only the trivial solution $c_k$'s ...
mathseeker's user avatar
1 vote
0 answers
19 views

Exponential families as families of limite distributions of Markov processes

An exponential family verifies a maximum entropy property: each density is the maximum entropy density given the expectation of its sufficient statistic. On the other hand, from my understanding, the ...
Chevallier's user avatar
1 vote
0 answers
18 views

Convention Regarding Exponential Family Natural Parametrization

I am self-studying the 2nd edition of Casella and Berger's Statistical Inference and I'm working through the problems on exponential families. They define an exponential family of probability density ...
Georgy Zhukov's user avatar
1 vote
1 answer
57 views

Hypothesis testing involving two parameters in a 2-parameter exponential family

Let $(X, Y)$ be distributed according to the exponential family density $f_{\theta_{1}, \theta_{2}}(x,y)=c( \theta_1 , \theta_2 )h(x,y)exp( \theta_1x+ \theta_2y)$. Show that the only unbiased test for ...
Tapi's user avatar
  • 291
0 votes
0 answers
16 views

Likelihood ratio exponential family under permutation of parameters

I'm reading "ASYMPTOTIC NORMALITY OF MAXIMUM LIKELIHOOD AND ITS VARIATIONAL APPROXIMATION FOR STOCHASTIC BLOCKMODELS" Bickel et al. 2013. In their proof of Lemma 3, they claim a result and I ...
Josh Willcox's user avatar
2 votes
1 answer
84 views

Finding the limiting distribution of $\sqrt{n} (\hat{\tau} - \tau)$ as $n \rightarrow \infty$ for $N(\mu, \mu^2 \tau)$

Let $X_i$ for $i = 1, ..., n$ be a random sample from the distribution $N(\mu, \mu^2 \tau)$ with unknown parameters $\mu \in (\infty, 0) \cup (0 ,\infty), \tau > 0$. Find and justify the mle $\hat{\...
Stats_Rock's user avatar
0 votes
0 answers
24 views

UMP two sided tests for exponential families

Consider a random variable $X$ with density $$f(x : θ) = C(θ)e^{η(θ)T(x)}h(x), θ ∈ Θ$$. Assume that $η(θ)$ is strictly increasing in $θ$ and that the family is full rank. Show that there will not be ...
user671269's user avatar
0 votes
0 answers
26 views

Can expected variance of GLM be expressed through gradients under non-cannonical link functions?

In GLMs (generalized linear models), one can typically obtain an estimate of the expected variance of the response variable given the predictors as a transformation of the same parameter that defines ...
anymous.asker's user avatar
3 votes
1 answer
97 views

What concentration $\kappa \in [0, \infty)$ maximizes the entropy of the von Mises-Fisher distribution?

I'd like to prove what concentration parameter $\kappa \in [0, \infty)$ maximizes the (differential) entropy of a von-Mises Fisher Distribution. The differential entropy of of a von Mises-Fisher ...
Rylan Schaeffer's user avatar
0 votes
0 answers
36 views

Natural Exponential family with polynomial partition function

Consider a natural exponential family: \begin{equation} f(x|\theta) = h(x) e^{ x \theta -A(\theta) } \end{equation} We assume that we are supported on $\mathbb{R}$ (i.e., $h(x)>0$ for $x \in \...
Lisa's user avatar
  • 131
0 votes
1 answer
41 views

Obtaining the geometry of $\varphi_{\theta}(x)$ where the mean and variance are $\theta$-dependent

Consider an (unnormalized) univariate distribution in the exponential family, which is in canonical form: $$\varphi_{\theta}(x)= \exp \bigg( \frac{\theta}{\log x} \bigg)$$ $x\in(0,1).$ $\varphi_{\...
geocalc33's user avatar
2 votes
1 answer
131 views

Showing incompleteness of density

You observe a sample of 100 independent observations $X_i$ from a population with the density $$ g(x)=C \sqrt{\lambda} \exp \left(-\lambda x^2-\lambda^2 x^4\right), \quad-\infty<x<\infty $$ ...
Stats_Rock's user avatar
0 votes
0 answers
24 views

Can I adjust lambda by 1 or 2 S.E. in exponential decay to potential improvements in retention

(Similar example data to a post I made posted yesterday but a completely different question) I have an exponential decay with off-set function that looks like this: ...
Doug Fir's user avatar
  • 1,558
0 votes
0 answers
23 views

What is meaning/geometric interpretation of the Expectation parameters of a multivariate gaussian (exponential family)?

According to page 17 of Statistical exponential families the expectation parameters for a multivariate gaussian are given by $$ \boldsymbol{H}=\left(\mu,-(\Sigma+\mu\mu^T)\right)=(\eta,H) $$ I am ...
ADDonut's user avatar
1 vote
1 answer
80 views

van der Vaart Asymptotic Statistics, page 38, why does $e_\theta'=\operatorname{Cov}_{\theta}t(X)$?

On Page 38 of van der Vaart's Asymptotic Statistics (near the bottom of the page), it says By differentiating $E_\theta t(X)$ under the expectation sign (which is justified by the lemma), we see that ...
ExcitedSnail's user avatar
  • 2,596
2 votes
0 answers
25 views

Is an exponential family such with respect to any dominating measure?

Let $\{P_\vartheta \mid \vartheta \in \Theta\}$ be a family of probability measures on $(\mathbb R^n,\mathfrak B_{\mathbb R^n})$, parametrized by the subset $\Theta$ of $\mathbb R^m$. Consider the ...
Federico's user avatar
  • 131
3 votes
1 answer
282 views

Is there an exponential family such that its natural parameter mapping is non-invertible or has non-convex range?

On the Wikipedia article for exponential families the density of a distribution on a measure space $(X, \xi)$ from an exponential family is written as $$f_{\theta} \colon X \to \mathbb{R}_{\ge 0}, \...
ViktorStein's user avatar
0 votes
0 answers
33 views

Calculate the first and second moment of high-order polynomial exponential family

Suppose that X is from the density $$ f(x, \lambda, \mu)=(\lambda / C) e^{-\lambda^4(x-\mu)^4}, \quad \text { for all } \quad x \in \mathbb{R}, \quad C=\frac{1}{2} \Gamma(1 / 4) $$ where $\theta=(\...
Stats_Rock's user avatar
2 votes
1 answer
81 views

Distribution of the exponential of a Gamma distributed random variable

I have a random variable $X$ that follows a Gamma distribution. $$ X \sim \text{Gamma}[\alpha, \beta] $$ I want to know the distribution of $Y$, i.e., $$ Y = a - \exp\left(-b X\right) $$
CfourPiO's user avatar
  • 265
5 votes
1 answer
180 views

A lemma concerning the distribution of sufficient statistic from exponential family

I understand Lemma 8 in Chapter 1 from Lehmann's Testing Statistical Hypotheses [or Lemma 2.7.2 in Lehmann and Romano] as follows: If the pdf of an exponential family is $$p_{\theta}(x)=\exp\bigg\{\...
rryan's user avatar
  • 53
1 vote
1 answer
36 views

Prove covariance between sufficient statistic and logarithm of base measure in exponential family is equal to zero

Exponential family form is $$f_X(x) = h(x)\exp(\eta(\theta)\cdot T(x) - A(\theta))$$ I know $$\operatorname{Cov}(T(x), \log(h(x)) = 0.$$ But how can I prove it?
user388375's user avatar
0 votes
0 answers
50 views

Are there maximum entropy distributions with fixed moments of a certain order?

A characterization of the multivariate Gaussian distribution with a fixed mean and covariance matrix is that it is the unique probability distribution that maximizes differential entropy. That is, the ...
Quizzical's user avatar
  • 101
10 votes
2 answers
541 views

How to show that $\{N(\theta,1):\theta \in \Omega\}$ is not a complete family of distributions when $\Omega$ is finite?

Consider the $\{N(\theta,1):\theta \in \Omega\}$ family of distributions where $\Omega=\{-1,0,1\}$. I am trying to show that this is not a complete family. That is, if $X\sim N(\theta,1)$, I need to ...
StubbornAtom's user avatar
  • 10.9k
0 votes
0 answers
9 views

General form of the dispersion function for GLMs

Generalised linear models assume the response to belong to exponential dispersion model family with density (or probability mass function) of the form $$ p(y \mid \theta, \phi) = \exp \left \{ \frac{y ...
Othman El Hammouchi's user avatar
2 votes
1 answer
164 views

For some $\tau=\tau(\theta)$, there exists an unbiased estimator (UMVUE), then the distribution belongs to an exponential family

I read the textbook in Cramer-Rao lower bound (CRLB). Here is a theroem For some $\tau=\tau(\theta)$, there exists an unbiased estimator $\hat{\tau}$ of $\tau$ such that $Var(\hat{\tau})$ attains the ...
Hermi's user avatar
  • 667
0 votes
0 answers
40 views

Why do we want to constrain E[ln(x)] in some maximum entropy models?

If we look at the table of distributions in the exponential family, we will see some sufficient statistics have $\log(x)$, which means we have put constraints on $\mathbb{E}[\log(X)]$ when formulating ...
Ziyuan's user avatar
  • 1,736
1 vote
1 answer
55 views

Find out which exponential distribution the data belongs to

I'm doing a GLM homework, and I'm stuck with the following problem: Suppose that data ($Y_i$; $\mathbf{X}_i$); $i = 1, . . . , n$ are observed, where $\mathbf{X}_i$ is a p-dimensional vector for ...
alice123019's user avatar
0 votes
1 answer
316 views

How to prove the Poisson link function is a canonical link function?

So I'm a 3rd year undergraduate doing my thesisin football score models right now. In my thesis I want to include a proof of what the link function for the Poisson distribution is and why it relates ...
Nikhil Handa's user avatar
1 vote
0 answers
54 views

Expectation of Fisher score not equal to 0 when parametrize Categorical distribution differently

Expectation of Fisher score should equal to zero. The prove can be found in many palces, such as wikipedia. But I tried a categorical distribution that is not parameterizatized minimally, the expected ...
Haotian Chen's user avatar
2 votes
0 answers
49 views

Fenchel conjugate of the cumulant function (Exponential Family)

We have a minimal exponential family $f(x) = h(x) \exp(\langle\theta, t(x)\rangle - A(\theta)).$ The canonical parameter space is $\Theta = \{ \theta \in \mathbb{R}^d \colon \int h(x) \exp(\langle\...
Phil's user avatar
  • 616
0 votes
0 answers
123 views

Mean Parametrization of Bernoulli Distribution(Exponential Family)

In class, we defined an exponential family to be of distributions with density of form $$h(x) \exp(\langle\theta, T(x)\rangle-A(\theta))$$ $\langle~,~\rangle$ is the inner product and $\theta \in \...
Phil's user avatar
  • 616
1 vote
1 answer
87 views

How to check if an exponential family is regular?

A strictly $k$-parameter exponential family $$f=\exp\left(\sum \eta_i(\theta)T_i(x)-B(\theta)\right)h(x)$$ is regular if the natural parameter space $\eta(\theta)$ contains a $k$-dimensional open set. ...
orangecat's user avatar
  • 163
2 votes
1 answer
191 views

Writing exponential family in canonical form

I have the following pdf with support $x>0$: $$f_{\mu}(x)=\frac{1}{\sqrt{2\pi x^3}}\textrm{exp}\left(-\frac{(x-\mu)^2}{2\mu^2x}\right)$$ This belongs to the exponential family, and I write this in ...
pecer10012's user avatar
0 votes
0 answers
125 views

Smallest threshold for hypothesis test with asymptotic level alpha

Consider a distribution with parameter $\lambda$ that has density $$f_\lambda(x)=\frac{x^4}{24\lambda^5}e^{\frac{-x}{\lambda}},x>0$$ Let $X_1,...,X_n$ be $n$ independent random variables drawn from ...
pecer10012's user avatar
1 vote
0 answers
104 views

Conditions on the log partition of exponential family generating cumulants

Is any minimality condition of the parameterization required such that the derivative of the log partition function is equal to the mean? The pdf of an exponential family distribution is given by $$p(...
louixp's user avatar
  • 31
1 vote
3 answers
110 views

Why is $\theta\mapsto p_{\theta}$ one-to-one $\iff$ $(n+1)$ functions $\{F_1,\ldots,F_n,1\}$ are linearly independent?

If an n-dimensional model $S=\{p_{\theta}| \theta\in\Theta\}$ can be expressed in terms of the functions $\{C,F_1,\ldots,F_n\}$ on a sample space $X$ and a function $\psi\in \Theta$(parameter space) ...
Andyale's user avatar
  • 133
0 votes
0 answers
77 views

Is it possible to derive the probit model by writing it in exponential family form?

We always use latent variable approach to derive probit model, is there any way to derive probit model from the exponential family form (by using link function)? Also, does logit-normal distribution ...
doraemon's user avatar
  • 198
0 votes
0 answers
36 views

What probability distribution matches $1/(e^x - 1)?$

I've been looking at the proof of Planck's Law. This is the first law that triggered the science of quantum theory by introducing the idea of quanta (which come in the form of packets of energy of ...
shalso's user avatar
  • 1
2 votes
1 answer
100 views

Does this distribution belong to exponential family and does its support depend on $p?$

I have this bivariate distribution and I would like to tell if it is in the exponential family: $$f(y_{i},d_{i}|\theta_{1}\,\theta_{2}\,p)=\left(\left(\frac{1}{\theta_{1}}\right)\exp\left(\frac{-y_{i}}...
Iambereftoflordship's user avatar
4 votes
1 answer
453 views

Can the Beta-regression be written in the GLM form?

The Beta distribution is: $$p(y)=\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}y^{\alpha-1}(1-y)^{\beta-1} $$ It's part of the exponential family. We can reparametrize this with using mean ...
Maverick Meerkat's user avatar
-1 votes
1 answer
59 views

Why does $E\left(y_{i t}\right)=a^{\prime}\left(\theta_{i t}\right)$? in the context of assuming some GEE marginal density?

In generalized estimating equations we have a glm-response variable. To establish notation, we let $Y_{i}=\left(y_{i 1}, \ldots, y_{i n_{i}}\right)^{\text {T }}$ be the $n_{i} \times 1$ vector of ...
user avatar
1 vote
0 answers
29 views

Where is the h function of the exponential family gone?

The problem comes from equation (4.86) of text "Pattern Recognition and Machine Learning" written by Christopher M. Bishop $$a_k({\bf x})={\bf\lambda}_k^T{\bf x}+\ln g({\bf\lambda}_k)+\ln p(\...
zzzhhh's user avatar
  • 323
2 votes
1 answer
218 views

GAM with opposite outcomes with different families

I'm building GAMs and I have some doubts regarding the family to use. I'm fitting GAMs because I expect some non-linear relationships between the response variable and some covariates. I've checked a ...
mto23's user avatar
  • 637
0 votes
0 answers
68 views

UMP Test and UMVUE when there are nuisance parameters

Consider $X_1,...,X_n \sim Weibull(\theta, c)$ where $c>0$ is unknown. Several textbook examples consider when $c$ is known, but here, we consider when $c$ is unknown. Suppose now we wanted to find ...
s l's user avatar
  • 87
1 vote
1 answer
331 views

Is the Generalized Dirichlet Distribution an Exponential Family?

Is the Generalized Dirichlet distribution an exponential family? If so, what is its log-normalizer, sufficient statistics, and carrier measure?
Neil G's user avatar
  • 15.1k
2 votes
1 answer
103 views

Impact of sample size when using exponential random graph modeling

I am working on revising a manuscript centered on identifying the drivers behind certain types of student interaction. One critique that I'm having trouble addressing is that they are worried about my ...
Dave Esparza's user avatar
2 votes
1 answer
244 views

What is the name of the functions in exponential dispersion family?

If an exponential family is given by: $g(y|\theta) = exp\{\theta^TT(y)-A(\theta)\}h(y)$ then the functions $h(y)$, $A(\theta)$ and $T(y)$ are defined by names: $T(y)$ is a sufficient statistic $A(\...
DHJ's user avatar
  • 107

1
2 3 4 5
9