Skip to main content

Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
1 vote
0 answers
28 views

Convexity of negative log-likelihood of exponential family distribution [closed]

Let $p(y; \theta^Tx) = b(y) \space \exp\big(\theta^T x y - a(\theta^T x)\big)$, where $x$ and $\theta$ are $d$-dimensional vectors and $y$ a scalar. If I'm not mistaken, the negative log-likelihood ...
Arturo Sbr's user avatar
0 votes
0 answers
33 views

Distribution of the sample covariance of a multivariate exponential family

I am wondering if there is a known form for the distribution of the sample covariance matrix of a random variable that follows a multivariate exponential family distribution. I guess it would be a ...
aurora_s's user avatar
0 votes
0 answers
20 views

Exponential family expression for bivariate Gaussian

Consider a bivariate Gaussian distribution with the following density: $$f(x,y) = \frac{1}{2\pi \sqrt{1-\rho^2}}e^{-\frac{1}{2(1-\rho^2)}(x^2-2\rho xy+y^2)}$$ How can we write it in the exponential ...
efsdfmo12's user avatar
  • 123
0 votes
0 answers
22 views

GLM with canonical link: Why linear regression over the natural parameter?

So I've been wondering why it is "natural" to extend linear models where we assume $Y\sim N(\mu,\Sigma)$ and try to fit $E[Y|X]=X\beta$ to generalized linear models where we assume $Y_i\sim ...
R.V.N.'s user avatar
  • 123
0 votes
0 answers
42 views

How to derive the expectation of $\log[a \theta_k + b]$ in Dirichlet distribution?

Given that $\boldsymbol{\theta} \sim \mathrm{Dir}(\boldsymbol{\alpha})$, then $E_{p(\boldsymbol{\theta} \mid \boldsymbol{\alpha})}[\log{\theta_k}] = \Psi(\alpha_k) - \Psi(\sum_{k'=1}^K \alpha_{k'})$, ...
Henry Zha's user avatar
1 vote
0 answers
19 views

Sample from a piecewise exponential distribution [duplicate]

Given a distribution $p(x)\propto \exp(\min_{i=1}^N [a_i^Tx + b_i])$, where $x$ and $a_i$ are both D-dimensional vectors, $b_i$ is a scalar, and $(a_i,b_i)$ are known. How can I sample from it? This ...
user avatar
1 vote
1 answer
44 views

Can we find a variance formula in terms of the log of the density? ($f(t) = e^{u(t)}$, need variance in terms of $u$)

Given a density $f(t)$, say over $t\in \mathbb{R}^n$ absolutely continuous with Lebesgue measure. Write $u(t) = \log(f(t))$ taking values in $[-\infty, \infty)$ so $f = \exp(u)$. Can we find a general ...
travelingbones's user avatar
6 votes
3 answers
133 views

Is Pitman-Koopman-Darmois Theorem valid for discrete random variables?

I am interested in the Pitman-Koopman-Darmois theorem. I'm having a hard time finding a simple rigorous version of this theorem as I struggle finding sources. This helpful post provides three sources ...
Pohoua's user avatar
  • 2,618
1 vote
0 answers
43 views

How does reparametrization of the Fisher information matrix change the variance expression for the sufficient statistics?

If I have an exponential family distribution of the form $$p_{\theta}(x) = e^{\theta^T\cdot t(x) - \psi(\theta)},$$ where $\theta$ is a vector of parameters, $t(x)$ is a vector of sufficient ...
absolutelyzeroEQ's user avatar
0 votes
0 answers
27 views

Statistical test for bias to even sizes

I'm studying an experiment which should follow a type of birth-death process, which tells me the number of cells at a certain time from the same lineage. So, I have a discrete empirical distribution ...
mathsisfun's user avatar
0 votes
0 answers
51 views

Find $E[Y]$ when $f(x,y) = \frac{x}{3}e^{-xy}$

Truth be told, I don't really have an issue with this problem in general, but in it's calculation. Let me explain. We need to find $E[Y]$ when $f(x,y) = \frac{x}{3}e^{-xy}$, $1<x<4$ and $y>0$...
Anweshan Goswami's user avatar
2 votes
0 answers
86 views

Order of an exponential family possibly less than the dimension of the convex support and parameter space

I am going though Brown's book on the Fundamentals of Exponential Families. I am puzzled by the statement that he makes on page 16 where it is possible for $\text{Order of the exponential family} <...
amc's user avatar
  • 121
1 vote
1 answer
83 views

In a GLM, how do the dimensions of the linear predictor and the range of the link function always align?

Let $\mathbf{\vec y}$ be the response vector. Then, we can write the exponential family as : $$ \large p(y;\boldsymbol{\eta})=h(y) \exp \left(\boldsymbol{\eta} \cdot \mathbf{T}(y)-A(\boldsymbol{\eta})\...
Sagnik Taraphdar's user avatar
4 votes
1 answer
132 views

Why is this derivation of the mean of the gamma distribution using the log-partition function incorrect?

I am using this formulation of the exponential family : $$ \large f_{X}(x;\boldsymbol{\eta})=h(x) \exp \left(\boldsymbol{\eta} \cdot \mathbf{T}(x)-A(\boldsymbol{\eta})\right) $$ The gamma distribution ...
Sagnik Taraphdar's user avatar
1 vote
1 answer
42 views

How to prove that of set of Exponential Family Distribution functions are linearly independent?

Is it possible to prove that a set of exponential family distributions are linearly independent? I want to show that $\sum_k c_{k} F_{k}(\pmb{x},\pmb{\theta})=0$ has only the trivial solution $c_k$'s ...
mathseeker's user avatar
1 vote
0 answers
24 views

Exponential families as families of limite distributions of Markov processes

An exponential family verifies a maximum entropy property: each density is the maximum entropy density given the expectation of its sufficient statistic. On the other hand, from my understanding, the ...
Chevallier's user avatar
1 vote
0 answers
19 views

Convention Regarding Exponential Family Natural Parametrization

I am self-studying the 2nd edition of Casella and Berger's Statistical Inference and I'm working through the problems on exponential families. They define an exponential family of probability density ...
Georgy Zhukov's user avatar
2 votes
1 answer
73 views

Hypothesis testing involving two parameters in a 2-parameter exponential family

Let $(X, Y)$ be distributed according to the exponential family density $f_{\theta_{1}, \theta_{2}}(x,y)=c( \theta_1 , \theta_2 )h(x,y)exp( \theta_1x+ \theta_2y)$. Show that the only unbiased test for ...
Tapi's user avatar
  • 311
0 votes
0 answers
16 views

Likelihood ratio exponential family under permutation of parameters

I'm reading "ASYMPTOTIC NORMALITY OF MAXIMUM LIKELIHOOD AND ITS VARIATIONAL APPROXIMATION FOR STOCHASTIC BLOCKMODELS" Bickel et al. 2013. In their proof of Lemma 3, they claim a result and I ...
Josh Willcox's user avatar
2 votes
1 answer
91 views

Finding the limiting distribution of $\sqrt{n} (\hat{\tau} - \tau)$ as $n \rightarrow \infty$ for $N(\mu, \mu^2 \tau)$

Let $X_i$ for $i = 1, ..., n$ be a random sample from the distribution $N(\mu, \mu^2 \tau)$ with unknown parameters $\mu \in (\infty, 0) \cup (0 ,\infty), \tau > 0$. Find and justify the mle $\hat{\...
Stats_Rock's user avatar
1 vote
0 answers
53 views

UMP two sided tests for exponential families

Consider a random variable $X$ with density $$f(x : θ) = C(θ)e^{η(θ)T(x)}h(x), θ ∈ Θ$$. Assume that $η(θ)$ is strictly increasing in $θ$ and that the family is full rank. Show that there will not be ...
user671269's user avatar
1 vote
0 answers
35 views

Can expected variance of GLM be expressed through gradients under non-cannonical link functions?

In GLMs (generalized linear models), one can typically obtain an estimate of the expected variance of the response variable given the predictors as a transformation of the same parameter that defines ...
anymous.asker's user avatar
3 votes
1 answer
116 views

What concentration $\kappa \in [0, \infty)$ maximizes the entropy of the von Mises-Fisher distribution?

I'd like to prove what concentration parameter $\kappa \in [0, \infty)$ maximizes the (differential) entropy of a von-Mises Fisher Distribution. The differential entropy of of a von Mises-Fisher ...
Rylan Schaeffer's user avatar
0 votes
0 answers
37 views

Natural Exponential family with polynomial partition function

Consider a natural exponential family: \begin{equation} f(x|\theta) = h(x) e^{ x \theta -A(\theta) } \end{equation} We assume that we are supported on $\mathbb{R}$ (i.e., $h(x)>0$ for $x \in \...
Lisa's user avatar
  • 131
0 votes
1 answer
42 views

Obtaining the geometry of $\varphi_{\theta}(x)$ where the mean and variance are $\theta$-dependent

Consider an (unnormalized) univariate distribution in the exponential family, which is in canonical form: $$\varphi_{\theta}(x)= \exp \bigg( \frac{\theta}{\log x} \bigg)$$ $x\in(0,1).$ $\varphi_{\...
geocalc33's user avatar
2 votes
1 answer
135 views

Showing incompleteness of density

You observe a sample of 100 independent observations $X_i$ from a population with the density $$ g(x)=C \sqrt{\lambda} \exp \left(-\lambda x^2-\lambda^2 x^4\right), \quad-\infty<x<\infty $$ ...
Stats_Rock's user avatar
0 votes
0 answers
26 views

Can I adjust lambda by 1 or 2 S.E. in exponential decay to potential improvements in retention

(Similar example data to a post I made posted yesterday but a completely different question) I have an exponential decay with off-set function that looks like this: ...
Doug Fir's user avatar
  • 1,588
0 votes
0 answers
24 views

What is meaning/geometric interpretation of the Expectation parameters of a multivariate gaussian (exponential family)?

According to page 17 of Statistical exponential families the expectation parameters for a multivariate gaussian are given by $$ \boldsymbol{H}=\left(\mu,-(\Sigma+\mu\mu^T)\right)=(\eta,H) $$ I am ...
ADDonut's user avatar
1 vote
1 answer
94 views

van der Vaart Asymptotic Statistics, page 38, why does $e_\theta'=\operatorname{Cov}_{\theta}t(X)$?

On Page 38 of van der Vaart's Asymptotic Statistics (near the bottom of the page), it says By differentiating $E_\theta t(X)$ under the expectation sign (which is justified by the lemma), we see that ...
ExcitedSnail's user avatar
  • 2,966
2 votes
0 answers
27 views

Is an exponential family such with respect to any dominating measure?

Let $\{P_\vartheta \mid \vartheta \in \Theta\}$ be a family of probability measures on $(\mathbb R^n,\mathfrak B_{\mathbb R^n})$, parametrized by the subset $\Theta$ of $\mathbb R^m$. Consider the ...
Federico's user avatar
  • 131
3 votes
1 answer
317 views

Is there an exponential family such that its natural parameter mapping is non-invertible or has non-convex range?

On the Wikipedia article for exponential families the density of a distribution on a measure space $(X, \xi)$ from an exponential family is written as $$f_{\theta} \colon X \to \mathbb{R}_{\ge 0}, \...
ViktorStein's user avatar
2 votes
1 answer
101 views

Distribution of the exponential of a Gamma distributed random variable

I have a random variable $X$ that follows a Gamma distribution. $$ X \sim \text{Gamma}[\alpha, \beta] $$ I want to know the distribution of $Y$, i.e., $$ Y = a - \exp\left(-b X\right) $$
CfourPiO's user avatar
  • 235
5 votes
1 answer
188 views

A lemma concerning the distribution of sufficient statistic from exponential family

I understand Lemma 8 in Chapter 1 from Lehmann's Testing Statistical Hypotheses [or Lemma 2.7.2 in Lehmann and Romano] as follows: If the pdf of an exponential family is $$p_{\theta}(x)=\exp\bigg\{\...
rryan's user avatar
  • 65
1 vote
1 answer
38 views

Prove covariance between sufficient statistic and logarithm of base measure in exponential family is equal to zero

Exponential family form is $$f_X(x) = h(x)\exp(\eta(\theta)\cdot T(x) - A(\theta))$$ I know $$\operatorname{Cov}(T(x), \log(h(x)) = 0.$$ But how can I prove it?
user388375's user avatar
11 votes
2 answers
658 views

How to show that $\{N(\theta,1):\theta \in \Omega\}$ is not a complete family of distributions when $\Omega$ is finite?

Consider the $\{N(\theta,1):\theta \in \Omega\}$ family of distributions where $\Omega=\{-1,0,1\}$. I am trying to show that this is not a complete family. That is, if $X\sim N(\theta,1)$, I need to ...
StubbornAtom's user avatar
  • 11.4k
2 votes
1 answer
234 views

For some $\tau=\tau(\theta)$, there exists an unbiased estimator (UMVUE), then the distribution belongs to an exponential family

I read the textbook in Cramer-Rao lower bound (CRLB). Here is a theroem For some $\tau=\tau(\theta)$, there exists an unbiased estimator $\hat{\tau}$ of $\tau$ such that $Var(\hat{\tau})$ attains the ...
Hermi's user avatar
  • 747
1 vote
1 answer
57 views

Find out which exponential distribution the data belongs to

I'm doing a GLM homework, and I'm stuck with the following problem: Suppose that data ($Y_i$; $\mathbf{X}_i$); $i = 1, . . . , n$ are observed, where $\mathbf{X}_i$ is a p-dimensional vector for ...
alice123019's user avatar
0 votes
1 answer
440 views

How to prove the Poisson link function is a canonical link function?

So I'm a 3rd year undergraduate doing my thesisin football score models right now. In my thesis I want to include a proof of what the link function for the Poisson distribution is and why it relates ...
Nikhil Handa's user avatar
1 vote
0 answers
62 views

Expectation of Fisher score not equal to 0 when parametrize Categorical distribution differently

Expectation of Fisher score should equal to zero. The prove can be found in many palces, such as wikipedia. But I tried a categorical distribution that is not parameterizatized minimally, the expected ...
Haotian Chen's user avatar
2 votes
0 answers
50 views

Fenchel conjugate of the cumulant function (Exponential Family)

We have a minimal exponential family $f(x) = h(x) \exp(\langle\theta, t(x)\rangle - A(\theta)).$ The canonical parameter space is $\Theta = \{ \theta \in \mathbb{R}^d \colon \int h(x) \exp(\langle\...
Phil's user avatar
  • 636
0 votes
0 answers
131 views

Mean Parametrization of Bernoulli Distribution(Exponential Family)

In class, we defined an exponential family to be of distributions with density of form $$h(x) \exp(\langle\theta, T(x)\rangle-A(\theta))$$ $\langle~,~\rangle$ is the inner product and $\theta \in \...
Phil's user avatar
  • 636
1 vote
1 answer
100 views

How to check if an exponential family is regular?

A strictly $k$-parameter exponential family $$f=\exp\left(\sum \eta_i(\theta)T_i(x)-B(\theta)\right)h(x)$$ is regular if the natural parameter space $\eta(\theta)$ contains a $k$-dimensional open set. ...
orangecat's user avatar
  • 183
2 votes
1 answer
249 views

Writing exponential family in canonical form

I have the following pdf with support $x>0$: $$f_{\mu}(x)=\frac{1}{\sqrt{2\pi x^3}}\textrm{exp}\left(-\frac{(x-\mu)^2}{2\mu^2x}\right)$$ This belongs to the exponential family, and I write this in ...
pecer10012's user avatar
0 votes
0 answers
170 views

Smallest threshold for hypothesis test with asymptotic level alpha

Consider a distribution with parameter $\lambda$ that has density $$f_\lambda(x)=\frac{x^4}{24\lambda^5}e^{\frac{-x}{\lambda}},x>0$$ Let $X_1,...,X_n$ be $n$ independent random variables drawn from ...
pecer10012's user avatar
1 vote
0 answers
126 views

Conditions on the log partition of exponential family generating cumulants

Is any minimality condition of the parameterization required such that the derivative of the log partition function is equal to the mean? The pdf of an exponential family distribution is given by $$p(...
louixp's user avatar
  • 31
1 vote
3 answers
118 views

Why is $\theta\mapsto p_{\theta}$ one-to-one $\iff$ $(n+1)$ functions $\{F_1,\ldots,F_n,1\}$ are linearly independent?

If an n-dimensional model $S=\{p_{\theta}| \theta\in\Theta\}$ can be expressed in terms of the functions $\{C,F_1,\ldots,F_n\}$ on a sample space $X$ and a function $\psi\in \Theta$(parameter space) ...
Andyale's user avatar
  • 133
0 votes
0 answers
87 views

Is it possible to derive the probit model by writing it in exponential family form?

We always use latent variable approach to derive probit model, is there any way to derive probit model from the exponential family form (by using link function)? Also, does logit-normal distribution ...
doraemon's user avatar
  • 250
0 votes
0 answers
37 views

What probability distribution matches $1/(e^x - 1)?$

I've been looking at the proof of Planck's Law. This is the first law that triggered the science of quantum theory by introducing the idea of quanta (which come in the form of packets of energy of ...
shalso's user avatar
  • 1
2 votes
1 answer
125 views

Does this distribution belong to exponential family and does its support depend on $p?$

I have this bivariate distribution and I would like to tell if it is in the exponential family: $$f(y_{i},d_{i}|\theta_{1}\,\theta_{2}\,p)=\left(\left(\frac{1}{\theta_{1}}\right)\exp\left(\frac{-y_{i}}...
Iambereftoflordship's user avatar
4 votes
1 answer
666 views

Can the Beta-regression be written in the GLM form?

The Beta distribution is: $$p(y)=\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}y^{\alpha-1}(1-y)^{\beta-1} $$ It's part of the exponential family. We can reparametrize this with using mean ...
Maverick Meerkat's user avatar

1
2 3 4 5
9