Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Gamma family as conjugate prior of Inverse Gaussian with known $\mu$

I want to show that, when $\mu=\mu_0$, then gamma family $\Gamma(a,b)$ is a conjugate prior to inverse Gaussian with density $f(x,\mu,\lambda)=\sqrt{\frac{\lambda}{2\pi x^2}}exp[-\frac{\lambda(x-\mu)^...
4
votes
1answer
45 views

Practical method to do MLE for natural parameters in exponential family

I encountered the following question in my research and I hope this is the correct place to post it. I'm following the notation in this lecture note by Michael I. Jordan. Assume random vector $X$ ...
4
votes
0answers
36 views

Why is the EM algorithm well suited for exponential families?

I've been brushing up on the EM algorithm, and while I feel like I understand the basics, I keep seeing the claim made (e.g. here, here, among several others) that EM works particularly well for ...
0
votes
0answers
18 views

Multivariate canonical exponential family

Consider the canonical d-dimensional exponential family with densities $$p(x)=exp\left(\langle\theta,T(x)\rangle-A(\theta)\right)h(x),\theta\in\Omega$$ with $\Omega\subset\Omega_0=\{\theta:A(\theta)&...
0
votes
0answers
10 views

What are the hyperparameters and base measure in the conjugate prior for the exponential family?

Setup Suppose we have an exponential family model $\{P_{\theta} : \theta \in \Theta\}$. Let the density function of a random variable $X$ and the prior on $\theta$ have following forms: $$ \begin{...
0
votes
1answer
22 views

Jointly complete and sufficient statistics for multivariate normal distribution

Consider the random sample X from the multivariate normal distribution where xi are i.i.d as N(µ,Σ). *Show that the sample mean x̄ and Sample covariance matrix S are jointly complete and sufficient ...
0
votes
1answer
22 views

Canonical form and exponential family

Suppose you have a random variable X, who's distribution depends on $\theta$. If X is a part of the exponential family of distributions, X can be written in a certain form, namely: $$f_\theta(x)=h(x)*...
0
votes
1answer
23 views

How to prove part of exponential family [closed]

We saw that in order to prove a distribution is a part of a exponential family we should be able to write it in this form: f(y;θ)=exp(a(y)b(θ)+c(θ)+d(y) And to prove that: All statistics T are ...
0
votes
1answer
48 views

Does density belong to exponential family?

$$f(x;\theta) = 2x\theta\exp({-x^2})\left( \frac{\exp({-x^2})}{1-\exp({-x^2})}\right)^{\theta\ - 1}\mathbb I_{(\mathbb R_{++})}(x) $$ with $\theta \in \mathbb R_{++} $ does $f(x;\theta)$ belong to ...
2
votes
0answers
22 views

What family of full support probability distributions satisfy that the density of any point in the domain vanishes as the variance goes to infinity?

Let $f(x,\sigma^2)$ be a representative element of a family of PDF's with full support over the reals that is indexed by their variance $\sigma^2$. Under what general conditions of the family of ...
3
votes
1answer
37 views

determine the minimal exponential family form

Suppose we have a random variable $X$ having p.d.f of the form $$f(x|\theta)=\exp\{c(\theta)'T(x)-B(\theta)\}h(x),$$ then we say $X$ is from exponential family. Further, we say that an exponential ...
1
vote
0answers
21 views

Exponential Family distribution with non-open natural parameter space? [duplicate]

Is there any example of canonical exponential family distribution with a natural parameter space, that is not open? An $k$ dimensional canonical exponential family means having a p.d.f (w.r.t base ...
4
votes
1answer
60 views

$\mathbb{E}(X^2+Y^2)=-2\rho$? $(X,Y)$ is from standard bivariate normal distribution and $Cov(X,Y)=\rho$

How to use log partition function to derive $\mathbb{E}(X^2+Y^2)$, where $(X,Y)$ is from standard bivariate normal distribution? By standard bivariate normal I mean $\mu_x=\mu_y=0$ and $\sigma^2_X=\...
3
votes
1answer
51 views

Example of curved exponential family with $T$ being a complete statistic?

Is there any example of curved exponential family with $T$ being a complete statistic? Here $T$ is the sufficient statistic.
1
vote
0answers
16 views

UMP test for exponential family when sufficient statistics $T$ is a vector

Assume we have a random sample $X_1,\dots,X_n$ from a distribution of the form $f(x_i;\theta) = h(x)g(\theta)\exp(\eta(\theta) T(x))$ and we wish to test $H_0: \theta \leq \theta_0, H_1: \theta > \...
3
votes
0answers
28 views

Can the parameter space of a single parameter exponential family in canonical form be a closed interval?

I know that the parameter space has to be convex (that I've proved using Jensen's inequality). Therefore, since it has to be a subset of $\mathbb R$, it follows that it has to be an interval or a ...
7
votes
1answer
174 views

Is $\mathrm{binomial}(n, p)$ family both full and curved for $n$ fixed?

Let $n$ be a fixed positive integer. The binomial$(n, p)$ family is given by $$f(x|p)=\tbinom{n}{x}p^x(1-p)^{n-x}\tag{1}.$$ We may rewrite (1) as $$f(x|p)=\tbinom{n}{x}(1-p)^n\exp\left[x\log\frac{p}{1-...
1
vote
2answers
63 views

Completeness of a statistic - Open ball

I was studying the slides of the course in statistics, but there is a theorem that is not clear for me. This chapter was about finding a complete statistic, and it explains that it can be found with ...
0
votes
0answers
63 views

What is the log-normalizer of the conjugate prior of an exponential family?

Let's say that you have a distribution $F$ in the exponential family with density \begin{align} \newcommand{\mbx}{\mathbf x} \newcommand{\btheta}{\boldsymbol{\theta}} f(\mbx \mid \btheta) &= \exp\...
1
vote
1answer
24 views

Linear relations between statistics in exponential family of distributions

I am reading about point estimation from Theory of Point Estimation by Lehmann and Casella (1999). I couldn't understand the following point mentioned in p.24, under the exponential family of ...
2
votes
0answers
44 views

Mean and variance of the Beta distribution using identities of exponential families

I was studying the part of exponential families from Statistical Inference (George Casella, Roger L. Berger) and they give the following definition of an exponential family: $$ f(x|\pmb{\theta}) = h(x)...
4
votes
2answers
166 views

Distribution of the sufficient statistic in the exponential family?

Suppose $\boldsymbol X$ belongs to the exponential family, $$ f_X\!\left(\,\mathbf{x} ; \boldsymbol \theta\,\right) = h(\mathbf{x}) \, \exp\!\Big(\,\boldsymbol\eta({\boldsymbol \theta}) \cdot \mathbf{...
0
votes
0answers
43 views

how to choose family in CVgam in r?

I'm trying to run a repeated k-fold cross validation of my GAM model, but I can't specify my family in CVgam in R. I want to compare poisson GAM and Negative binomial GAM. How can I solve this ...
1
vote
0answers
20 views

Calculating the conditional expectation of an exponential family [closed]

If we have $X$ with a density depending on the scalar parameter $\theta$, where the density is from of the exponential family: $f(x;\theta) = \exp(\theta x−\phi(\theta))h(x)$. Also we have that $\...
0
votes
0answers
25 views

Examples of functions in generalized linear models

Let $X \in {R}^{n\times p}, Y \in R^{n}$ and $\beta \in R^p$, I am trying to find examples of generalized linear models where the posterior $g_{\beta|y} \propto \exp(-f(y;X,\beta) - \lambda\|\beta\|_2^...
0
votes
0answers
40 views

Normal Conjugate Prior, Known Mean and Unknown Variance?

For Normal distribution, with know mean and unknown variance. When $\tau = 1/\sigma^2$ ~ Gamma(). In such has posterior of $\tau$ has the following distribution: $p(\tau|\alpha, \beta, x) \sim G(\...
2
votes
1answer
44 views

Reparametrization and its effect on sufficient/complete/minimal statistics

Suppose $X_1 \sim Pois(\lambda_1), X_2 \sim Pois(\lambda_2), X_3 \sim Pois(\lambda_1+\lambda_2)$. Separately I can find a sufficient, complete and minimal statistic for each of them. But considering ...
0
votes
0answers
37 views

Can I build deviance residuals from an XGBoost model that learns an exponential family parameter?

I'm taking a course on GLMs after a few years of using machine learning models. The good about GLMs is how the probabilistic model ties in with the estimation and evaluation. So I'm trying to transfer ...
1
vote
0answers
69 views

PDF of Beta Distribution written as exponential family form

I am trying to write the pdf of a beta random variable in its biparametric canonic form such as: Function 1 $$ f_Y(y; \theta, \phi) = exp \{ \phi[y \theta - b(\theta)] + c(y, \phi) \} \mathbb{1}_A(...
0
votes
0answers
10 views

Interpretation of concentration of posteriors in the limit of infinitely many independent versus dependent random variables

Disclaimer: the setup and specific example may not be a minimal example to illustrate the point, but I am not well-versed in these topics enough to construct a smaller example without accidentally ...
0
votes
0answers
24 views

Reparameterizing PDF to get components of exponential family

For the following problem, I am trying to identify the components of the exponential family in the form: $\exp(y\theta - b(\theta))/a(\phi) + c(y; \phi)$ Namely, I need to identify the $\theta, b(\...
0
votes
0answers
253 views

Extreme Value (Gumbel) distribution a member of exponential family

This is a question for discussion in my Linear Model class. I am having a hard time showing that the distribution belongs to the exponential family PDF: $f(y; \theta) = 1/\varphi \exp([y − \theta]/\...
2
votes
0answers
27 views

GLMM and Exponential Family

I am studying now GLMMs (Generalized Linear Mixed Models). From my understanding, in order to estimate the parameters of this model, you need to arrive at the marginal probability by integrating a ...
1
vote
1answer
83 views

Find the prior distribution for the natural parameter of an exponential family

Show that for the binomial likelihood $y$ ~$Bin(n, \theta)$, $p(\theta) \propto \theta^{-1} (1-\theta)^{-1}$ is the uniform prior distribution for the natural parameter of the exponential family. I am ...
1
vote
2answers
285 views

Does Linear regression needs target variable to be normally distributed. (GLM context)?

I came across the assumptions of linear regression that said: -->The residuals should be normally distributed. GLM(Generalized Linear model) assumes that target variable should follow one of the ...
2
votes
2answers
76 views

If Y has an exponential family distribution show that $E(\frac{\partial L}{\partial \theta}) = 0$

I'm working in a self study fashion preparing for a course I'm going to take this semester in generalised linear models. The question is, given that the Y random variable belongs to the exponential ...
2
votes
1answer
78 views

Is a maximum likelihood estimator in an exponential family always sufficient?

An exponential family (under natural parameterization) is such that $p(X|\eta)=h(X)\exp\{\eta^TT(X)-A(\eta)\}$, where $X$ is the data, $\eta$ is the natural parameter, and $h,T,A$ are some functions (...
0
votes
0answers
19 views

History of the curved exponential family

Does anyone know the first person who introduced the curved exponential family and also which paper it was first presented? I vaguely remember that it might be Fisher who wrote about it in a paper on ...
3
votes
1answer
147 views

Conjugate priors outside exponential family

The usual exception I have come across regarding non-existence of conjugate prior outside the exponential family is the uniform distribution on $(0,\theta)$ (i.e. $U(0,\theta)$) where $\theta$ has a ...
2
votes
1answer
81 views

How do I find the MLE of the APE distribution in R?

The random variable $Y$ is said to have a two-parameter APE distribution, denoted by $\text{APE}(\alpha, \lambda)$, with the shape parameter $\alpha>0$ and scale parameter $\lambda>0$ if the ...
1
vote
2answers
81 views

Bayesian inference with an arbitrary prior

A classical problem in Bayesian inference arises when we wish to learn about (say) the fraction $\theta$ of balls in an urn that are white; and do so by sampling from the urn with replacement. In such ...
5
votes
0answers
169 views

Let $X_1,X_2,\dots,X_n$ be random sample from Poisson($\theta$). Find MVUE of $e^{-2\theta}$

Question: Let $X_1,X_2,\dots,X_n$ be random sample from Poisson($\theta$). Find MVUE of $e^{-2\theta}$ My attempt has been by modifying the answer from this question: The Poisson distribution is a one-...
0
votes
0answers
81 views

UMVUE of Bernoulli random variables

Let $X_1, X_2..... X_n$ be a random sample from a Bernoulli population with parameter $p$. A sufficient statistic is $\sum_{i=1}^{n}X_i$. If we define $$ U(X_1,X_2,\ldots,X_n)= \begin{cases}1/2n &\...
0
votes
0answers
20 views

What is the conjugate prior for the hypoexponential distribution?

Can't find it anywhere. I know Gamma is the conjugate prior for the exponential distribution (one parameter) but for the sum of exponential distributions (the hypoexponential distribution), I can't ...
0
votes
0answers
10 views

Count Metric Inter-arrival Times

I have a process that I would like to simulate. I have count data for the process in minutes (ie 60/minute). Because I do not have access to more granular data, I cannot model the interarrival times ...
6
votes
2answers
189 views

Intuition for why the (log) partition function matters?

I'm on a quest for the intuition behind the fact that theoretical introductions to approximate inference focus so much on the log partition function. Say we have a regular exponential family $$p(\...
2
votes
0answers
31 views

Why does the canonical parameter give a link function? Why does this relate $E[Y]$ to $x^T \beta$?

If I have a pdf in the form $f(y|\theta,\phi)=\text{exp}\bigg(\frac{y\theta-b(\theta)}{a(\phi)}+c(y,\phi)\bigg)$, then $\theta$ is called the canonical parameter. I'm told we can get a link function $...
3
votes
1answer
62 views

Saddlepoint approximation for Exponential family

I read the following in a book: The saddlepoint approximation of an exponential family density function is $$\tilde P(y;\mu,\phi) = \frac{1}{\sqrt{2\pi \phi V(y)}}exp(-\frac{d(y, \mu)}{2\phi})$$ Where ...
1
vote
1answer
75 views

n'th cumulant (of a CGF) for exponential family / exponential dispersion model

The n'th cumulant is defined to be the n'th derivative of the CGF (cumulant generating function). $$\kappa_n = \frac{d^n K(t)}{dt^n} |_{t=0} $$ But I'm reading in a book (p.215, chapter5, eq. 5.8) ...
0
votes
1answer
47 views

marginal conditional from joint of three r.v.'s

I have two random variables, $X$ and $Y$. I know that $X \sim \text{Gamma}(a,b)$ and $Y \sim \text{Gamma}(c,d)$. Furthermore, I know that $Z \sim \text{Poisson}(XY)$. I know the joint distribution ...

1
2 3 4 5
7