Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
5
votes
0answers
55 views

Let $X_1,X_2,\dots,X_n$ be random sample from Poisson($\theta$). Find MVUE of $e^{-2\theta}$

Question: Let $X_1,X_2,\dots,X_n$ be random sample from Poisson($\theta$). Find MVUE of $e^{-2\theta}$ My attempt has been by modifying the answer from this question: The Poisson distribution is a one-...
0
votes
0answers
37 views

UMVUE of Bernoulli random variables

Let $X_1, X_2..... X_n$ be a random sample from a Bernoulli population with parameter $p$. A sufficient statistic is $\sum_{i=1}^{n}X_i$. If we define $$ U(X_1,X_2,\ldots,X_n)= \begin{cases}1/2n &\...
0
votes
0answers
11 views

What is the conjugate prior for the hypoexponential distribution?

Can't find it anywhere. I know Gamma is the conjugate prior for the exponential distribution (one parameter) but for the sum of exponential distributions (the hypoexponential distribution), I can't ...
0
votes
0answers
5 views

Count Metric Inter-arrival Times

I have a process that I would like to simulate. I have count data for the process in minutes (ie 60/minute). Because I do not have access to more granular data, I cannot model the interarrival times ...
6
votes
2answers
103 views

Intuition for why the (log) partition function matters?

I'm on a quest for the intuition behind the fact that theoretical introductions to approximate inference focus so much on the log partition function. Say we have a regular exponential family $$p(\...
2
votes
0answers
25 views

Why does the canonical parameter give a link function? Why does this relate $E[Y]$ to $x^T \beta$?

If I have a pdf in the form $f(y|\theta,\phi)=\text{exp}\bigg(\frac{y\theta-b(\theta)}{a(\phi)}+c(y,\phi)\bigg)$, then $\theta$ is called the canonical parameter. I'm told we can get a link function $...
1
vote
0answers
19 views

Saddlepoint approximation for Exponential family

I read the following in a book: The saddlepoint approximation of an exponential family density function is $$\tilde P(y;\mu,\phi) = \frac{1}{\sqrt{2\pi \phi V(y)}}exp(-\frac{d(y, \mu)}{2\phi})$$ ...
1
vote
1answer
20 views

n'th cumulant (of a CGF) for exponential family / exponential dispersion model

The n'th cumulant is defined to be the n'th derivative of the CGF (cumulant generating function). $$\kappa_n = \frac{d^n K(t)}{dt^n} |_{t=0} $$ But I'm reading in a book (p.215, chapter5, eq. 5.8) ...
0
votes
1answer
44 views

marginal conditional from joint of three r.v.'s

I have two random variables, $X$ and $Y$. I know that $X \sim \text{Gamma}(a,b)$ and $Y \sim \text{Gamma}(c,d)$. Furthermore, I know that $Z \sim \text{Poisson}(XY)$. I know the joint distribution ...
6
votes
1answer
92 views

How do I find the UMVUE of $\sqrt{\alpha}$ here?

new user here self-studying some mathematical statistics. I came across this problem and am stuck. Problem: Suppose that for $i = 1, ... , n$, the positive random variables $X_i$ are independent and ...
0
votes
0answers
16 views

Differentiating the Log-Normalizer of the Gamma distribution

Converting the Gamma distribution to the Exponential family form, you get: $$f(x) = \frac{\beta^{\alpha}}{\Gamma(\alpha)}x^{\alpha-1}e^{-\beta x} = e^{(\alpha-1)\ln(x) -\beta x -(\ln(\Gamma(\alpha)-\...
1
vote
0answers
15 views

Show bivariate normal distribution with non-diagonal covariance belongs to curved exponential family?

Question: Suppose that $(X_{i}, Y_{i})$, $i = 1, \dots ,n$ are sampled i.i.d. from the two-dimensional normal distribution $$ \begin{bmatrix} X & Y \end{bmatrix} \sim \mathcal{N}\left( \begin{...
0
votes
1answer
17 views

Exponential families and Gibbs measures? Any relation?

The title says it all. The formulas for exponential families and Gibbs measures seems very similar. Is there any relationship, or some kind of translation table?
1
vote
1answer
51 views

Prove that argmin of exponential distributions has multinomial distribution [closed]

Let's say we have $ T_1,T_2,\cdots,T_n \sim Exp$ and $P(X_1>a)=e^{-\lambda_1 a},P(X_2>a)=e^{-\lambda_2 a},\cdots,P(X_n>a)=e^{-\lambda_n a} $. How can I describe $\DeclareMathOperator*...
1
vote
1answer
39 views

Definition of $k$-parameter exponential family

I am currently studying the concept of sufficient statistics in mathematical statistics. The following definition is presented: Definition: $k$-parameter exponential family Let $\mathbf{Y} \sim f_\...
0
votes
0answers
28 views

Division of Multivariate Normal Distributions

$$ \newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\nc}[2]{\newcommand{#1}{#2}} \nc{\vx}{\vect{x}} \nc{\vmu}{\vect{\mu}} \nc{\vSigma}{\vect{\Sigma}} \nc{\vtheta}{\vect{\theta}} $$ ...
0
votes
0answers
103 views

Bayesian Linear Regression and the Exponential Family

In a straight forward linear regression model, assuming a fixed input $\mathbf{x}$, and additive noise with unit variance we can write: \begin{equation} p(y\mid \mathbf{x,w})=\frac{1}{\sqrt{2\pi}\...
0
votes
2answers
48 views

What is the idea behind generalized linear models?

I am watching andrew ng's video lectures on machine learning. I am trying to understand what is even the point of generalized linear models. I understand what goes on step by step in deriving things ...
0
votes
0answers
34 views

Is there just one exponential family? Or are there many exponential families? [duplicate]

I'm confused by the phrasing I've seen about exponential families. What does it mean to say "an" exponential family. Why not "the" exponential family? From a pdf from Berkely: "we define an ...
2
votes
0answers
40 views

Are there nonparametric generative models for datasets?

Typically when I see generative models, e.g., Latent Dirichlet Allocation (JMLR) or Linear/Quadratic Discriminant Analysis (wikipedia LDA), they are probabilistic models that belong to the exponential ...
1
vote
1answer
46 views

Finding Uniformly Most Powerful test

My Attempt Comparing $f(x;\theta)$ with the form $a(\theta)b(x)exp[c(\theta)d(x)]$ , we get $d(x) = log (1-x)$ and $ c(\theta ) = \theta -1 $ as monotone , increasing function in $\theta$ and ...
3
votes
1answer
125 views

explanation of why an UMVUE doesn't necessarily have to achieve the CRLB?

I'm studying uniformly minimum variance unbiased estimator(UMVUE). I have seen question on this site asking why the UMVUE doesn't achieve the CRLB(Cramer Rao lower bound), and all of the answers have ...
2
votes
0answers
18 views

Show Moment estimator and MLE are the same only for exponential family

Show Moment estimator and MLE are the same only for exponential family. I know MM is variant while MLE is not under transformation. Will this be the starting point?
0
votes
1answer
31 views

Derivation of the Objective Function for Expectation Propagation

I was reading Expectation Propagation As A Way Of Life and the original paper by Minka Expectation Propagation for Approximate Bayesian Inference and they both say that a fixed point of the EP ...
0
votes
0answers
13 views

Exponential family with finite variance

I am wondering if there exists any popular exponential family with finite variance (that does not depend on parameter $\theta$) In other words, is there family of the form: $$ p_\theta(x) = h(x)e^{\...
0
votes
0answers
47 views

Exponential family and efficient estimator

In my lecture notes there is the notion of efficiency related to the exponential family. More precisely, the lecturer stated that for an exponential family an efficient estimator always exists. How is ...
1
vote
1answer
38 views

Connection between subgaussian/subexponential and exponential family

I am wondering if there is any relationship between subgaussian/subexponential with (one parameter) exponential family. In particular, is there any sub-family density that belongs to both ...
5
votes
1answer
137 views

Is the t distribution a member of the exponential family?

From what I understand, the exponential family is defined as $$f(y;\theta,\phi) = \exp\left(\frac{y\theta - b(\theta)}{a(\phi)}+c(y,\phi)\right) $$ I've read (but not seen shown anywhere), that the ...
0
votes
1answer
38 views

How to derive a pdf of Complete Sufficient Statistic of exponential family

While studying Mathematical statistics through "Introduction to Mathematical Statistics 7th" (by Hogg and Craig), I've been stuck in the Theorem above. The answer of the exercise 7.5.8 is not given in ...
1
vote
1answer
95 views

Is the Erlang Distribution member of the Exponential family?

Given the Erlang distribution $f(x)={\begin{cases}\displaystyle {\frac {\lambda ^{n}x^{{n-1}}}{(n-1)!}}\,{\mathrm {e}}^{{-\lambda x}}&x\geq 0\\0&x<0\end{cases}}$ I want to determine, ...
0
votes
0answers
17 views

Reparameterize b(K,pi) in terms of theta

Question: $X_1, ... , X_n$ follows a binomial distribution with parameters K and 0 < $\pi\ <1.$ Use properties of Regular Exponential Class of distributions to show that the sample total $T = \...
1
vote
0answers
57 views

Convergence in distribution of parameters of exponential family

I am taking a course in inference where we have to find an approximate confidence interval for a Rayleigh distributed variable. The correct answer to this question states: Since we have an ...
0
votes
0answers
11 views

Finding prior conjugate for reparametrized model

Let $X_i$ be iid Bernoulli$(\pi)$ for $i=1,...,n$. My task is to find the prior conjugate for $\theta$, where $\theta$ is the natural parameter of the sampling model. The sampling model can be ...
1
vote
1answer
18 views

Is a range of values from an exponential distribution still exponentially distributed?

I have to generate numbers of two different exponential distribution ($e_1, e_2$) with parameters respectively $\lambda_1$ and $\lambda_2 = k \lambda_1$, with $0<k<1$. But I also want to ...
2
votes
1answer
51 views

What is the broadest context where $\bar{X}$ is complete sufficient for $\mathbb E(X)$?

$\bar{X}$ is complete sufficient for $\mathbb E(X)$ if $X$ is Normal with known standard deviation $\sigma$. Are there broader contexts? Like for exponential families in general or more general than ...
0
votes
0answers
7 views

What is the log-normaliser for the Normal-Wishart distribution?

Suppose we have a $NW(\mu, \Lambda | \mu_0, \lambda_0, \nu, \mathbf{W}) = N(\mu | \mu_0, (\lambda_0\Lambda)^{-1})W(\Lambda | \nu, \mathbf{W})$, how do we derive the log-normaliser for this? I would ...
0
votes
1answer
139 views

Can distributions that are in the exponential family, but not the natural exponential family, be formed as GLM?

The lognormal and beta distributions are in the exponential family but not the natural exponential family. Generalized Linear Models are often advertised as being models for response variables that ...
1
vote
0answers
91 views

Is this exponential family pdf of full rank?

Suppose $X_1, ..., X_n$ are iid random variables from a distribution with Lebesgue pdf $$f_\theta(x) = \exp\left\{-\left(\frac{x-\mu}{\sigma}\right)^4 - \xi(\theta)\right\}$$ It is easy to show that ...
1
vote
1answer
107 views

Fisher Information for general one parameter exponential family (inconsistent with Poisson distribution)

For one of my hw questions, I was asked to derive Fisher Information for one parameter exponential family. Here's my approach: $$L(\theta) = f(x\mid\theta) = e^{\theta T(x) - \eta(\theta)}h(x)$$ $$\...
1
vote
1answer
33 views

Show that Y from a exponential family has $\phi = 1/k$ if $Y=Z/k$ where $Z \sim B(k, p)$

Suppose Y from an exponential family $$ f(y;\theta,\phi) = \exp\left\{\frac{y\theta - b(\theta)}{a(\phi)} + c(y, \phi)\right\} $$ I'm struggling to show that if $Y = Z / k$ and $Z \sim B(k, p)$ (...
1
vote
1answer
52 views

Exponential family definition appears vacuous

I am going through Michael Jordan's notes on exponential families and an exponential family is defined w.r.t. functions $h(\cdot), T(\cdot)$, and parameter $\eta$ such that $$ p(x | \eta) = h(x) \exp\...
2
votes
1answer
90 views

Ambiguity with math notation

I was just solving several problems and get stuck into the one with some weird notation. I wasn't able to understand it, though I know the meaning behind the character (angle bracket) in more broader ...
0
votes
0answers
18 views

Moment parametrization for exponential families where the observation is not a sufficient statistics

Consider a probability distribution belonging to an exponential family $$ f(y; \theta) = c(y) \exp ( \eta(\theta)^\top T(y) - \kappa(\theta)) $$ In the case where $\eta(\theta) = \theta$ (always ...
0
votes
0answers
42 views

Expectation of exponential family distributions

Is there a closed form of the following marginal (one dimensional data) $\pi(\theta|y) = \mathbb{E}_{x \sim \pi_R(x|y)} \pi(\theta|x)$, where both $\pi, \pi_R$ are exponential family distributions?
0
votes
0answers
51 views

If $f(x|\theta)$ is conjugate to $p(\theta)$ then is $f(x|r\theta)$ conjugate to $p(\theta)$?

If exponential family $f(x|\theta)$ is conjugate to $p(\theta)$ then is $f(x|r\theta)$ for $r>0$ conjugate to $p(\theta)$? If not, what can we do about it in terms of sampling to make use of ...
0
votes
1answer
33 views

Percentages in utilization of different distributions

I'm sure that this question has been asked before on CV but, in drilling through many pages of previous CV questions, no matches surfaced. Regardless, I'm confident some observant participant will be ...
1
vote
0answers
75 views

Role of different exponential families in generalized linear models

I've gone through a variety of introductions to generalized linear models (GLM), and there's always a point in the discussion that confuses me. The story often begins saying that $P(y|x)$ belongs to a ...
2
votes
1answer
87 views

Definition of exponential family with dispersion parameter

I was recently reading a discussion of generalized linear models that considered the response to come from an exponential family with a dispersion parameter so $$ f(y|\theta,\phi) = \exp\left(\frac{y\...
20
votes
2answers
2k views

Advantages of the Exponential Family: why should we study it and use it?

So here I am studying inference. I would like that someone could enumerate the advantages of the exponential family. By exponential family, I mean the distributions which are given as \begin{align*} f(...
5
votes
1answer
95 views

Name and interpretation of “$h(x)$” in exponential family

The exponential family is defined (in many sources) as: $$p(x | \theta) = h(x) \exp\{\theta^TT(x) - A(\theta)\}$$ where: $T(x)$ is a sufficient statistic, $\theta$ is a canonical parameter, and $A(...

1
2 3 4 5 6