Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
2
votes
0answers
25 views

Are there nonparametric generative models for datasets?

Typically when I see generative models, e.g., Latent Dirichlet Allocation (JMLR) or Linear/Quadratic Discriminant Analysis (wikipedia LDA), they are probabilistic models that belong to the exponential ...
1
vote
1answer
28 views

Finding Uniformly Most Powerful test

My Attempt Comparing $f(x;\theta)$ with the form $a(\theta)b(x)exp[c(\theta)d(x)]$ , we get $d(x) = log (1-x)$ and $ c(\theta ) = \theta -1 $ as monotone , increasing function in $\theta$ and ...
3
votes
1answer
61 views

explanation of why an UMVUE doesn't necessarily have to achieve the CRLB?

I'm studying uniformly minimum variance unbiased estimator(UMVUE). I have seen question on this site asking why the UMVUE doesn't achieve the CRLB(Cramer Rao lower bound), and all of the answers have ...
2
votes
0answers
12 views

Show Moment estimator and MLE are the same only for exponential family

Show Moment estimator and MLE are the same only for exponential family. I know MM is variant while MLE is not under transformation. Will this be the starting point?
0
votes
1answer
18 views

Derivation of the Objective Function for Expectation Propagation

I was reading Expectation Propagation As A Way Of Life and the original paper by Minka Expectation Propagation for Approximate Bayesian Inference and they both say that a fixed point of the EP ...
0
votes
0answers
11 views

Exponential family with finite variance

I am wondering if there exists any popular exponential family with finite variance (that does not depend on parameter $\theta$) In other words, is there family of the form: $$ p_\theta(x) = h(x)e^{\...
0
votes
0answers
39 views

Exponential family and efficient estimator

In my lecture notes there is the notion of efficiency related to the exponential family. More precisely, the lecturer stated that for an exponential family an efficient estimator always exists. How is ...
0
votes
0answers
13 views

Connection between subgaussian/subexponential and exponential family

I am wondering if there is any relationship between subgaussian/subexponential with (one parameter) exponential family. In particular, is there any sub-family density that belongs to both ...
4
votes
1answer
72 views

Is the t distribution a member of the exponential family?

From what I understand, the exponential family is defined as $$f(y;\theta,\phi) = \exp\left(\frac{y\theta - b(\theta)}{a(\phi)}+c(y,\phi)\right) $$ I've read (but not seen shown anywhere), that the ...
0
votes
1answer
31 views

How to derive a pdf of Complete Sufficient Statistic of exponential family

While studying Mathematical statistics through "Introduction to Mathematical Statistics 7th" (by Hogg and Craig), I've been stuck in the Theorem above. The answer of the exercise 7.5.8 is not given in ...
1
vote
1answer
78 views

Is the Erlang Distribution member of the Exponential family?

Given the Erlang distribution $f(x)={\begin{cases}\displaystyle {\frac {\lambda ^{n}x^{{n-1}}}{(n-1)!}}\,{\mathrm {e}}^{{-\lambda x}}&x\geq 0\\0&x<0\end{cases}}$ I want to determine, ...
0
votes
0answers
14 views

Reparameterize b(K,pi) in terms of theta

Question: $X_1, ... , X_n$ follows a binomial distribution with parameters K and 0 < $\pi\ <1.$ Use properties of Regular Exponential Class of distributions to show that the sample total $T = \...
1
vote
0answers
55 views

Convergence in distribution of parameters of exponential family

I am taking a course in inference where we have to find an approximate confidence interval for a Rayleigh distributed variable. The correct answer to this question states: Since we have an ...
0
votes
0answers
10 views

Finding prior conjugate for reparametrized model

Let $X_i$ be iid Bernoulli$(\pi)$ for $i=1,...,n$. My task is to find the prior conjugate for $\theta$, where $\theta$ is the natural parameter of the sampling model. The sampling model can be ...
1
vote
1answer
18 views

Is a range of values from an exponential distribution still exponentially distributed?

I have to generate numbers of two different exponential distribution ($e_1, e_2$) with parameters respectively $\lambda_1$ and $\lambda_2 = k \lambda_1$, with $0<k<1$. But I also want to ...
2
votes
1answer
48 views

What is the broadest context where $\bar{X}$ is complete sufficient for $\mathbb E(X)$?

$\bar{X}$ is complete sufficient for $\mathbb E(X)$ if $X$ is Normal with known standard deviation $\sigma$. Are there broader contexts? Like for exponential families in general or more general than ...
0
votes
0answers
6 views

What is the log-normaliser for the Normal-Wishart distribution?

Suppose we have a $NW(\mu, \Lambda | \mu_0, \lambda_0, \nu, \mathbf{W}) = N(\mu | \mu_0, (\lambda_0\Lambda)^{-1})W(\Lambda | \nu, \mathbf{W})$, how do we derive the log-normaliser for this? I would ...
0
votes
1answer
68 views

Can distributions that are in the exponential family, but not the natural exponential family, be formed as GLM?

The lognormal and beta distributions are in the exponential family but not the natural exponential family. Generalized Linear Models are often advertised as being models for response variables that ...
1
vote
0answers
54 views

Is this exponential family pdf of full rank?

Suppose $X_1, ..., X_n$ are iid random variables from a distribution with Lebesgue pdf $$f_\theta(x) = \exp\left\{-\left(\frac{x-\mu}{\sigma}\right)^4 - \xi(\theta)\right\}$$ It is easy to show that ...
1
vote
1answer
41 views

Fisher Information for general one parameter exponential family (inconsistent with Poisson distribution)

For one of my hw questions, I was asked to derive Fisher Information for one parameter exponential family. Here's my approach: $$L(\theta) = f(x\mid\theta) = e^{\theta T(x) - \eta(\theta)}h(x)$$ $$\...
1
vote
1answer
24 views

Show that Y from a exponential family has $\phi = 1/k$ if $Y=Z/k$ where $Z \sim B(k, p)$

Suppose Y from an exponential family $$ f(y;\theta,\phi) = \exp\left\{\frac{y\theta - b(\theta)}{a(\phi)} + c(y, \phi)\right\} $$ I'm struggling to show that if $Y = Z / k$ and $Z \sim B(k, p)$ (...
1
vote
1answer
49 views

Exponential family definition appears vacuous

I am going through Michael Jordan's notes on exponential families and an exponential family is defined w.r.t. functions $h(\cdot), T(\cdot)$, and parameter $\eta$ such that $$ p(x | \eta) = h(x) \exp\...
2
votes
1answer
87 views

Ambiguity with math notation

I was just solving several problems and get stuck into the one with some weird notation. I wasn't able to understand it, though I know the meaning behind the character (angle bracket) in more broader ...
0
votes
0answers
16 views

Moment parametrization for exponential families where the observation is not a sufficient statistics

Consider a probability distribution belonging to an exponential family $$ f(y; \theta) = c(y) \exp ( \eta(\theta)^\top T(y) - \kappa(\theta)) $$ In the case where $\eta(\theta) = \theta$ (always ...
0
votes
0answers
40 views

Expectation of exponential family distributions

Is there a closed form of the following marginal (one dimensional data) $\pi(\theta|y) = \mathbb{E}_{x \sim \pi_R(x|y)} \pi(\theta|x)$, where both $\pi, \pi_R$ are exponential family distributions?
0
votes
0answers
47 views

If $f(x|\theta)$ is conjugate to $p(\theta)$ then is $f(x|r\theta)$ conjugate to $p(\theta)$?

If exponential family $f(x|\theta)$ is conjugate to $p(\theta)$ then is $f(x|r\theta)$ for $r>0$ conjugate to $p(\theta)$? If not, what can we do about it in terms of sampling to make use of ...
0
votes
1answer
32 views

Percentages in utilization of different distributions

I'm sure that this question has been asked before on CV but, in drilling through many pages of previous CV questions, no matches surfaced. Regardless, I'm confident some observant participant will be ...
1
vote
0answers
44 views

Role of different exponential families in generalized linear models

I've gone through a variety of introductions to generalized linear models (GLM), and there's always a point in the discussion that confuses me. The story often begins saying that $P(y|x)$ belongs to a ...
1
vote
1answer
59 views

Definition of exponential family with dispersion parameter

I was recently reading a discussion of generalized linear models that considered the response to come from an exponential family with a dispersion parameter so $$ f(y|\theta,\phi) = \exp\left(\frac{y\...
20
votes
2answers
1k views

Advantages of the Exponential Family: why should we study it and use it?

So here I am studying inference. I would like that someone could enumerate the advantages of the exponential family. By exponential family, I mean the distributions which are given as \begin{align*} f(...
5
votes
1answer
82 views

Name and interpretation of “$h(x)$” in exponential family

The exponential family is defined (in many sources) as: $$p(x | \theta) = h(x) \exp\{\theta^TT(x) - A(\theta)\}$$ where: $T(x)$ is a sufficient statistic, $\theta$ is a canonical parameter, and $A(...
0
votes
1answer
59 views

Let $X\sim\text{Rayleigh}(\theta^{2})$. Prove that $T_{n}$ is consistent, given that $T_{n}(\textbf{X}) = \frac{1}{2n}\sum_{i=1}^{n}x^{2}_{i}$

Let $X\sim\text{Rayleigh}(\theta^{2})$. Prove that $T_{n}$ is consistent, given that $$T_{n}(\textbf{X}) = \frac{1}{2n}\sum_{i=1}^{n}x^{2}_{i}$$ MY ATTEMPT To begin with, let us notice that \begin{...
2
votes
0answers
56 views

Finding the mean and variance for pmf $P(X_i = x)=-\theta^x/x\log(1-\theta)$?

I'd like to verify that my working/thinking is correct. This is a problem from Keener's book, but the answer isn't provided. Let $X$ have distribution $P(X = x)=-\theta^x/x\log(1-\theta)$ for $x=1,2,\...
0
votes
0answers
28 views

How do we obtain the canonical parameter from the variance function in the context of exponential family distributions?

I am assuming the following definition of the exponential family: \begin{align*} f(y,\theta,\phi) = \exp\left\{\phi[y\theta - b(\theta)] + c(y,\phi)\right\} \end{align*} In such context, the variance ...
0
votes
2answers
89 views

Is the zero truncated Poisson Distribution part of the Exponential Family? [duplicate]

This is the density of a truncated Poisson: $$P(X = x \mid X > 0) = \frac{\lambda ^ x e^{- \lambda} }{x ! \left ( 1 - e^{- \lambda} \right )}$$ To show that it's member of the Exponential ...
0
votes
0answers
11 views

How do we get the canonical parameter $\theta$ from the exponential famility through the variance function?

The exponential family can be described by \begin{align*} f(y|\theta,\phi) = \exp\left\{\phi^{-1}(y\theta - b(\theta) + c(y,\phi)\right\} \end{align*} It also can be shown that $\textbf{E}(Y) = \mu = ...
0
votes
0answers
80 views

Does the family of binomial distributions conditioned on $X > 0$ belong to the exponential family?

Does the family of distributions where $p(x,\theta)$ is the conditional frequency function of a binomial $\mathcal{B}(n,\theta)$, variable $X$, given that $X > 0$, belong to the exponencial family? ...
0
votes
0answers
614 views

Exponential family and geometric distribution: how do we prove the sum of independent geometric random variables has negative binomial distribution?

Let $X$ be the number of failures before the first success in a sequence of Bernoulli trials with probability of success $\theta$. Then $P_{\theta}[X = k] = (1-\theta)^{k}\theta$, $k = 1,2,\ldots$ ...
2
votes
0answers
38 views

Maximum entropy probability distribution over non-negative support and finite mean?

I'm trying to derive which univariate probability distribution maximizes entropy, assuming finite mean $\mu$ and non-negative support $[0, \infty)$. I know that the answer is the exponential ...
0
votes
0answers
31 views

Does conjugate prior for natural exponential family needs jacobian to transform natural parameter back to original parameter?

From bayesian theory, we have that if $f(x|\eta) \propto \exp(\eta \cdot T(x)- A(\eta))$ - a natural exponential family, then the prior conjugate of $\eta$ is $\pi^*(\eta | \mu, \lambda) \propto \exp(\...
1
vote
0answers
38 views

If $X$ is from (canonical) exponential family, do we always know the distribution of $T(X)$?

Assume $X$ are generated by a distribution from exponential family, $$ f(X; \theta) = h(X)\exp\{\eta(\theta)T(X) - b(\theta)\}$$ After solving several exercises with various distribution functions, ...
0
votes
1answer
32 views

Reagarding the base measure h(x) in the exponential family

most authors define the canonical form of the exponential family as $$ p(\mathbf{x} | \boldsymbol{\theta})=h(\mathbf{x}) \exp (\boldsymbol{\eta}(\boldsymbol{\theta}) \cdot \mathbf{T}(\mathbf{x})-A(\...
0
votes
0answers
56 views

Proof that one distribution is a GLM (general linear model)

Given $Y_1, Y_2,...,Y_n$ i.i.d random variables, where $Y_i|x_i \sim N(\mu_i, \sigma^2)$ and $\mu_i = \beta_0 + \log(\beta_1 + \beta_2x_i)$. How do I proof that the distribution is a GLM (general ...
0
votes
0answers
28 views

Mixed parameterization of sample from normal distribution

I am studying exponential families and mixed parameterizations. Now, I am told that $$ \mathbf{\theta} = \begin{bmatrix}\mu\\ -\frac{1}{2\sigma^2}\end{bmatrix} $$ is the parameter in a variation-...
0
votes
0answers
95 views

obtain (minimal) sufficient statistic for $\gamma$ knowing the canonical statistic $\theta(\gamma)$

A statistical model for a data set y is an exponential family , with canonical parameter vector $\theta= (\theta_1,\theta_2,..\theta_k)$ and canonical statistic $t(y) = (t_1(y),t_2(y),..t_k(y))$ if ...
3
votes
0answers
172 views

UMVUE for $g(p) = \mathbb{E}_p[X^2]$, where X follows a geometric distribution

I have a random variable X with pmf $$p_\lambda(x) = (1-p)^{x-1}p, \ \ x = 1,2,3,\ldots, \ \ p \in (0,1)$$ and I am trying to find a UMVUE for $$g(p) = \mathbb{E}_p[X^2]$$. Here is my attempt so ...
1
vote
0answers
81 views

Find a UMP test in exponential family - first stage

Consider the exponential family $f(x;\theta) = c(\theta)\exp\{b(\theta) T(x))\}h(x)$ and I would like to find a UMP test for $$H_0: \theta \leq 0\quad \hbox{vs}\quad H_1: \theta > 0.$$ It is ...
1
vote
1answer
69 views

Is the marginal distribution of a maximum entropy distribution also a maximum entropy distribution?

I am considering under what circumstances maximum entropy distributions are closed under marginalization. The main case I am interested is the following setting: a finite set of discrete random ...
0
votes
0answers
140 views

Sufficient Statistic and MLE

Suppose $X_1, \dots, X_n \sim B(1,p)$. Show that a sufficient statistic for $\theta = (1-p)^2$ is $T(x) = \sum X_i$ and that the MLE for $\theta$ is $(1-\frac{1}{n}T)^2$. I am having a lot of ...
0
votes
1answer
51 views

What is matrix norm in (Collins, 2001)

I am studying "A generalization of PCA to the exponential family" (Collins et al., 2001) and I don't understand some notations. What is the meaning of the matrix squared norm on page 6 ? Is it a ...

1 2 3 4 5 6