Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

0
votes
0answers
14 views

obtain (minimal) sufficient statistic for $\gamma$ knowing the canonical statistic $\theta(\gamma)$

A statistical model for a data set y is an exponential family , with canonical parameter vector $\theta= (\theta_1,\theta_2,..\theta_k)$ and canonical statistic $t(y) = (t_1(y),t_2(y),..t_k(y))$ if ...
3
votes
0answers
33 views

UMVUE for $g(p) = \mathbb{E}_p[X^2]$, where X follows a geometric distribution

I have a random variable X with pmf $$p_\lambda(x) = (1-p)^{x-1}p, \ \ x = 1,2,3,\ldots, \ \ p \in (0,1)$$ and I am trying to find a UMVUE for $$g(p) = \mathbb{E}_p[X^2]$$. Here is my attempt so ...
1
vote
1answer
27 views

Is the marginal distribution of a maximum entropy distribution also a maximum entropy distribution?

I am considering under what circumstances maximum entropy distributions are closed under marginalization. The main case I am interested is the following setting: a finite set of discrete random ...
0
votes
0answers
43 views

Sufficient Statistic and MLE

Suppose $X_1, \dots, X_n \sim B(1,p)$. Show that a sufficient statistic for $\theta = (1-p)^2$ is $T(x) = \sum X_i$ and that the MLE for $\theta$ is $(1-\frac{1}{n}T)^2$. I am having a lot of ...
0
votes
1answer
43 views

What is matrix norm in (Collins, 2001)

I am studying "A generalization of PCA to the exponential family" (Collins et al., 2001) and I don't understand some notations. What is the meaning of the matrix squared norm on page 6 ? Is it a ...
7
votes
1answer
221 views

Distribution of sum of independent exponentials with random number of summands

Let $\tau_i\sim\exp\left(\lambda\right)$ be independent and identically distributed exponentials with parameter $\lambda$. Then, for given $n$, the sum of these values $$T_n := \sum_{i=0}^n \tau_i$$ ...
2
votes
1answer
42 views

Expectation of Sufficient Statistic

Consider $X \sim B(n,p)$ with pmf $P(X=x) = {{n}\choose{x}} p^x (1-p)^{n-x}$. The general exponential form of an exponential family distribution is $p(x|\theta) = f(x) g(\theta) e^{\phi(\theta)^T T(...
1
vote
1answer
57 views

Gamma Distribution Sufficient Statistics

I've been asked to show the gamma distribution can be written in the form $p(x|\alpha, \beta) = f(x) g(\alpha, \beta) e^{h(\alpha,\beta)^T T(x)}$ where $T(x)$ is a sufficient statistic. .... I have ...
3
votes
0answers
31 views

Which properties yield the exponential family of distributions?

It seems like every resource that discusses exponential families simply defines the family of distributions, explains why it's useful and then derives some of its properties. I have only seen one ...
1
vote
1answer
30 views

Relationship between exponential families and moment generating functions

I have recently been playing around with some change of measure arguments for shifting the mean of a sub-gaussian distribution. It occurred to me however, that sub-gaussianity might not be the natural ...
0
votes
2answers
84 views

Independence of Beta ratios of Gamma variates

If $X= x_1/(x_1+x_2+x_3)$ and $Y= x_2/(x_1+x_2+x_3)$ where $x_1, x_2, x_3$ are independent $\chi^2$-distributed random variables with d.f. $-n_1,n_2, n_3$ respectively. Are $X$ and $Y$ independent? I ...
2
votes
0answers
41 views

Are the cumulants of sufficient statistics finite for the exponential family?

Under what conditions are the cumulants of the sufficient statistic finite for an exponential family? If we have $$ p(x \mid \theta) = \exp(\theta \cdot T(x) - A(\theta)) $$ then the derivatives of $...
0
votes
0answers
187 views

Gamma distribution as a member of exponential family

In my lecture notes I have that the distribution of a random variable $Y$ is said to be in the exponential family if it can be written as $f(y;\theta)=exp(a(y)b(\theta)+c(\theta)+d(y))$, where $a,b,c$ ...
3
votes
0answers
81 views

Finding the UMVUE of $\theta^2$ where $f_X(x\mid\theta) =\frac{x}{\theta^2}e^{-x/\theta}I_{(0,\infty)}(x)$

Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\frac{x}{\theta^2}e^{-x/\theta}I_{(0,\infty)}(x)$$ where $\theta >0$. Give the UMVUE of ${\theta^2}$ I ...
2
votes
1answer
145 views

Some questions about exponential families

Regarding the book The Bayesian Choice I understand most of chapter three on exponential families, but there are two parts I have trouble understanding. The first is Consider$$f(x|\theta)=h(x)\...
0
votes
0answers
19 views

Predict node attribute from network variables

ERGM models the probability of a tie forming in a network. Is there a way of using ERGM, or an equivalent model, where the response variable is an attribute of the node, not a tie? Basically, turning ...
1
vote
1answer
60 views

UMP Test of $H_0 : \theta \leq 0.1$ vs. $H_1 : \theta \gt 0.1$ for iid Geometric Random Variables

Consider $90$ iid geometric random variables having parameter $\theta$ (and thus mean $1/\theta$), and a UMP test of the null hypothesis that $\theta \leq 0.1$ against the alternative that $\theta ...
0
votes
1answer
226 views

Optimization using the optim function in R with a two parameter exponential distribution

I'm having trouble trying to optimize a two-parameter exponential distribution, by finding the maximum likelihood function and then using the function optim() in R ...
1
vote
1answer
45 views

Is this a member of an exponential family?

I strongly believe that this distribution does not belong to the exponential family: $f(x;\theta) = \frac{\theta}{2}^{|x|}(1 - \theta)^{1-|x|}I_{\{-1, 0, 1\}}(x)$. I have to write $f(x;\theta)$ as $...
2
votes
0answers
49 views

What are necessary & sufficient conditions for exponential family representation to have complete statistic $T(X)$?

My textbook gives the following theorem for exponential families: Let $X_1, \dots, X_n$ be a random sample from an exponential family with pmf/pdf of the form $$f(x|\theta) = h(x) c(\theta) \exp (w(...
5
votes
1answer
309 views

Find UMVUE of $\theta$ where $f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$

As a slight modification of my previous problem: Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$$ where $\...
6
votes
2answers
258 views

Finding UMVUE of $\theta e^{-\theta}$ where $X_i\sim\text{Pois}(\theta)$

Suppose $X_1, X_2, . . . , X_n$ are i.i.d Poisson ($\theta$) random variables, where $\theta\in(0,\infty)$. Give the UMVUE of $\theta e^{-\theta}$ I found a similar problem here. I have that the ...
9
votes
1answer
181 views

Find UMVUE of $\frac{1}{\theta}$ where $f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$

Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$$ where $\theta >0$. Give the UMVUE of $\frac{1}{\theta}$ ...
2
votes
1answer
133 views

Is the canonical parameter (and therefore the canonical link function) for a Gamma not unique?

Consider $Y_1, \dots, Y_n$ independent from the Gamma distribution. For $y > 0$: $$\begin{align} f(y \mid \alpha, \beta) &= \dfrac{1}{\beta^{\alpha}\Gamma(\alpha)}y^{\alpha-1}e^{-y/\beta} \\ &...
4
votes
1answer
242 views

Deriving the canonical link for a binomial distribution

I define an exponential dispersion family as any distribution whose PMF/PDF is $$f(y \mid \boldsymbol\theta) = \exp\left\{\phi[y\theta - b(\theta)] + c(y, \phi) \right\}\text{, } y \in \Omega$$ where ...
2
votes
0answers
23 views

Does any choice of sufficient statistic and natural parameter create valid exponential family pdfs?

So exponential families can be parameterized as $f(x|\theta) = h(x)e^{T(x)n(\theta)-A(n)}$. I'm trying to understand what the conditions are on $h(x)$, $T(x)$, and $n(\theta)$ are for $f(x|\theta)$ ...
0
votes
0answers
49 views

Support of an exponential family in canonical form

Suppose $Y$ is a random variable in the exponential family, with pmf/pdf $$f(y) = \exp\left[\sum_{j=1}^{s}\theta_jT_j(y)-B(\theta)+c(y) \right]$$ for $y \in \Omega$ (the support of $Y$), and where $$...
5
votes
1answer
74 views

Jointly sufficient statistics of a multi-parameter exponential family

Let $f_X$ be a joint density function that comes from an $s$-parameter exponential family with sufficient statistics $(T_1, T_2, \dots, T_s)$ so that the density $f_X$ can be expressed as $$f_{X|\...
4
votes
1answer
84 views

How many natural parameters are really in the exponential family conjugate prior?

The exponential family with natural parameter $\theta$ can be written $$ p(x|\theta)=h_\ell(x)\exp(\theta^Tt(x)-a_\ell(\theta)) $$ with conjugate prior $$ p(\theta|\lambda)=h_c(\theta)\exp(\lambda_1^T\...
1
vote
0answers
107 views

Weibull distribution is not a glm, why?

It is a generalized linear model with a linear combination of covariates related to the response via a canonical link function. Why weibull distribution is not a glm?
0
votes
0answers
34 views

Unsure about the distribution of the canonical statistic [duplicate]

Iam a bit unsure if the distribution of the canonical statistic below is correct ? I suspect i would need idenpendent variables to show the correct answer. Let $Y_1,Y_2,...Y_n$ be a sample form a ...
5
votes
1answer
120 views

Is the negative exponential distribution a member of the exponential family?

Please correct me if I am wrong. The general form of k-parameter exponential family is $f(x;\boldsymbol{\theta}) = a(\boldsymbol{\theta})g(x) \exp\{\sum_{i=1}^{k}b_(\boldsymbol{\theta}) R_i(x)\}$ ...
3
votes
1answer
61 views

Sampling parameters from exponential family

So suppose PDF $f_{X|\theta}(x_1,...,x_n;\theta_1,...,\theta_m)$ is from the exponential family. Is there any theory or general guidelines for sampling parameters from this PDF? This question is not ...
0
votes
0answers
394 views

Show the lognormal distribution belongs in the exponential family

I just wanted to verify that my attempt is correct. Thank you in advance for reading this. A distribution is said to belong in the exponential family if its probability density function can be ...
0
votes
0answers
17 views

How to model exponential decay with seasonality

I am trying to model sales driven by marketing tactics with promotional deadlines in Excel. The overall trend of the sales driven by each marketing tactic decays exponentially following the launch ...
0
votes
0answers
91 views

Convert exponential to Bernoulli

If I have a binary variable x, with distribution p(x) = exp{Cx}, how do I put this into the canonical Bernoulli form so as to get the probability p that x=1 that I ...
3
votes
2answers
106 views

intuitive interpretation of canonical parameterization of beta distribution

For exponential family, e.g. Beta distirbution, someone argues that the canonical parameterization is better than the traditional $Beta(\alpha,\beta)$ way. The canonical parameters are defined as $n^...
1
vote
0answers
27 views

Why do we care about maximum entropy? [duplicate]

One justification for the ubiquity of the (multivariate) normal distribution in statistical/machine learning modeling is that it maximizes entropy among distributions with mean $\mu$ and variance ...
1
vote
0answers
17 views

Tools for self-study: Constructing and understanding systems of distributional families

I am looking for a resource that probably does not exist, but, well, hope springs eternal. I have become increasingly interested in the process by which distributions are discovered or invented. ...
0
votes
0answers
156 views

Minimal sufficient statistic for multivariate normal

I have the following iid. variables $X_1,..,X_n,Y_1,..,Y_m$ with distribution $X_i\sim N(\mu_1,\sigma_1^2), Y_j\sim N(\mu_2,\sigma_2^2)$. How do I find the minimal sufficient statistic for $(\mu_1,\...
1
vote
0answers
48 views

Is my understanding of “family of distributions” correct?

As I was looking to understand the concept of "family of distributions", I stumbled upon this answer. However, I was a bit confused with answer and I'm hoping that someone may be able to clarify for ...
0
votes
0answers
48 views

Family use in glm in R for index's

I have a set of fluctuating asymmetry indices data sheets and would like to run a GLM using this index as the dependent variable and the other characteristics of the sheet as predictor variables (...
8
votes
1answer
142 views

Support vector machines (SVMs) are the zero temperature limit of logistic regression?

I was had a quick discussion recently with a knowledgeable friend who mentioned that SVMs are the zero temperature limit of logistic regression. The rationale involved marginal polytopes and fenchel ...
0
votes
2answers
137 views

When is a pdf not a member of the exponential family

The classic example given in class was when the support depends on the parameter $\theta$. If this is not the case, can the distribution always be written as a member of the exponential family?
0
votes
0answers
73 views

GLM: How to find the natural parameter when the response variable is under an arbitrary function

I'm attending General Linear Models this period. I'd like to know how do i find the natural parameter when $y$ isn't alone. Below is the exponential family form I'm working with: $\left(\frac{y_{i}\...
4
votes
1answer
54 views

property of the normalising constant in an exponential family

A statistical model for the data set $\bf{y}$ is an exponential family with canonical parameter vector $\theta = (\theta_1,.. \theta_k)$ and canonical statistic $\bf{t(y)} $=$(t_1(\boldsymbol{y}),.....
0
votes
1answer
62 views

MME for exponential family

Let $X_1, X_2,...,X_n$ be iid random variables having pdf $$f(x|\theta) = \frac{1}{x \sqrt{2\pi\theta}}e^{(-[\log x]^2/[2\theta])} I_{(0,\inf)}(x)$$ where $\theta > 0$. Determine the MME of $\theta$...
2
votes
1answer
415 views

Exponential Family with Dispersion Parameter Distributions

Defining the exponential dispersion family by $exp\left(\frac{x_{i}\theta - b(\theta_{i})}{\phi} + c(x_{i}, \phi, w_{i})\right)$ I'd like to change the usual Inverse-Gaussian density below to the ...
3
votes
2answers
88 views

What are the properties of the “unfolded” gamma distribution generalization of a normal distribution?

In a prior post, I developed an "unfolded" gamma distribution generalization of a normal distribution as an example of how to relate a gamma distribution to a normal distribution. This yielded $$ \...
0
votes
1answer
45 views

Mean of Exponential Families

Show that E{$e^{\delta T(Y)}|\eta$} = $e^{A(\eta + \delta) - A(\eta)}$. Then, prove that E{$T(Y)|\eta$} = $\frac{\partial A(\eta)}{\partial \eta}$. First, we can begin by recognizing the fact that a ...