Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

0
votes
0answers
11 views

How to specify a scaled-t family in brms (R)?

I built a model with the gam function from MGCV using the scat family because my Y variable is heavily tailed (https://rdrr.io/cran/mgcv/man/scat.html). The residuals of the model are fine, and almost ...
0
votes
0answers
18 views

Double exponential family

The double exponential family is defined by $\bar{f}(y;\mu,\theta) =c(\mu,\theta)\theta^\frac{1}{2}e_Y(y;\mu)^\theta e_Y(y;y)^{1-\theta} $ For the double Poisson distribution, several papers state ...
1
vote
1answer
39 views

Is this a member of an exponential family?

I strongly believe that this distribution does not belong to the exponential family: $f(x;\theta) = \frac{\theta}{2}^{|x|}(1 - \theta)^{1-|x|}I_{\{-1, 0, 1\}}(x)$. I have to write $f(x;\theta)$ as $...
2
votes
0answers
17 views

What are necessary & sufficient conditions for exponential family representation to have complete statistic $T(X)$?

My textbook gives the following theorem for exponential families: Let $X_1, \dots, X_n$ be a random sample from an exponential family with pmf/pdf of the form $$f(x|\theta) = h(x) c(\theta) \exp (w(...
4
votes
1answer
100 views

Find UMVUE of $\theta$ where $f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$

As a slight modification of my previous problem: Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$$ where $\...
4
votes
2answers
93 views

Finding UMVUE of $\theta e^{-\theta}$ where $X_i\sim\text{Pois}(\theta)$

Suppose $X_1, X_2, . . . , X_n$ are i.i.d Poisson ($\theta$) random variables, where $\theta\in(0,\infty)$. Give the UMVUE of $\theta e^{-\theta}$ I found a similar problem here. I have that the ...
7
votes
1answer
108 views

Find UMVUE of $\frac{1}{\theta}$ where $f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$

Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf $$f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$$ where $\theta >0$. Give the UMVUE of $\frac{1}{\theta}$ ...
2
votes
1answer
48 views

Is the canonical parameter (and therefore the canonical link function) for a Gamma not unique?

Consider $Y_1, \dots, Y_n$ independent from the Gamma distribution. For $y > 0$: $$\begin{align} f(y \mid \alpha, \beta) &= \dfrac{1}{\beta^{\alpha}\Gamma(\alpha)}y^{\alpha-1}e^{-y/\beta} \\ &...
4
votes
1answer
131 views

Deriving the canonical link for a binomial distribution

I define an exponential dispersion family as any distribution whose PMF/PDF is $$f(y \mid \boldsymbol\theta) = \exp\left\{\phi[y\theta - b(\theta)] + c(y, \phi) \right\}\text{, } y \in \Omega$$ where ...
2
votes
0answers
22 views

Does any choice of sufficient statistic and natural parameter create valid exponential family pdfs?

So exponential families can be parameterized as $f(x|\theta) = h(x)e^{T(x)n(\theta)-A(n)}$. I'm trying to understand what the conditions are on $h(x)$, $T(x)$, and $n(\theta)$ are for $f(x|\theta)$ ...
0
votes
0answers
23 views

Support of an exponential family in canonical form

Suppose $Y$ is a random variable in the exponential family, with pmf/pdf $$f(y) = \exp\left[\sum_{j=1}^{s}\theta_jT_j(y)-B(\theta)+c(y) \right]$$ for $y \in \Omega$ (the support of $Y$), and where $$...
5
votes
1answer
39 views

Jointly sufficient statistics of a multi-parameter exponential family

Let $f_X$ be a joint density function that comes from an $s$-parameter exponential family with sufficient statistics $(T_1, T_2, \dots, T_s)$ so that the density $f_X$ can be expressed as $$f_{X|\...
4
votes
1answer
46 views

How many natural parameters are really in the exponential family conjugate prior?

The exponential family with natural parameter $\theta$ can be written $$ p(x|\theta)=h_\ell(x)\exp(\theta^Tt(x)-a_\ell(\theta)) $$ with conjugate prior $$ p(\theta|\lambda)=h_c(\theta)\exp(\lambda_1^T\...
1
vote
0answers
59 views

Weibull distribution is not a glm, why?

It is a generalized linear model with a linear combination of covariates related to the response via a canonical link function. Why weibull distribution is not a glm?
0
votes
0answers
32 views

Unsure about the distribution of the canonical statistic [duplicate]

Iam a bit unsure if the distribution of the canonical statistic below is correct ? I suspect i would need idenpendent variables to show the correct answer. Let $Y_1,Y_2,...Y_n$ be a sample form a ...
5
votes
1answer
58 views

Is the negative exponential distribution a member of the exponential family?

Please correct me if I am wrong. The general form of k-parameter exponential family is $f(x;\boldsymbol{\theta}) = a(\boldsymbol{\theta})g(x) \exp\{\sum_{i=1}^{k}b_(\boldsymbol{\theta}) R_i(x)\}$ ...
3
votes
1answer
53 views

Sampling parameters from exponential family

So suppose PDF $f_{X|\theta}(x_1,...,x_n;\theta_1,...,\theta_m)$ is from the exponential family. Is there any theory or general guidelines for sampling parameters from this PDF? This question is not ...
0
votes
0answers
189 views

Show the lognormal distribution belongs in the exponential family

I just wanted to verify that my attempt is correct. Thank you in advance for reading this. A distribution is said to belong in the exponential family if its probability density function can be ...
0
votes
0answers
9 views

How to model exponential decay with seasonality

I am trying to model sales driven by marketing tactics with promotional deadlines in Excel. The overall trend of the sales driven by each marketing tactic decays exponentially following the launch ...
0
votes
0answers
51 views

Convert exponential to Bernoulli

If I have a binary variable x, with distribution p(x) = exp{Cx}, how do I put this into the canonical Bernoulli form so as to get the probability p that x=1 that I ...
3
votes
2answers
56 views

intuitive interpretation of canonical parameterization of beta distribution

For exponential family, e.g. Beta distirbution, someone argues that the canonical parameterization is better than the traditional $Beta(\alpha,\beta)$ way. The canonical parameters are defined as $n^...
1
vote
0answers
26 views

Why do we care about maximum entropy? [duplicate]

One justification for the ubiquity of the (multivariate) normal distribution in statistical/machine learning modeling is that it maximizes entropy among distributions with mean $\mu$ and variance ...
1
vote
0answers
15 views

Tools for self-study: Constructing and understanding systems of distributional families

I am looking for a resource that probably does not exist, but, well, hope springs eternal. I have become increasingly interested in the process by which distributions are discovered or invented. ...
0
votes
0answers
106 views

Minimal sufficient statistic for multivariate normal

I have the following iid. variables $X_1,..,X_n,Y_1,..,Y_m$ with distribution $X_i\sim N(\mu_1,\sigma_1^2), Y_j\sim N(\mu_2,\sigma_2^2)$. How do I find the minimal sufficient statistic for $(\mu_1,\...
1
vote
0answers
45 views

Is my understanding of “family of distributions” correct?

As I was looking to understand the concept of "family of distributions", I stumbled upon this answer. However, I was a bit confused with answer and I'm hoping that someone may be able to clarify for ...
0
votes
0answers
32 views

Family use in glm in R for index's

I have a set of fluctuating asymmetry indices data sheets and would like to run a GLM using this index as the dependent variable and the other characteristics of the sheet as predictor variables (...
8
votes
1answer
135 views

Support vector machines (SVMs) are the zero temperature limit of logistic regression?

I was had a quick discussion recently with a knowledgeable friend who mentioned that SVMs are the zero temperature limit of logistic regression. The rationale involved marginal polytopes and fenchel ...
0
votes
2answers
84 views

When is a pdf not a member of the exponential family

The classic example given in class was when the support depends on the parameter $\theta$. If this is not the case, can the distribution always be written as a member of the exponential family?
0
votes
0answers
58 views

GLM: How to find the natural parameter when the response variable is under an arbitrary function

I'm attending General Linear Models this period. I'd like to know how do i find the natural parameter when $y$ isn't alone. Below is the exponential family form I'm working with: $\left(\frac{y_{i}\...
4
votes
1answer
50 views

property of the normalising constant in an exponential family

A statistical model for the data set $\bf{y}$ is an exponential family with canonical parameter vector $\theta = (\theta_1,.. \theta_k)$ and canonical statistic $\bf{t(y)} $=$(t_1(\boldsymbol{y}),.....
0
votes
1answer
48 views

MME for exponential family

Let $X_1, X_2,...,X_n$ be iid random variables having pdf $$f(x|\theta) = \frac{1}{x \sqrt{2\pi\theta}}e^{(-[\log x]^2/[2\theta])} I_{(0,\inf)}(x)$$ where $\theta > 0$. Determine the MME of $\theta$...
2
votes
1answer
209 views

Exponential Family with Dispersion Parameter Distributions

Defining the exponential dispersion family by $exp\left(\frac{x_{i}\theta - b(\theta_{i})}{\phi} + c(x_{i}, \phi, w_{i})\right)$ I'd like to change the usual Inverse-Gaussian density below to the ...
3
votes
2answers
83 views

What are the properties of the “unfolded” gamma distribution generalization of a normal distribution?

In a prior post, I developed an "unfolded" gamma distribution generalization of a normal distribution as an example of how to relate a gamma distribution to a normal distribution. This yielded $$ \...
0
votes
1answer
44 views

Mean of Exponential Families

Show that E{$e^{\delta T(Y)}|\eta$} = $e^{A(\eta + \delta) - A(\eta)}$. Then, prove that E{$T(Y)|\eta$} = $\frac{\partial A(\eta)}{\partial \eta}$. First, we can begin by recognizing the fact that a ...
6
votes
4answers
591 views

What is the rationale behind the exponential family of distributions?

From elementary probability course, the probability distributions such as Gaussian, Poisson or exponential all have a good motivation. After staring at the formula of the exponential family ...
2
votes
0answers
40 views

Tweedie index parameters are restricted: why?

The Tweedie distribution have variance like var(y) = $\phi \mu^p$ for any real p not between 0 and 1. I read in many places (even wikipedia) that p can take any real value except between 0 and 1. ...
2
votes
0answers
62 views

Parametric family problems

I came across such a problem that I cannot solve: Let $\mathcal{P} = \{\mathbb{P}_\theta : \theta \in \mathbb{R}\}$ be a parametric family over $\{0,1\} \times \mathbb{R}$ defined in the following ...
2
votes
1answer
202 views

Maximum Likelihood Estimation on Exponential Family

Does any one know of a good source for the study of maximum likelihood estimation on a general exponential family of the form $f(x;\theta)=a(\theta)g(x)\exp[{\sum_{i=1}^{k} b_i(\theta)R_i(x)}]$? Any ...
12
votes
3answers
523 views

Definition of family of a distribution?

Does a family of a distribution have a different definition for statistics than in other disciplines? In general, a family of curves is a set of curves, each of which is given by a function or ...
1
vote
0answers
22 views

Does sufficiency implies the reference measure to be a function of the sufficient statistic in exponential family?

Given the definition of exponential family $$ f(x\mid\theta) = \exp\left( \eta(\theta)^\top T(x) + h(x) - A(\theta) \right) $$ for $\theta\in\Theta$ and $x\in\mathcal{X}$, ...
0
votes
0answers
154 views

Show that Sufficient statistic is complete

I have the following Exponential distribution: $$f_\theta(x) = \theta^2x e^{-\theta x}\mathbb{1}_{[0,+\infty[}(x) $$ I am asked the two following questions: ...
0
votes
0answers
125 views

Hidden Markov Models (HMM) with exponential family distributions

HMM learning with Baum Welch (EM) is well known, but the parameterization is all the elements in the transition and emission matrices $a_{ij}, b_i(k)$, that is the number of parameters is $\mathcal{O}(...
0
votes
1answer
160 views

Moment generating function of the natural sufficient statistics of Gamma distribution

I have $X_1,X_2 ... X_n$ Gamma-distributed r.v with density: First of all I showed, that Gamma distribution belongs to exponential family and can be represented in form I found, that for Gamma ...
4
votes
1answer
232 views

Does exponential family of distributions have finite expected value?

I am curious about this question, because in definitions I have never seen this property. Is it true? If yes, why?
1
vote
1answer
189 views

Relationship between MLE and Pitman–Koopman–Darmois Theorem

Is there a relationship between the applicability of Maximum Likelihood Estimation and Pitman-Koopman-Darmois Theorem? I mean if the dimensionality of the sufficient statistics depend on the sample ...
5
votes
1answer
240 views

Skewness of Tweedie distribution

Tweedie distributions are a family of distributions from the exponential dispersion family that have power-law mean-variance relationship: \begin{align} \mathbb E[X] &= \mu \\ \operatorname{Var}[...
3
votes
1answer
97 views

How can GAMLSS relax the GLM exponential family assumption?

Generalized Additive Models for Location, Scale and Shape "relax the assumption of exponential families" in comparison to GLM's or GAM's. This is a direct quote from the paper by Rigby and ...
1
vote
1answer
108 views

Exponential Twisting

I am with a difficult to prove the next relation about exponential twisting. According to Monte Carlo Methods and Models in Finance and Insurance by Ralf Korn, Elke Korn, Gerald Kroisandt. The ...
4
votes
1answer
69 views

Can we absorb the partition function into the natural parameter vector?

Given an exponential family distribution of the form $$ f_X(x)=h(x)e^{\phi(\theta)^TT(x)-A(\theta)} $$ with natural parameter $\eta=\phi(\theta)$, sufficient statistic $T(x)$, and log partition ...
3
votes
1answer
259 views

Is an exponential family closed under convolution?

Let $$ y = x_1 + x_2 + x_3 $$ where $x_1, x_2, x_3$ are draws of random variables of the same family and different parameters: $p(x_1| \theta_1), p(x_2| \theta_2), p(x_3| \theta_3)$ We know that, ...