Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
0
votes
0answers
66 views

What is the log-normalizer of the conjugate prior of an exponential family?

Let's say that you have a distribution $F$ in the exponential family with density \begin{align} \newcommand{\mbx}{\mathbf x} \newcommand{\btheta}{\boldsymbol{\theta}} f(\mbx \mid \btheta) &= \exp\...
1
vote
1answer
29 views

Linear relations between statistics in exponential family of distributions

I am reading about point estimation from Theory of Point Estimation by Lehmann and Casella (1999). I couldn't understand the following point mentioned in p.24, under the exponential family of ...
2
votes
0answers
60 views

Mean and variance of the Beta distribution using identities of exponential families

I was studying the part of exponential families from Statistical Inference (George Casella, Roger L. Berger) and they give the following definition of an exponential family: $$ f(x|\pmb{\theta}) = h(x)...
4
votes
2answers
199 views

Distribution of the sufficient statistic in the exponential family?

Suppose $\boldsymbol X$ belongs to the exponential family, $$ f_X\!\left(\,\mathbf{x} ; \boldsymbol \theta\,\right) = h(\mathbf{x}) \, \exp\!\Big(\,\boldsymbol\eta({\boldsymbol \theta}) \cdot \mathbf{...
0
votes
0answers
79 views

how to choose family in CVgam in r?

I'm trying to run a repeated k-fold cross validation of my GAM model, but I can't specify my family in CVgam in R. I want to compare poisson GAM and Negative binomial GAM. How can I solve this ...
1
vote
0answers
25 views

Calculating the conditional expectation of an exponential family [closed]

If we have $X$ with a density depending on the scalar parameter $\theta$, where the density is from of the exponential family: $f(x;\theta) = \exp(\theta x−\phi(\theta))h(x)$. Also we have that $\...
0
votes
0answers
25 views

Examples of functions in generalized linear models

Let $X \in {R}^{n\times p}, Y \in R^{n}$ and $\beta \in R^p$, I am trying to find examples of generalized linear models where the posterior $g_{\beta|y} \propto \exp(-f(y;X,\beta) - \lambda\|\beta\|_2^...
0
votes
0answers
127 views

Normal Conjugate Prior, Known Mean and Unknown Variance?

For Normal distribution, with know mean and unknown variance. When $\tau = 1/\sigma^2$ ~ Gamma(). In such has posterior of $\tau$ has the following distribution: $p(\tau|\alpha, \beta, x) \sim G(\...
2
votes
1answer
71 views

Reparametrization and its effect on sufficient/complete/minimal statistics

Suppose $X_1 \sim Pois(\lambda_1), X_2 \sim Pois(\lambda_2), X_3 \sim Pois(\lambda_1+\lambda_2)$. Separately I can find a sufficient, complete and minimal statistic for each of them. But considering ...
0
votes
0answers
72 views

Can I build deviance residuals from an XGBoost model that learns an exponential family parameter?

I'm taking a course on GLMs after a few years of using machine learning models. The good about GLMs is how the probabilistic model ties in with the estimation and evaluation. So I'm trying to transfer ...
1
vote
0answers
129 views

PDF of Beta Distribution written as exponential family form

I am trying to write the pdf of a beta random variable in its biparametric canonic form such as: Function 1 $$ f_Y(y; \theta, \phi) = exp \{ \phi[y \theta - b(\theta)] + c(y, \phi) \} \mathbb{1}_A(...
0
votes
0answers
12 views

Interpretation of concentration of posteriors in the limit of infinitely many independent versus dependent random variables

Disclaimer: the setup and specific example may not be a minimal example to illustrate the point, but I am not well-versed in these topics enough to construct a smaller example without accidentally ...
0
votes
0answers
34 views

Reparameterizing PDF to get components of exponential family

For the following problem, I am trying to identify the components of the exponential family in the form: $\exp(y\theta - b(\theta))/a(\phi) + c(y; \phi)$ Namely, I need to identify the $\theta, b(\...
0
votes
0answers
434 views

Extreme Value (Gumbel) distribution a member of exponential family

This is a question for discussion in my Linear Model class. I am having a hard time showing that the distribution belongs to the exponential family PDF: $f(y; \theta) = 1/\varphi \exp([y − \theta]/\...
2
votes
0answers
34 views

GLMM and Exponential Family

I am studying now GLMMs (Generalized Linear Mixed Models). From my understanding, in order to estimate the parameters of this model, you need to arrive at the marginal probability by integrating a ...
1
vote
1answer
108 views

Find the prior distribution for the natural parameter of an exponential family

Show that for the binomial likelihood $y$ ~$Bin(n, \theta)$, $p(\theta) \propto \theta^{-1} (1-\theta)^{-1}$ is the uniform prior distribution for the natural parameter of the exponential family. I am ...
2
votes
2answers
750 views

Does Linear regression needs target variable to be normally distributed. (GLM context)?

I came across the assumptions of linear regression that said: -->The residuals should be normally distributed. GLM(Generalized Linear model) assumes that target variable should follow one of the ...
3
votes
2answers
131 views

If Y has an exponential family distribution show that $E(\frac{\partial L}{\partial \theta}) = 0$

I'm working in a self study fashion preparing for a course I'm going to take this semester in generalised linear models. The question is, given that the Y random variable belongs to the exponential ...
2
votes
1answer
115 views

Is a maximum likelihood estimator in an exponential family always sufficient?

An exponential family (under natural parameterization) is such that $p(X|\eta)=h(X)\exp\{\eta^TT(X)-A(\eta)\}$, where $X$ is the data, $\eta$ is the natural parameter, and $h,T,A$ are some functions (...
0
votes
0answers
22 views

History of the curved exponential family

Does anyone know the first person who introduced the curved exponential family and also which paper it was first presented? I vaguely remember that it might be Fisher who wrote about it in a paper on ...
3
votes
1answer
219 views

Conjugate priors outside exponential family

The usual exception I have come across regarding non-existence of conjugate prior outside the exponential family is the uniform distribution on $(0,\theta)$ (i.e. $U(0,\theta)$) where $\theta$ has a ...
2
votes
1answer
94 views

How do I find the MLE of the APE distribution in R?

The random variable $Y$ is said to have a two-parameter APE distribution, denoted by $\text{APE}(\alpha, \lambda)$, with the shape parameter $\alpha>0$ and scale parameter $\lambda>0$ if the ...
1
vote
2answers
95 views

Bayesian inference with an arbitrary prior

A classical problem in Bayesian inference arises when we wish to learn about (say) the fraction $\theta$ of balls in an urn that are white; and do so by sampling from the urn with replacement. In such ...
5
votes
0answers
265 views

Let $X_1,X_2,\dots,X_n$ be random sample from Poisson($\theta$). Find MVUE of $e^{-2\theta}$

Question: Let $X_1,X_2,\dots,X_n$ be random sample from Poisson($\theta$). Find MVUE of $e^{-2\theta}$ My attempt has been by modifying the answer from this question: The Poisson distribution is a one-...
0
votes
0answers
108 views

UMVUE of Bernoulli random variables

Let $X_1, X_2..... X_n$ be a random sample from a Bernoulli population with parameter $p$. A sufficient statistic is $\sum_{i=1}^{n}X_i$. If we define $$ U(X_1,X_2,\ldots,X_n)= \begin{cases}1/2n &\...
0
votes
0answers
23 views

What is the conjugate prior for the hypoexponential distribution?

Can't find it anywhere. I know Gamma is the conjugate prior for the exponential distribution (one parameter) but for the sum of exponential distributions (the hypoexponential distribution), I can't ...
7
votes
2answers
482 views

Intuition for why the (log) partition function matters?

I'm on a quest for the intuition behind the fact that theoretical introductions to approximate inference focus so much on the log partition function. Say we have a regular exponential family $$p(\...
2
votes
0answers
33 views

Why does the canonical parameter give a link function? Why does this relate $E[Y]$ to $x^T \beta$?

If I have a pdf in the form $f(y|\theta,\phi)=\text{exp}\bigg(\frac{y\theta-b(\theta)}{a(\phi)}+c(y,\phi)\bigg)$, then $\theta$ is called the canonical parameter. I'm told we can get a link function $...
3
votes
1answer
79 views

Saddlepoint approximation for Exponential family

I read the following in a book: The saddlepoint approximation of an exponential family density function is $$\tilde P(y;\mu,\phi) = \frac{1}{\sqrt{2\pi \phi V(y)}}exp(-\frac{d(y, \mu)}{2\phi})$$ Where ...
1
vote
1answer
117 views

n'th cumulant (of a CGF) for exponential family / exponential dispersion model

The n'th cumulant is defined to be the n'th derivative of the CGF (cumulant generating function). $$\kappa_n = \frac{d^n K(t)}{dt^n} |_{t=0} $$ But I'm reading in a book (p.215, chapter5, eq. 5.8) ...
0
votes
1answer
49 views

marginal conditional from joint of three r.v.'s

I have two random variables, $X$ and $Y$. I know that $X \sim \text{Gamma}(a,b)$ and $Y \sim \text{Gamma}(c,d)$. Furthermore, I know that $Z \sim \text{Poisson}(XY)$. I know the joint distribution ...
7
votes
1answer
222 views

How do I find the UMVUE of $\sqrt{\alpha}$ here?

new user here self-studying some mathematical statistics. I came across this problem and am stuck. Problem: Suppose that for $i = 1, ... , n$, the positive random variables $X_i$ are independent and ...
1
vote
0answers
26 views

Show bivariate normal distribution with non-diagonal covariance belongs to curved exponential family?

Question: Suppose that $(X_{i}, Y_{i})$, $i = 1, \dots ,n$ are sampled i.i.d. from the two-dimensional normal distribution $$ \begin{bmatrix} X & Y \end{bmatrix} \sim \mathcal{N}\left( \begin{...
1
vote
1answer
62 views

Exponential families and Gibbs measures? Any relation?

The title says it all. The formulas for exponential families and Gibbs measures seems very similar. Is there any relationship, or some kind of translation table?
1
vote
1answer
176 views

Prove that argmin of exponential distributions has multinomial distribution [closed]

Let's say we have $ T_1,T_2,\cdots,T_n \sim Exp$ and $P(X_1>a)=e^{-\lambda_1 a},P(X_2>a)=e^{-\lambda_2 a},\cdots,P(X_n>a)=e^{-\lambda_n a} $. How can I describe $\DeclareMathOperator*...
1
vote
1answer
77 views

Definition of $k$-parameter exponential family

I am currently studying the concept of sufficient statistics in mathematical statistics. The following definition is presented: Definition: $k$-parameter exponential family Let $\mathbf{Y} \sim f_\...
0
votes
0answers
208 views

Division of Multivariate Normal Distributions

$$ \newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\nc}[2]{\newcommand{#1}{#2}} \nc{\vx}{\vect{x}} \nc{\vmu}{\vect{\mu}} \nc{\vSigma}{\vect{\Sigma}} \nc{\vtheta}{\vect{\theta}} $$ ...
0
votes
0answers
250 views

Bayesian Linear Regression and the Exponential Family

In a straight forward linear regression model, assuming a fixed input $\mathbf{x}$, and additive noise with unit variance we can write: \begin{equation} p(y\mid \mathbf{x,w})=\frac{1}{\sqrt{2\pi}\...
0
votes
2answers
130 views

What is the idea behind generalized linear models?

I am watching andrew ng's video lectures on machine learning. I am trying to understand what is even the point of generalized linear models. I understand what goes on step by step in deriving things ...
1
vote
0answers
50 views

Is there just one exponential family? Or are there many exponential families? [duplicate]

I'm confused by the phrasing I've seen about exponential families. What does it mean to say "an" exponential family. Why not "the" exponential family? From a pdf from Berkely: "we define an ...
3
votes
0answers
112 views

Are there nonparametric generative models for datasets?

Typically when I see generative models, e.g., Latent Dirichlet Allocation (JMLR) or Linear/Quadratic Discriminant Analysis (wikipedia LDA), they are probabilistic models that belong to the exponential ...
1
vote
1answer
123 views

Finding Uniformly Most Powerful test

My Attempt Comparing $f(x;\theta)$ with the form $a(\theta)b(x)exp[c(\theta)d(x)]$ , we get $d(x) = log (1-x)$ and $ c(\theta ) = \theta -1 $ as monotone , increasing function in $\theta$ and ...
3
votes
1answer
589 views

explanation of why an UMVUE doesn't necessarily have to achieve the CRLB?

I'm studying uniformly minimum variance unbiased estimator(UMVUE). I have seen question on this site asking why the UMVUE doesn't achieve the CRLB(Cramer Rao lower bound), and all of the answers have ...
2
votes
0answers
31 views

Show Moment estimator and MLE are the same only for exponential family

Show Moment estimator and MLE are the same only for exponential family. I know MM is variant while MLE is not under transformation. Will this be the starting point?
1
vote
1answer
75 views

Derivation of the Objective Function for Expectation Propagation

I was reading Expectation Propagation As A Way Of Life and the original paper by Minka Expectation Propagation for Approximate Bayesian Inference and they both say that a fixed point of the EP ...
0
votes
0answers
109 views

Exponential family and efficient estimator

In my lecture notes there is the notion of efficiency related to the exponential family. More precisely, the lecturer stated that for an exponential family an efficient estimator always exists. How is ...
1
vote
2answers
305 views

Connection between subgaussian/subexponential and exponential family

I am wondering if there is any relationship between subgaussian/subexponential with (one parameter) exponential family. In particular, is there any sub-family density that belongs to both ...
6
votes
1answer
591 views

Is the t distribution a member of the exponential family?

From what I understand, the exponential family is defined as $$f(y;\theta,\phi) = \exp\left(\frac{y\theta - b(\theta)}{a(\phi)}+c(y,\phi)\right) $$ I've read (but not seen shown anywhere), that the ...
0
votes
1answer
58 views

How to derive a pdf of Complete Sufficient Statistic of exponential family

While studying Mathematical statistics through "Introduction to Mathematical Statistics 7th" (by Hogg and Craig), I've been stuck in the Theorem above. The answer of the exercise 7.5.8 is not given in ...
1
vote
1answer
177 views

Is the Erlang Distribution member of the Exponential family?

Given the Erlang distribution $f(x)={\begin{cases}\displaystyle {\frac {\lambda ^{n}x^{{n-1}}}{(n-1)!}}\,{\mathrm {e}}^{{-\lambda x}}&x\geq 0\\0&x<0\end{cases}}$ I want to determine, ...

1
2
3 4 5
7