Questions tagged [exponential-family]

A set of distributions (eg, normal, $\chi^2$, Poisson, etc) that share a specific form. Many of the distributions in the exponential family are standard, workhorse distributions in statistics, w/ convenient statistical properties.

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

Normal Conjugate Prior, Known Mean and Unknown Variance?

For Normal distribution, with know mean and unknown variance. When $\tau = 1/\sigma^2$ ~ Gamma(). In such has posterior of $\tau$ has the following distribution: $p(\tau|\alpha, \beta, x) \sim G(\...
2
votes
1answer
22 views

Reparametrization and its effect on sufficient/complete/minimal statistics

Suppose $X_1 \sim Pois(\lambda_1), X_2 \sim Pois(\lambda_2), X_3 \sim Pois(\lambda_1+\lambda_2)$. Separately I can find a sufficient, complete and minimal statistic for each of them. But considering ...
0
votes
0answers
13 views

Can I build deviance residuals from an XGBoost model that learns an exponential family parameter?

I'm taking a course on GLMs after a few years of using machine learning models. The good about GLMs is how the probabilistic model ties in with the estimation and evaluation. So I'm trying to transfer ...
1
vote
0answers
25 views

PDF of Beta Distribution written as exponential family form

I am trying to write the pdf of a beta random variable in its biparametric canonic form such as: Function 1 $$ f_Y(y; \theta, \phi) = exp \{ \phi[y \theta - b(\theta)] + c(y, \phi) \} \mathbb{1}_A(...
0
votes
0answers
9 views

Interpretation of concentration of posteriors in the limit of infinitely many independent versus dependent random variables

Disclaimer: the setup and specific example may not be a minimal example to illustrate the point, but I am not well-versed in these topics enough to construct a smaller example without accidentally ...
0
votes
0answers
18 views

Reparameterizing PDF to get components of exponential family

For the following problem, I am trying to identify the components of the exponential family in the form: $\exp(y\theta - b(\theta))/a(\phi) + c(y; \phi)$ Namely, I need to identify the $\theta, b(\...
0
votes
0answers
42 views

Extreme Value (Gumbel) distribution a member of exponential family

This is a question for discussion in my Linear Model class. I am having a hard time showing that the distribution belongs to the exponential family PDF: $f(y; \theta) = 1/\varphi \exp([y − \theta]/\...
2
votes
0answers
19 views

GLMM and Exponential Family

I am studying now GLMMs (Generalized Linear Mixed Models). From my understanding, in order to estimate the parameters of this model, you need to arrive at the marginal probability by integrating a ...
1
vote
1answer
41 views

Find the prior distribution for the natural parameter of an exponential family

Show that for the binomial likelihood $y$ ~$Bin(n, \theta)$, $p(\theta) \propto \theta^{-1} (1-\theta)^{-1}$ is the uniform prior distribution for the natural parameter of the exponential family. I am ...
1
vote
2answers
64 views

Does Linear regression needs target variable to be normally distributed. (GLM context)?

I came across the assumptions of linear regression that said: -->The residuals should be normally distributed. GLM(Generalized Linear model) assumes that target variable should follow one of the ...
1
vote
2answers
62 views

If Y has an exponential family distribution show that $E(\frac{\partial L}{\partial \theta}) = 0$

I'm working in a self study fashion preparing for a course I'm going to take this semester in generalised linear models. The question is, given that the Y random variable belongs to the exponential ...
2
votes
1answer
49 views

Is a maximum likelihood estimator in an exponential family always sufficient?

An exponential family (under natural parameterization) is such that $p(X|\eta)=h(X)\exp\{\eta^TT(X)-A(\eta)\}$, where $X$ is the data, $\eta$ is the natural parameter, and $h,T,A$ are some functions (...
0
votes
0answers
18 views

History of the curved exponential family

Does anyone know the first person who introduced the curved exponential family and also which paper it was first presented? I vaguely remember that it might be Fisher who wrote about it in a paper on ...
3
votes
1answer
101 views

Conjugate priors outside exponential family

The usual exception I have come across regarding non-existence of conjugate prior outside the exponential family is the uniform distribution on $(0,\theta)$ (i.e. $U(0,\theta)$) where $\theta$ has a ...
2
votes
1answer
76 views

How do I find the MLE of the APE distribution in R?

The random variable $Y$ is said to have a two-parameter APE distribution, denoted by $\text{APE}(\alpha, \lambda)$, with the shape parameter $\alpha>0$ and scale parameter $\lambda>0$ if the ...
1
vote
2answers
72 views

Bayesian inference with an arbitrary prior

A classical problem in Bayesian inference arises when we wish to learn about (say) the fraction $\theta$ of balls in an urn that are white; and do so by sampling from the urn with replacement. In such ...
5
votes
0answers
85 views

Let $X_1,X_2,\dots,X_n$ be random sample from Poisson($\theta$). Find MVUE of $e^{-2\theta}$

Question: Let $X_1,X_2,\dots,X_n$ be random sample from Poisson($\theta$). Find MVUE of $e^{-2\theta}$ My attempt has been by modifying the answer from this question: The Poisson distribution is a one-...
0
votes
0answers
51 views

UMVUE of Bernoulli random variables

Let $X_1, X_2..... X_n$ be a random sample from a Bernoulli population with parameter $p$. A sufficient statistic is $\sum_{i=1}^{n}X_i$. If we define $$ U(X_1,X_2,\ldots,X_n)= \begin{cases}1/2n &\...
0
votes
0answers
15 views

What is the conjugate prior for the hypoexponential distribution?

Can't find it anywhere. I know Gamma is the conjugate prior for the exponential distribution (one parameter) but for the sum of exponential distributions (the hypoexponential distribution), I can't ...
0
votes
0answers
8 views

Count Metric Inter-arrival Times

I have a process that I would like to simulate. I have count data for the process in minutes (ie 60/minute). Because I do not have access to more granular data, I cannot model the interarrival times ...
6
votes
2answers
113 views

Intuition for why the (log) partition function matters?

I'm on a quest for the intuition behind the fact that theoretical introductions to approximate inference focus so much on the log partition function. Say we have a regular exponential family $$p(\...
2
votes
0answers
29 views

Why does the canonical parameter give a link function? Why does this relate $E[Y]$ to $x^T \beta$?

If I have a pdf in the form $f(y|\theta,\phi)=\text{exp}\bigg(\frac{y\theta-b(\theta)}{a(\phi)}+c(y,\phi)\bigg)$, then $\theta$ is called the canonical parameter. I'm told we can get a link function $...
3
votes
1answer
50 views

Saddlepoint approximation for Exponential family

I read the following in a book: The saddlepoint approximation of an exponential family density function is $$\tilde P(y;\mu,\phi) = \frac{1}{\sqrt{2\pi \phi V(y)}}exp(-\frac{d(y, \mu)}{2\phi})$$ Where ...
1
vote
1answer
33 views

n'th cumulant (of a CGF) for exponential family / exponential dispersion model

The n'th cumulant is defined to be the n'th derivative of the CGF (cumulant generating function). $$\kappa_n = \frac{d^n K(t)}{dt^n} |_{t=0} $$ But I'm reading in a book (p.215, chapter5, eq. 5.8) ...
0
votes
1answer
45 views

marginal conditional from joint of three r.v.'s

I have two random variables, $X$ and $Y$. I know that $X \sim \text{Gamma}(a,b)$ and $Y \sim \text{Gamma}(c,d)$. Furthermore, I know that $Z \sim \text{Poisson}(XY)$. I know the joint distribution ...
7
votes
1answer
106 views

How do I find the UMVUE of $\sqrt{\alpha}$ here?

new user here self-studying some mathematical statistics. I came across this problem and am stuck. Problem: Suppose that for $i = 1, ... , n$, the positive random variables $X_i$ are independent and ...
0
votes
0answers
20 views

Differentiating the Log-Normalizer of the Gamma distribution

Converting the Gamma distribution to the Exponential family form, you get: $$f(x) = \frac{\beta^{\alpha}}{\Gamma(\alpha)}x^{\alpha-1}e^{-\beta x} = e^{(\alpha-1)\ln(x) -\beta x -(\ln(\Gamma(\alpha)-\...
1
vote
0answers
21 views

Show bivariate normal distribution with non-diagonal covariance belongs to curved exponential family?

Question: Suppose that $(X_{i}, Y_{i})$, $i = 1, \dots ,n$ are sampled i.i.d. from the two-dimensional normal distribution $$ \begin{bmatrix} X & Y \end{bmatrix} \sim \mathcal{N}\left( \begin{...
0
votes
1answer
22 views

Exponential families and Gibbs measures? Any relation?

The title says it all. The formulas for exponential families and Gibbs measures seems very similar. Is there any relationship, or some kind of translation table?
1
vote
1answer
64 views

Prove that argmin of exponential distributions has multinomial distribution [closed]

Let's say we have $ T_1,T_2,\cdots,T_n \sim Exp$ and $P(X_1>a)=e^{-\lambda_1 a},P(X_2>a)=e^{-\lambda_2 a},\cdots,P(X_n>a)=e^{-\lambda_n a} $. How can I describe $\DeclareMathOperator*...
1
vote
1answer
44 views

Definition of $k$-parameter exponential family

I am currently studying the concept of sufficient statistics in mathematical statistics. The following definition is presented: Definition: $k$-parameter exponential family Let $\mathbf{Y} \sim f_\...
0
votes
0answers
53 views

Division of Multivariate Normal Distributions

$$ \newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\nc}[2]{\newcommand{#1}{#2}} \nc{\vx}{\vect{x}} \nc{\vmu}{\vect{\mu}} \nc{\vSigma}{\vect{\Sigma}} \nc{\vtheta}{\vect{\theta}} $$ ...
0
votes
0answers
138 views

Bayesian Linear Regression and the Exponential Family

In a straight forward linear regression model, assuming a fixed input $\mathbf{x}$, and additive noise with unit variance we can write: \begin{equation} p(y\mid \mathbf{x,w})=\frac{1}{\sqrt{2\pi}\...
0
votes
2answers
60 views

What is the idea behind generalized linear models?

I am watching andrew ng's video lectures on machine learning. I am trying to understand what is even the point of generalized linear models. I understand what goes on step by step in deriving things ...
0
votes
0answers
37 views

Is there just one exponential family? Or are there many exponential families? [duplicate]

I'm confused by the phrasing I've seen about exponential families. What does it mean to say "an" exponential family. Why not "the" exponential family? From a pdf from Berkely: "we define an ...
3
votes
0answers
66 views

Are there nonparametric generative models for datasets?

Typically when I see generative models, e.g., Latent Dirichlet Allocation (JMLR) or Linear/Quadratic Discriminant Analysis (wikipedia LDA), they are probabilistic models that belong to the exponential ...
1
vote
1answer
53 views

Finding Uniformly Most Powerful test

My Attempt Comparing $f(x;\theta)$ with the form $a(\theta)b(x)exp[c(\theta)d(x)]$ , we get $d(x) = log (1-x)$ and $ c(\theta ) = \theta -1 $ as monotone , increasing function in $\theta$ and ...
3
votes
1answer
182 views

explanation of why an UMVUE doesn't necessarily have to achieve the CRLB?

I'm studying uniformly minimum variance unbiased estimator(UMVUE). I have seen question on this site asking why the UMVUE doesn't achieve the CRLB(Cramer Rao lower bound), and all of the answers have ...
2
votes
0answers
22 views

Show Moment estimator and MLE are the same only for exponential family

Show Moment estimator and MLE are the same only for exponential family. I know MM is variant while MLE is not under transformation. Will this be the starting point?
0
votes
1answer
39 views

Derivation of the Objective Function for Expectation Propagation

I was reading Expectation Propagation As A Way Of Life and the original paper by Minka Expectation Propagation for Approximate Bayesian Inference and they both say that a fixed point of the EP ...
0
votes
0answers
14 views

Exponential family with finite variance

I am wondering if there exists any popular exponential family with finite variance (that does not depend on parameter $\theta$) In other words, is there family of the form: $$ p_\theta(x) = h(x)e^{\...
0
votes
0answers
54 views

Exponential family and efficient estimator

In my lecture notes there is the notion of efficiency related to the exponential family. More precisely, the lecturer stated that for an exponential family an efficient estimator always exists. How is ...
1
vote
1answer
57 views

Connection between subgaussian/subexponential and exponential family

I am wondering if there is any relationship between subgaussian/subexponential with (one parameter) exponential family. In particular, is there any sub-family density that belongs to both ...
5
votes
1answer
210 views

Is the t distribution a member of the exponential family?

From what I understand, the exponential family is defined as $$f(y;\theta,\phi) = \exp\left(\frac{y\theta - b(\theta)}{a(\phi)}+c(y,\phi)\right) $$ I've read (but not seen shown anywhere), that the ...
0
votes
1answer
44 views

How to derive a pdf of Complete Sufficient Statistic of exponential family

While studying Mathematical statistics through "Introduction to Mathematical Statistics 7th" (by Hogg and Craig), I've been stuck in the Theorem above. The answer of the exercise 7.5.8 is not given in ...
1
vote
1answer
101 views

Is the Erlang Distribution member of the Exponential family?

Given the Erlang distribution $f(x)={\begin{cases}\displaystyle {\frac {\lambda ^{n}x^{{n-1}}}{(n-1)!}}\,{\mathrm {e}}^{{-\lambda x}}&x\geq 0\\0&x<0\end{cases}}$ I want to determine, ...
0
votes
0answers
17 views

Reparameterize b(K,pi) in terms of theta

Question: $X_1, ... , X_n$ follows a binomial distribution with parameters K and 0 < $\pi\ <1.$ Use properties of Regular Exponential Class of distributions to show that the sample total $T = \...
1
vote
0answers
66 views

Convergence in distribution of parameters of exponential family

I am taking a course in inference where we have to find an approximate confidence interval for a Rayleigh distributed variable. The correct answer to this question states: Since we have an ...
0
votes
0answers
13 views

Finding prior conjugate for reparametrized model

Let $X_i$ be iid Bernoulli$(\pi)$ for $i=1,...,n$. My task is to find the prior conjugate for $\theta$, where $\theta$ is the natural parameter of the sampling model. The sampling model can be ...
1
vote
1answer
19 views

Is a range of values from an exponential distribution still exponentially distributed?

I have to generate numbers of two different exponential distribution ($e_1, e_2$) with parameters respectively $\lambda_1$ and $\lambda_2 = k \lambda_1$, with $0<k<1$. But I also want to ...

1
2 3 4 5 6