The tag has no usage guidance, but it has a tag wiki.

learn more… | top users | synonyms

2
votes
1answer
12 views

GLM (conditional/unconditional) distribution

Based on my readings about GLM, I am pretty sure that when we say the distribution of the response variable $y$ is a member of exponential family of distribution, what we really mean is that ...
3
votes
2answers
201 views

Why computing P(x,D) is simpler than P(x|D) in exponential bayesian networks?

I am reading this tutorial on variational inference and wonder why the statement in the question title which is mentioned on page 3 is true.
3
votes
1answer
45 views

Can a generalized linear model use shifted exponential as residual distribution?

I am facing a modeling problem: $t_{ij} = D_i + T_j + \epsilon_{ij}, i=0...641, j\in\mathbb{N}$ where $\epsilon_{ij}$ follows exponential distribution, $\epsilon_{ij} \sim \lambda e^{-\lambda t}, \...
0
votes
0answers
11 views

Mixed Model To Fit Exponential Decay Process “Within Subjects”, then regress the half-life on a between-subjects factor

Here is my problem: I have a set of $N = 3000$ reactions following first order kinetics, and I have 6 timepoints taken at 3 minute intervals for each of these $N$ reactions, where I am measuring my ...
3
votes
1answer
73 views

What r parameter is used in a negative binomial regression?

From my understanding of the negative binomial regression, we have $Y_i|X_i; \theta$ is distributed $Neg Binom (r_i, p_i)$, where $r_i$ is known and fixed (analogous to a fixed $\sigma^2$ when we ...
0
votes
1answer
37 views

Help Deriving Variance Function - Binomial GLM

I'm having difficulty replicating/deriving a result in GLM's for Binomial data. That is, if $Y \sim Bin(n, \mu)$ and we put the distribution of $Y/n$ into exponential family form (with a dispersion ...
3
votes
1answer
47 views

Are there any non exponential family distributions with conjugate priors? [duplicate]

I believed I had been taught that only exponential family distributions have conjugate priors but I have recently read that ' all exponential family distributions have conjugate priors', leaving the ...
0
votes
0answers
6 views

Why is the parameter and the random variable swapped in this conjugate distribution pdf?

I'm reading a journal titled Claims reserving in the hierarchical generalized linear model (hglm) by Gigante, Picech and Sigalotti. In the distributional assumption for the unobserved risk parameters, ...
0
votes
1answer
21 views

Show that Weibull distribution belongs to a one dimensional exponential family

It is given that $f_\eta(y) = h(y)exp(\eta T(y)-A^*(\eta))$ $P_Y(y)= \frac{k}{\lambda} (\frac{y}{\lambda})^{k-1}exp(-(\frac{y}{\lambda})^k)$ What i did was by arranging $P_Y(y)$ to get $\frac{k}{\...
2
votes
0answers
26 views

Expected time between two events

I'm having trouble with the following problem: Consider a game between two players A and B. Player A must complete three tasks each of which take an exponentially distributed amount of time with ...
1
vote
0answers
13 views

Computing the partition-function of an exponential family member

I am working on a Monte Carlo Expectation Propagation problem. In that context I got the following property: $ I = \sum\limits_i w^{(i)} \log p_\eta(x^{(i)}) $ where $\{w^{(i)}\}_i$ are weights, $...
0
votes
0answers
21 views

What is the difference between a “natural” sufficient statistic and a sufficient statistic?

Typically, in dealing with properties of exponential families, which are parametrized as the following: $$ f_X(x| \theta) = h(x)\ \exp\Big(\ \eta(\theta) T(x) - A(\theta)\ \Big) $$ where $h(x)$ and ...
0
votes
0answers
27 views

Zero-truncated Poisson model

In the theory of generalised linear models, you may use the exponential family to find the mean and variance of certain distributions. How would the mean and expectation of the zero-truncated Poisson ...
0
votes
1answer
36 views

Does the following pmf belong to the exponential family?

I recently saw* a pmf: $f(y)=\frac{\mu^y}{(y!)^\theta z(\mu,\theta)}$, where $z(\mu,\theta) = \sum_{i=0}^{\infty}\frac{\mu^i}{(i!)^\theta}$. * It is a bonus question on a homework assignment. My ...
1
vote
0answers
27 views

Variance of Distributions from the Exponential Family

I want to understand how the variance of an exponential family behaves. To take a very concrete example. Let consider the unit ball $B$ in d dimensions. Consider the following distribution over unit ...
0
votes
0answers
27 views

Bounds of integration query

My question is for personal research. When finding the distribution of say Z=X+Y where X and Y are both exponentially distributed r.v's where X,Y > 0, I use the convolution method. I do the same for ...
2
votes
1answer
26 views

GLM with empirical distribution

If I understand GLM correctly, to run a GLM model I need to specify the particular transformation $f$ that ensures the conditional distribution of $f(Y)$ given $X$ is from the exponential family. (I ...
0
votes
1answer
20 views

Example question about the exponential distribution

I was given the following question. My answer C was marked incorrect. My method of calculating it was to use the exponential distribution with the parameter $\lambda = 2$: $$\int_{0.75}^\infty 2e^{-...
5
votes
0answers
26 views

Is there a general expression for ancillary statistics in exponential families?

It is known that an i.i.d sample $X_1,\cdots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistics if $S(X)$ depends on the sample only through $\frac{X_1}{...
4
votes
1answer
57 views

Cramér-Rao Lower Bound for Exponential Families

I am having a problem with applying the Cramér-Rao inequality to identify the lower bound for the variance of an unbiased estimator and hoped that you guys could help me. The problem is the following: ...
1
vote
1answer
62 views

Exponential family: examples where scaling constant is data dependent

The general form of a exponential family distribution is given as $$p(x|\theta) = h(x) g(\theta) \exp(\theta^Tu(x))$$ where $h(x)$ is referred to as the "scaling constant" (e.g. in Murphy's ML book)....
0
votes
0answers
48 views

Self-Starting Nonlinear Regression Model

In the following two nonlinear regression models, is it possible for the dependent variable (y) or the independent variable (x) to be negative? \begin{align} y &= φ_1 \exp[-\exp(φ_2)x] + φ_3 \...
2
votes
1answer
35 views

Natural parameters of Bernoulli distribution

In Bishop's Pattern Recognition and Machine Learning, he states that exponential family distributions over $\mathbf{x}$ given parameters $\boldsymbol{\eta}$ can be written as $p(\mathbf{x} | \...
3
votes
1answer
38 views

Producing samples from exponential family when sufficient statistics are known

Suppose I have a distribution which belongs to an exponential family, of the form $$p(x) = \frac{\exp(-\sum_k \eta_k T_k(x))}{Z},$$ where $T_k(x)$ are a fixed set of sufficient statistics, $\eta_k$ ...
0
votes
0answers
20 views

Natural exponential family and GLMs

In the generalized linear modelling framework, it is often stated that the conditional distribution of the response needs to be in the exponential family (there are some extentions and refinements to ...
0
votes
0answers
185 views

A curious connection between the exponential family and natural exponential family

Here is an interesting way of generating a probability distribution in the natural exponential family (NEF) from a distribution in the exponential family (EF). Let $f(y) = h(y)exp[\eta T(y)-\psi(\eta)...
2
votes
1answer
141 views

Are complete statistics always sufficient?

I know that a complete sufficient statistic $T$ is such that 1) $T$ is sufficient for $\theta$, unknown parameter and 2) $T$ is complete. So, is it always the case? If the answer is not, what ...
1
vote
1answer
71 views

Exponential family regularity conditions

Given a generic pmf or pdf as $f(x;\theta)$, where $\theta$ is a vector-valued parameter, it can be reparimetrized into the exponential family version $f(x;\theta)=a(\theta)g(x)exp[{\sum_{i=1}^{k} b_i(...
0
votes
0answers
103 views

Time to second failure of independent exponential lifetimes

This maybe an easy question, but as I am a beginner, I need help. Suppose that an electronic system contains $n$ similar components that function independently of each other and that are connected ...
4
votes
3answers
190 views

Which parameter should be considered as “scale” parameter for Gamma distribution?

From Wikipedia and probably all statistics textbooks, we know that in the density of a Gamma random variable $$f(x; k, \theta) = \frac{1}{\Gamma(k)\theta^k}x^{k - 1}e^{-\frac{x}{\theta}}, \quad x > ...
3
votes
0answers
51 views

Dispersion parameters in GLM

I'm trying to find the motivation behind the extended form of the exponential family of distributions in the fundamental paper on GLM by Nelder and Wedderburn (Generalized Linear Models, J. R. Statist....
0
votes
0answers
49 views

Is the Dirichlet compound multinomial (DCM) distribution in the exponential family?

It occurs to me that the DCM (a.k.a multivariate Pòlya) distribution can be written in the exponential form when the number of draws, $n_1+n_2+\ldots+n_k=N$, is known: $$ p(n|a) = exp \left(tr \left( \...
1
vote
1answer
71 views

Conway–Maxwell–Poisson (CMP) distribution and exponential family

So I have a question here about the CMP distribution: My understanding is that $b(\theta)$ is only a function of $\theta$ but why is $v$ able to be included in that function, would $v$ not be a ...
2
votes
0answers
20 views

Effect of the measure on exponential families

This might be a very naive question. Wikipedia describes an exponential family as a distribution $f(x | \theta) = h(x)exp( - \theta x - A(\theta))$ where $A(\theta) = \int h(x)exp( - \theta x)$ ...
2
votes
1answer
164 views

Exponential family form of multinomial distribution

I feel like this must be a duplicate, but I don't know the magic words to find the appropriate post... The multinomial distribution is a member of the exponential family. I am used to seeing the "...
4
votes
1answer
61 views

complete sufficient statistic exercise

I have to find complete sufficient statistic of the following pdf $$f(x|\theta)=\frac{\theta}{(1+x)^{(1+\theta)}},\quad 0<x<\infty,\theta>0.$$ My Attempt: The joint density $$f(\mathbf x|\...
0
votes
0answers
61 views

covariance matrix of natural parameters and sufficient statistics in exp family

Let $f(x\mid\mathbf{\theta})$ be a pmf in exponential family, $\theta_i$s are natural parameters and $T_i(X)$ be sufficient statistics. We know that $\text{cov}(\hat\theta_i,\hat\theta_j)=[-l''(\hat\...
1
vote
0answers
32 views

Implication of Theorem on 'Empirical Bayes optimality' (Morris 1983)

My question is about Theorem 1 from Morris 1983; Parametric Empirical Bayes Inference (http://www.jstor.org/stable/2287098?seq=1#page_scan_tab_contents). Suppose Y has a univariate natural ...
4
votes
2answers
180 views

Find the joint distribution of $X_1$ and $\sum_{i=1}^n X_i$

This question is from Robert Hogg's Introduction to Mathematical Statistics 6th version question 7.6.7. The problem is : Let a random sample of size n be taken from a distribution with the pdf $$...
6
votes
1answer
87 views

Closed form function relating $\mu$ to the natural parameter for the logarithmic series distribution?

While answering another question here, I mentioned the logarithmic series distribution as a possible model for species per genus. In the course of looking at the pmf while answering that I realized ...
1
vote
0answers
100 views

Is log partition function guaranteed to be convex

In a paper by Wainwright and Jordan on page 62 it mentions that a log partition function is always convex. This is done by showing that the second derivative of the log partition function is the ...
0
votes
0answers
33 views

Extensions of bsts and CausalImpact to non-Gaussian exponential family distributions

The bsts and CausalImpact packages implement a state space time series model with an optional regularized regression component. ...
0
votes
0answers
27 views

GLM - exponential form

i stumbled upon the following formula in Kevin Murphy's machine learning book: I am familiar wiht the following formula for the exponential family: $$ 1/Z(\theta) h(x) \exp \left( \theta^T \phi(x)...
1
vote
0answers
332 views

Karlin-Rubin theorem and exponential family

Let $X_1,...X_n$ a random sample of $X$ with density $$f(x;\alpha,\beta)=\frac{\alpha}{x^{\alpha+1}}\beta^\alpha I_{[\beta,\infty]}(x)\space\alpha>0,\beta>0$$ with $\beta$ known. Find a ...
5
votes
0answers
77 views

Sufficient Statistic for non-exponential family distribution

Question: Let $X_1,X_2,....X_n$ be an iid sample from $N(\theta , 4 \theta^2 )$. I want to show that this model is not a member of the exponential family and to find a sufficient statistic for $\theta$...
0
votes
1answer
74 views

normalization constant for categorical distribution as exponential family

Let r.v. $X$ has categorical distribution. We can represent its pmf as $f(x\mid\vec{p})=\Pi_{i=1}^{K}p_i^{I[x=i]}=\exp[\sum_{i=1}^{K}I[x=i]\ln p_i]$, there is no explicit normalization constant (...
2
votes
1answer
427 views

Equivalent of z-score in exponential distribution

I have an exponentially distributed set of data, but there are a number of outliers at the margin. If I want to identify samples above the mean that I am 99.9+% confident are not the result of ...
3
votes
0answers
94 views

Sufficient statistics and UMVUE for joint poisson, bernoulli

Given a pair $(X,Y)$ of r.v.s such that: $$X \sim \text{Poisson}(\lambda)\quad \text{and}\quad Y \sim B(\frac{\lambda}{1+\lambda})$$ with $X,Y$ independent, determine a one-dimensional sufficient ...
3
votes
1answer
104 views

Role of base measure in exponential family

An exponential family distribution $p$ in the canonical form can be written as $p(x|\theta) = h(x)\exp(\theta^\top T(x) - A(\theta))$ where $A(\theta)$ is the log partition function, $T(x)$ is the ...
0
votes
1answer
33 views

Log Likelihoods of Exponential Families

How can one derive the log-likelihood of the saturated model of an exponential family in general? Differentiating the log likelihood w.r.t $\theta$ gives $y_i=\hat{\mu_i}$ but I don't think replacing ...