The tag has no usage guidance, but it has a tag wiki.

learn more… | top users | synonyms

5
votes
0answers
18 views

Is there a general expression for ancillary statistics in exponential families?

It is known that an i.i.d sample $X_1,\cdots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistics if $S(X)$ depends on the sample only through ...
4
votes
1answer
34 views

Cramér-Rao Lower Bound for Exponential Families

I am having a problem with applying the Cramér-Rao inequality to identify the lower bound for the variance of an unbiased estimator and hoped that you guys could help me. The problem is the following: ...
1
vote
1answer
30 views

Exponential family: examples where scaling constant is data dependent

The general form of a exponential family distribution is given as $$p(x|\theta) = h(x) g(\theta) \exp(\theta^Tu(x))$$ where $h(x)$ is referred to as the "scaling constant" (e.g. in Murphy's ML ...
0
votes
0answers
42 views

Self-Starting Nonlinear Regression Model

In the following two nonlinear regression models, is it possible for the dependent variable (y) or the independent variable (x) to be negative? \begin{align} y &= φ_1 \exp[-\exp(φ_2)x] + φ_3 ...
2
votes
1answer
26 views

Natural parameters of Bernoulli distribution

In Bishop's Pattern Recognition and Machine Learning, he states that exponential family distributions over $\mathbf{x}$ given parameters $\boldsymbol{\eta}$ can be written as $p(\mathbf{x} | ...
3
votes
1answer
28 views

Producing samples from exponential family when sufficient statistics are known

Suppose I have a distribution which belongs to an exponential family, of the form $$p(x) = \frac{\exp(-\sum_k \eta_k T_k(x))}{Z},$$ where $T_k(x)$ are a fixed set of sufficient statistics, $\eta_k$ ...
0
votes
0answers
12 views

Natural exponential family and GLMs

In the generalized linear modelling framework, it is often stated that the conditional distribution of the response needs to be in the exponential family (there are some extentions and refinements to ...
0
votes
0answers
128 views

A curious connection between the exponential family and natural exponential family

Here is an interesting way of generating a probability distribution in the natural exponential family (NEF) from a distribution in the exponential family (EF). Let $f(y) = h(y)exp[\eta ...
1
vote
1answer
68 views

Are complete statistics always sufficient?

I know that a complete sufficient statistic $T$ is such that 1) $T$ is sufficient for $\theta$, unknown parameter and 2) $T$ is complete. So, is it always the case? If the answer is not, what ...
1
vote
1answer
26 views

Exponential family regularity conditions

Given a generic pmf or pdf as $f(x;\theta)$, where $\theta$ is a vector-valued parameter, it can be reparimetrized into the exponential family version $f(x;\theta)=a(\theta)g(x)exp[{\sum_{i=1}^{k} ...
0
votes
0answers
60 views

Time to second failure of independent exponential lifetimes

This maybe an easy question, but as I am a beginner, I need help. Suppose that an electronic system contains $n$ similar components that function independently of each other and that are connected ...
4
votes
3answers
155 views

Which parameter should be considered as “scale” parameter for Gamma distribution?

From Wikipedia and probably all statistics textbooks, we know that in the density of a Gamma random variable $$f(x; k, \theta) = \frac{1}{\Gamma(k)\theta^k}x^{k - 1}e^{-\frac{x}{\theta}}, \quad x > ...
3
votes
0answers
36 views

Dispersion parameters in GLM

I'm trying to find the motivation behind the extended form of the exponential family of distributions in the fundamental paper on GLM by Nelder and Wedderburn (Generalized Linear Models, J. R. ...
0
votes
0answers
18 views

Is the Dirichlet compound multinomial (DCM) distribution in the exponential family?

It occurs to me that the DCM (a.k.a multivariate Pòlya) distribution can be written in the exponential form when the number of draws, $n_1+n_2+\ldots+n_k=N$, is known: $$ p(n|a) = exp \left(tr \left( ...
1
vote
1answer
43 views

Conway–Maxwell–Poisson (CMP) distribution and exponential family

So I have a question here about the CMP distribution: My understanding is that $b(\theta)$ is only a function of $\theta$ but why is $v$ able to be included in that function, would $v$ not be a ...
2
votes
0answers
17 views

Effect of the measure on exponential families

This might be a very naive question. Wikipedia describes an exponential family as a distribution $f(x | \theta) = h(x)exp( - \theta x - A(\theta))$ where $A(\theta) = \int h(x)exp( - \theta x)$ ...
2
votes
1answer
62 views

Exponential family form of multinomial distribution

I feel like this must be a duplicate, but I don't know the magic words to find the appropriate post... The multinomial distribution is a member of the exponential family. I am used to seeing the ...
1
vote
0answers
37 views

complete sufficient statistic exercise

I have to find complete sufficient statistic of the following pdf $$f(x|\theta)=\frac{\theta}{(1+x)^{(1+\theta)}},\quad 0<x<\infty,\theta>0.$$ My Attempt: The joint density $$f(\mathbf ...
0
votes
0answers
38 views

covariance matrix of natural parameters and sufficient statistics in exp family

Let $f(x\mid\mathbf{\theta})$ be a pmf in exponential family, $\theta_i$s are natural parameters and $T_i(X)$ be sufficient statistics. We know that ...
1
vote
0answers
31 views

Implication of Theorem on 'Empirical Bayes optimality' (Morris 1983)

My question is about Theorem 1 from Morris 1983; Parametric Empirical Bayes Inference (http://www.jstor.org/stable/2287098?seq=1#page_scan_tab_contents). Suppose Y has a univariate natural ...
4
votes
2answers
157 views

Find the joint distribution of $X_1$ and $\sum_{i=1}^n X_i$

This question is from Robert Hogg's Introduction to Mathematical Statistics 6th version question 7.6.7. The problem is : Let a random sample of size n be taken from a distribution with the pdf ...
6
votes
1answer
76 views

Closed form function relating $\mu$ to the natural parameter for the logarithmic series distribution?

While answering another question here, I mentioned the logarithmic series distribution as a possible model for species per genus. In the course of looking at the pmf while answering that I realized ...
1
vote
0answers
58 views

Is log partition function guaranteed to be convex

In a paper by Wainwright and Jordan on page 62 it mentions that a log partition function is always convex. This is done by showing that the second derivative of the log partition function is the ...
0
votes
0answers
22 views

Extensions of bsts and CausalImpact to non-Gaussian exponential family distributions

The bsts and CausalImpact packages implement a state space time series model with an optional regularized regression component. ...
0
votes
0answers
17 views

GLM - exponential form

i stumbled upon the following formula in Kevin Murphy's machine learning book: I am familiar wiht the following formula for the exponential family: $$ 1/Z(\theta) h(x) \exp \left( \theta^T ...
1
vote
0answers
170 views

Karlin-Rubin theorem and exponential family

Let $X_1,...X_n$ a random sample of $X$ with density $$f(x;\alpha,\beta)=\frac{\alpha}{x^{\alpha+1}}\beta^\alpha I_{[\beta,\infty]}(x)\space\alpha>0,\beta>0$$ with $\beta$ known. Find ...
5
votes
0answers
60 views

Sufficient Statistic for non-exponential family distribution

Question: Let $X_1,X_2,....X_n$ be an iid sample from $N(\theta , 4 \theta^2 )$. I want to show that this model is not a member of the exponential family and to find a sufficient statistic for ...
0
votes
1answer
52 views

normalization constant for categorical distribution as exponential family

Let r.v. $X$ has categorical distribution. We can represent its pmf as $f(x\mid\vec{p})=\Pi_{i=1}^{K}p_i^{I[x=i]}=\exp[\sum_{i=1}^{K}I[x=i]\ln p_i]$, there is no explicit normalization constant ...
2
votes
1answer
238 views

Equivalent of z-score in exponential distribution

I have an exponentially distributed set of data, but there are a number of outliers at the margin. If I want to identify samples above the mean that I am 99.9+% confident are not the result of ...
3
votes
0answers
86 views

Sufficient statistics and UMVUE for joint poisson, bernoulli

Given a pair $(X,Y)$ of r.v.s such that: $$X \sim \text{Poisson}(\lambda)\quad \text{and}\quad Y \sim B(\frac{\lambda}{1+\lambda})$$ with $X,Y$ independent, determine a one-dimensional ...
3
votes
1answer
53 views

Role of base measure in exponential family

An exponential family distribution $p$ in the canonical form can be written as $p(x|\theta) = h(x)\exp(\theta^\top T(x) - A(\theta))$ where $A(\theta)$ is the log partition function, $T(x)$ is the ...
0
votes
1answer
31 views

Log Likelihoods of Exponential Families

How can one derive the log-likelihood of the saturated model of an exponential family in general? Differentiating the log likelihood w.r.t $\theta$ gives $y_i=\hat{\mu_i}$ but I don't think replacing ...
0
votes
0answers
64 views

Multivariate normal: from canonical parameterization to mean parameterization (or vice versa)

In their book (https://www.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf) Wainwright and Jordan consider two types of parameterizations in the exponential family, the canonical parameterization ...
1
vote
1answer
120 views

Is pdf from power family distribution an exponential family?

In my statistics class we've been going over exponential families and sufficiency, which deviates from what's in the textbook. As such, now that I need to solve problems about exponential families I ...
1
vote
0answers
31 views

Does stable distribution belong to exponential family?

According to Hougaard(1986), positive stable distribution on R+ belongs to exponential family, how about the case the support of stable distribution being less than zero? The purpose of this question ...
0
votes
0answers
22 views

Subdistributions and superdistributions of exponential families

Suppose you have an n-parameter distribution f(x) such that, for some particular set of parameter values, it is an m < n parameter distribution g(x) which is a member of an exponential family, and ...
0
votes
0answers
30 views

Does the generalized beta distribution of McDonald and Xu constitute an exponential family?

Does the generalized beta distribution of McDonald and Xu, J. Econometrics 66 (1995) 133-152, constitute an exponential family? Can it be written in a way that makes this more obvious? Alternatively, ...
2
votes
0answers
45 views

KL-divergence as a negative log likelihood for exponential families

I am reading Distributed Estimation, Information Loss and Exponential Families, where the authors consider and compare two estimators for $\theta$ in the parametric model $p(x\mid\theta)$: the ...
0
votes
0answers
36 views

Exponential Families

I have a problem of estimating moments of an exponential family by integration by parts. Lets consider the exponential family in its canonical form. $f(x)=e^{\theta x-\psi(\theta)}h(x)$. The ...
-1
votes
1answer
40 views
3
votes
1answer
173 views

Exponential distribution: How to avoid negative predictor of $\lambda$?

I have a joint distribution $P(X, Y_{1}, Y_{2}, ....)$ which contains one univariate exponential distribution ($X$) and several univariate gaussian distributions ($Y_{1}, ...$). For details regarding ...
1
vote
0answers
121 views

Best goodness-of-fit measure to determine: is my dataset power law or exponentially distributed?

I would like to determine whether a discrete dataset that I'm modeling would be better fitted by an exponential function or a power law function. I'm aware that a chi-squared test may be a suitable ...
1
vote
1answer
147 views

Conditional distribution for Exponential family

We have a random variable $X$ that belongs to the exponential family with p.d.f. $$ P_X(x|\boldsymbol \theta) = h(x) \exp\left(\eta({\boldsymbol \theta}) . T(x) - A({\boldsymbol \theta}) \right) $$ ...
3
votes
1answer
193 views

Proving a distribution is a member of the simple exponential family

Does anyone have any tips/ideas/method for proving that a distribution is a member of the simple exponential family (SEF)? Or is the process unique to each distribution? For example, I am trying to ...
5
votes
1answer
377 views

How to derive the conjugate prior of an exponential family distribution

I am trying to derive the conjugate prior of the univariate Gaussian distribution over both the mean and the precision. I know that the prior I'm looking for is the normal-gamma distribution, but the ...
2
votes
1answer
795 views

ML estimate of exponential distribution (with censored data)

In Survival Analysis, you assume the survival time of a r.v. $X_i$ to be exponentially distributed. Considering now that I have $x_1,\dots,x_n$ "outcomes" of i.i.d r.v.'s $X_i$. Only some proportion ...
4
votes
1answer
85 views

Exponential family distribution with high-order statistics

An exponential family distribution in its simplest form is given by $p(x|\theta) = \exp(\theta^\top T(x) - A(\theta))$ where $T(x)$ is a vector of sufficient statistics, $\theta$ is its natural ...
1
vote
0answers
29 views

Families of distributions with bounded variations

For what family of probability distributions $f(x)$ do we have the following property? $$ \forall x, \quad \int f(u) - f(u+x) du \leq L\| x \| $$ for some $L$. Can we say anything about the ...
7
votes
1answer
304 views

Why the mixtures of conjugate priors is important?

I have a questions about the mixture of conjugate priors. I learnt and say the mixture of conjugate priors a couple of times when I am learning bayesian. I am wondering why this theorem is such ...
2
votes
1answer
137 views

How to prove Bernoulli distribution belongs to the exponential family

According to a book, a distribution belongs to the exponential family if it can be written in the form of I wrote the Bernoulli distribution as $\exp\Big(y \log\,[{\mu}/{(1-\mu)}] + ...