The tag has no usage guidance, but it has a tag wiki.

learn more… | top users | synonyms

6
votes
1answer
61 views

Closed form function relating $\mu$ to the natural parameter for the logarithmic series distribution?

While answering another question here, I mentioned the logarithmic series distribution as a possible model for species per genus. In the course of looking at the pmf while answering that I realized ...
1
vote
0answers
25 views

Is log partition function guaranteed to be convex

In a paper by Wainwright and Jordan on page 62 it mentions that a log partition function is always convex. This is done by showing that the second derivative of the log partition function is the ...
0
votes
0answers
11 views

Extensions of bsts and CausalImpact to non-Gaussian exponential family distributions

The bsts and CausalImpact packages implement a state space time series model with an optional regularized regression component. ...
0
votes
0answers
17 views

GLM - exponential form

i stumbled upon the following formula in Kevin Murphy's machine learning book: I am familiar wiht the following formula for the exponential family: $$ 1/Z(\theta) h(x) \exp \left( \theta^T ...
1
vote
0answers
99 views

Karlin-Rubin theorem and exponential family

Let $X_1,...X_n$ a random sample of $X$ with density $$f(x;\alpha,\beta)=\frac{\alpha}{x^{\alpha+1}}\beta^\alpha I_{[\beta,\infty]}(x)\space\alpha>0,\beta>0$$ with $\beta$ known. Find ...
5
votes
0answers
34 views

Sufficient Statistic for non-exponential family distribution

Question: Let $X_1,X_2,....X_n$ be an iid sample from $N(\theta , 4 \theta^2 )$. I want to show that this model is not a member of the exponential family and to find a sufficient statistic for ...
0
votes
1answer
32 views

normalization constant for categorical distribution as exponential family

Let r.v. $X$ has categorical distribution. We can represent its pmf as $f(x\mid\vec{p})=\Pi_{i=1}^{K}p_i^{I[x=i]}=\exp[\sum_{i=1}^{K}I[x=i]\ln p_i]$, there is no explicit normalization constant ...
2
votes
1answer
93 views

Equivalent of z-score in exponential distribution

I have an exponentially distributed set of data, but there are a number of outliers at the margin. If I want to identify samples above the mean that I am 99.9+% confident are not the result of ...
3
votes
0answers
54 views

Sufficient statistics and UMVUE for joint poisson, bernoulli

Given a pair $(X,Y)$ of r.v.s such that: $$X \sim \text{Poisson}(\lambda)\quad \text{and}\quad Y \sim B(\frac{\lambda}{1+\lambda})$$ with $X,Y$ independent, determine a one-dimensional ...
2
votes
0answers
23 views

Role of base measure in exponential family

An exponential family distribution $p$ in the canonical form can be written as $p(x|\theta) = h(x)\exp(\theta^\top T(x) - A(\theta))$ where $A(\theta)$ is the log partition function, $T(x)$ is the ...
0
votes
1answer
27 views

Log Likelihoods of Exponential Families

How can one derive the log-likelihood of the saturated model of an exponential family in general? Differentiating the log likelihood w.r.t $\theta$ gives $y_i=\hat{\mu_i}$ but I don't think replacing ...
0
votes
0answers
23 views

Multivariate normal: from canonical parameterization to mean parameterization (or vice versa)

In their book (https://www.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf) Wainwright and Jordan consider two types of parameterizations in the exponential family, the canonical parameterization ...
1
vote
1answer
60 views

Is pdf from power family distribution an exponential family?

In my statistics class we've been going over exponential families and sufficiency, which deviates from what's in the textbook. As such, now that I need to solve problems about exponential families I ...
1
vote
0answers
22 views

Does stable distribution belong to exponential family?

According to Hougaard(1986), positive stable distribution on R+ belongs to exponential family, how about the case the support of stable distribution being less than zero? The purpose of this question ...
0
votes
0answers
17 views

Subdistributions and superdistributions of exponential families

Suppose you have an n-parameter distribution f(x) such that, for some particular set of parameter values, it is an m < n parameter distribution g(x) which is a member of an exponential family, and ...
0
votes
0answers
23 views

Does the generalized beta distribution of McDonald and Xu constitute an exponential family?

Does the generalized beta distribution of McDonald and Xu, J. Econometrics 66 (1995) 133-152, constitute an exponential family? Can it be written in a way that makes this more obvious? Alternatively, ...
2
votes
0answers
36 views

KL-divergence as a negative log likelihood for exponential families

I am reading Distributed Estimation, Information Loss and Exponential Families, where the authors consider and compare two estimators for $\theta$ in the parametric model $p(x\mid\theta)$: the ...
0
votes
0answers
26 views

Exponential Families

I have a problem of estimating moments of an exponential family by integration by parts. Lets consider the exponential family in its canonical form. $f(x)=e^{\theta x-\psi(\theta)}h(x)$. The ...
-1
votes
1answer
33 views
3
votes
1answer
115 views

Exponential distribution: How to avoid negative predictor of $\lambda$?

I have a joint distribution $P(X, Y_{1}, Y_{2}, ....)$ which contains one univariate exponential distribution ($X$) and several univariate gaussian distributions ($Y_{1}, ...$). For details regarding ...
1
vote
0answers
86 views

Best goodness-of-fit measure to determine: is my dataset power law or exponentially distributed?

I would like to determine whether a discrete dataset that I'm modeling would be better fitted by an exponential function or a power law function. I'm aware that a chi-squared test may be a suitable ...
1
vote
1answer
88 views

Conditional distribution for Exponential family

We have a random variable $X$ that belongs to the exponential family with p.d.f. $$ P_X(x|\boldsymbol \theta) = h(x) \exp\left(\eta({\boldsymbol \theta}) . T(x) - A({\boldsymbol \theta}) \right) $$ ...
3
votes
1answer
116 views

Proving a distribution is a member of the simple exponential family

Does anyone have any tips/ideas/method for proving that a distribution is a member of the simple exponential family (SEF)? Or is the process unique to each distribution? For example, I am trying to ...
4
votes
1answer
170 views

How to derive the conjugate prior of an exponential family distribution

I am trying to derive the conjugate prior of the univariate Gaussian distribution over both the mean and the precision. I know that the prior I'm looking for is the normal-gamma distribution, but the ...
2
votes
1answer
315 views

ML estimate of exponential distribution (with censored data)

In Survival Analysis, you assume the survival time of a r.v. $X_i$ to be exponentially distributed. Considering now that I have $x_1,\dots,x_n$ "outcomes" of i.i.d r.v.'s $X_i$. Only some proportion ...
4
votes
1answer
74 views

Exponential family distribution with high-order statistics

An exponential family distribution in its simplest form is given by $p(x|\theta) = \exp(\theta^\top T(x) - A(\theta))$ where $T(x)$ is a vector of sufficient statistics, $\theta$ is its natural ...
1
vote
0answers
29 views

Families of distributions with bounded variations

For what family of probability distributions $f(x)$ do we have the following property? $$ \forall x, \quad \int f(u) - f(u+x) du \leq L\| x \| $$ for some $L$. Can we say anything about the ...
0
votes
0answers
30 views

Scaling of a Weibull variate

I am currently working on an inter-event time dataset. These times are Weibull distributed (I verified my fit with a KS-Test). However, I need to subdivide them into different phases, that are in turn ...
7
votes
1answer
224 views

Why the mixtures of conjugate priors is important?

I have a questions about the mixture of conjugate priors. I learnt and say the mixture of conjugate priors a couple of times when I am learning bayesian. I am wondering why this theorem is such ...
2
votes
1answer
80 views

How to prove Bernoulli distribution belongs to the exponential family

According to a book, a distribution belongs to the exponential family if it can be written in the form of I wrote the Bernoulli distribution as $\exp\Big(y \log\,[{\mu}/{(1-\mu)}] + ...
0
votes
1answer
190 views

MLE for two-parameter exponential distribution

I have to find the parameters of a two-parameter exponential distribution using the MLE. But imposing first order conditions, I do not find enough conditions to found both the paramenters. Any hint? ...
6
votes
3answers
103 views

Are log-linear models exponential models?

What is usually referred to as "log-linear models"? Is a log-linear model an exponential model where the normalization constant is 1 (since its logarithm needs to be a linear function)? Or is there ...
11
votes
1answer
337 views

When if ever is a median statistic a sufficient statistic?

I came across a remark on The Chemical Statistician that a sample median could often be a choice for a sufficient statistic but, besides the obvious case of one or two observations where it equals the ...
2
votes
0answers
241 views

“weight” input in glm.nb function in R. How exactly does the weight affect the likelihood?

I would like to understand how the weight argument of glm.nb is affecting the likelihood function. I understand that glm.nb find the MLE in an alternating iteration process where for a given theta the ...
1
vote
0answers
191 views

What is relationship between Fisher Information and Variance in natural exponential Family?

I know that $Var(\hat\theta)\geq 1/I(\theta)$ where $I(\theta)$ is Fisher information. Let take an example of natural exponential family with density $f(x)=\lambda\exp(-\lambda x)$. In this case we ...
0
votes
0answers
15 views

Is $A(\eta)$ function in Canonical exponential family differentiable?

I'm studying canonical exponential family, and I'm stuck in differentiability of $A(\eta)$. There's no assumption or comments about $A(\eta)$ in canonical exponential family. (I struggled to search ...
0
votes
0answers
25 views

What is a convex support? (Bickel&Doksum, Mathematical Statistics, Basic ideas…Vol1)

Bickel&Doksum, Mathematical Statistics, Basic ideas...Vol1 page 122, just above Cor2.3.1, it says: Define the convex support of a probability P to be the smallest convex set C such that ...
8
votes
1answer
118 views

Is the negative binomial not expressible as in the exponential family if there are 2 unknowns?

I had a homework assignment to express the negative binomial distribution as an exponential family of distributions given that the dispersion parameter was a known constant. This was fairly easy, but ...
1
vote
2answers
432 views

Best Fit for Exponential Data

I'm trying to better understand some of the theory behind fitting models that have a nonlinear link between the response and the predictors. ...
8
votes
2answers
497 views

What is meant by the term 'exponential family'? Why it is named so?

I have come across the term exponential family. The Bernoulli, Gaussian and many more distributions come under this exponential family. What would be the commonalities between them?
5
votes
1answer
415 views

Exponential Family: Observed vs. Expected Sufficient Statistics

My question arises from reading reading Minka's "Estimating a Dirichlet Distribution", which states the following without proof in the context of deriving a maximum-likelihood estimator for a ...
6
votes
1answer
74 views

The distribution of a sufficient statistic

If I understand correctly, a distribution in the exponential family... $$\underline X\sim f_{\underline\theta}(\underline x) = exp\{\sum\limits_{i}\eta_i(\underline\theta)T_i(\underline ...
0
votes
1answer
72 views

How to show linear model corresponds to exponential family?

I am confronted with the exercise below. I have no given solution, so I hope someone can tell me whether my solution is right or wrong. I want to show the deterministic linear model $\quad ...
2
votes
1answer
85 views

What the dimension of an exponential family tell us about that family?

In Wikipedia it is stated that: A vector exponential family is said to be curved if the dimension of $$ {\boldsymbol \theta} = \left (\theta_1, \theta_2, \ldots, \theta_d \right )^T$$ is ...
1
vote
0answers
60 views

sufficient statistic and KL-divergence: Confusion with an equation

I am reading a paper, which talks about minimising KL-divergence of any arbitrary distribution over a family of exponential distribution. So, given a distribution $p$, we want to compute its ...
8
votes
2answers
257 views

A stochastically increasing exponential family for which $\lim_{\theta\rightarrow\inf\Theta}\mbox{P}_\theta(X\leq x)\neq 1$

Question A little something that I've been wondering about for a while: Let $P_\theta$ be a stochastically increasing (one-parameter) exponential family on the sample space $\mathcal{X}$ with ...
0
votes
0answers
227 views

Sample size for a non-normal distribution

I'm quite new in this field. I hope my question makes sense. I have a database that stores information for around 10.000.000 projects. Each project has several features (let's call them X) like number ...
7
votes
1answer
307 views

Do the mean and the variance always exist for exponential family distributions?

Assume a scalar random variable $X$ belongs to a vector-parameter exponential family with p.d.f. $$ f_X(x|\boldsymbol \theta) = h(x) \exp\left(\sum_{i=1}^s \eta_i({\boldsymbol \theta}) T_i(x) - ...
1
vote
0answers
45 views

Sequence of Bernouilli trials with diminishing returns

Say I have a sequence of yes-no trials where the probability of winning decays over time as the supply of wins is gradually exhausted. Assume that for every trial the probability of replacement is not ...
8
votes
1answer
651 views

Does log likelihood in GLM have guaranteed convergence to global maxima?

My questions are: Are generalized linear models (GLMs) guaranteed to converge to a global maximum? If so, why? Furthermore, what constraints are there on the link function to insure convexity? ...