The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
17 views

GLM - exponential form

i stumbled upon the following formula in Kevin Murphy's machine learning book: I am familiar wiht the following formula for the exponential family: $$ 1/Z(\theta) h(x) \exp \left( \theta^T ...
1
vote
0answers
85 views

Karlin-Rubin theorem and exponential family

Let $X_1,...X_n$ a random sample of $X$ with density $$f(x;\alpha,\beta)=\frac{\alpha}{x^{\alpha+1}}\beta^\alpha I_{[\beta,\infty]}(x)\space\alpha>0,\beta>0$$ with $\beta$ known. Find ...
5
votes
0answers
32 views

Sufficient Statistic for non-exponential family distribution

Question: Let $X_1,X_2,....X_n$ be an iid sample from $N(\theta , 4 \theta^2 )$. I want to show that this model is not a member of the exponential family and to find a sufficient statistic for ...
0
votes
1answer
27 views

normalization constant for categorical distribution as exponential family

Let r.v. $X$ has categorical distribution. We can represent its pmf as $f(x\mid\vec{p})=\Pi_{i=1}^{K}p_i^{I[x=i]}=\exp[\sum_{i=1}^{K}I[x=i]\ln p_i]$, there is no explicit normalization constant ...
2
votes
1answer
77 views

Equivalent of z-score in exponential distribution

I have an exponentially distributed set of data, but there are a number of outliers at the margin. If I want to identify samples above the mean that I am 99.9+% confident are not the result of ...
3
votes
0answers
51 views

Sufficient statistics and UMVUE for joint poisson, bernoulli

Given a pair $(X,Y)$ of r.v.s such that: $$X \sim \text{Poisson}(\lambda)\quad \text{and}\quad Y \sim B(\frac{\lambda}{1+\lambda})$$ with $X,Y$ independent, determine a one-dimensional ...
2
votes
0answers
18 views

Role of base measure in exponential family

An exponential family distribution $p$ in the canonical form can be written as $p(x|\theta) = h(x)\exp(\theta^\top T(x) - A(\theta))$ where $A(\theta)$ is the log partition function, $T(x)$ is the ...
0
votes
1answer
27 views

Log Likelihoods of Exponential Families

How can one derive the log-likelihood of the saturated model of an exponential family in general? Differentiating the log likelihood w.r.t $\theta$ gives $y_i=\hat{\mu_i}$ but I don't think replacing ...
0
votes
0answers
19 views

Multivariate normal: from canonical parameterization to mean parameterization (or vice versa)

In their book (https://www.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf) Wainwright and Jordan consider two types of parameterizations in the exponential family, the canonical parameterization ...
1
vote
1answer
51 views

Is pdf from power family distribution an exponential family?

In my statistics class we've been going over exponential families and sufficiency, which deviates from what's in the textbook. As such, now that I need to solve problems about exponential families I ...
1
vote
0answers
19 views

Does stable distribution belong to exponential family?

According to Hougaard(1986), positive stable distribution on R+ belongs to exponential family, how about the case the support of stable distribution being less than zero? The purpose of this question ...
0
votes
0answers
16 views

Subdistributions and superdistributions of exponential families

Suppose you have an n-parameter distribution f(x) such that, for some particular set of parameter values, it is an m < n parameter distribution g(x) which is a member of an exponential family, and ...
0
votes
0answers
13 views

Does the generalized beta distribution of McDonald and Xu constitute an exponential family?

Does the generalized beta distribution of McDonald and Xu, J. Econometrics 66 (1995) 133-152, constitute an exponential family? Can it be written in a way that makes this more obvious? Alternatively, ...
2
votes
0answers
34 views

KL-divergence as a negative log likelihood for exponential families

I am reading Distributed Estimation, Information Loss and Exponential Families, where the authors consider and compare two estimators for $\theta$ in the parametric model $p(x\mid\theta)$: the ...
0
votes
0answers
25 views

Exponential Families

I have a problem of estimating moments of an exponential family by integration by parts. Lets consider the exponential family in its canonical form. $f(x)=e^{\theta x-\psi(\theta)}h(x)$. The ...
-1
votes
1answer
31 views
3
votes
1answer
106 views

Exponential distribution: How to avoid negative predictor of $\lambda$?

I have a joint distribution $P(X, Y_{1}, Y_{2}, ....)$ which contains one univariate exponential distribution ($X$) and several univariate gaussian distributions ($Y_{1}, ...$). For details regarding ...
1
vote
0answers
84 views

Best goodness-of-fit measure to determine: is my dataset power law or exponentially distributed?

I would like to determine whether a discrete dataset that I'm modeling would be better fitted by an exponential function or a power law function. I'm aware that a chi-squared test may be a suitable ...
1
vote
1answer
83 views

Conditional distribution for Exponential family

We have a random variable $X$ that belongs to the exponential family with p.d.f. $$ P_X(x|\boldsymbol \theta) = h(x) \exp\left(\eta({\boldsymbol \theta}) . T(x) - A({\boldsymbol \theta}) \right) $$ ...
3
votes
1answer
107 views

Proving a distribution is a member of the simple exponential family

Does anyone have any tips/ideas/method for proving that a distribution is a member of the simple exponential family (SEF)? Or is the process unique to each distribution? For example, I am trying to ...
4
votes
1answer
162 views

How to derive the conjugate prior of an exponential family distribution

I am trying to derive the conjugate prior of the univariate Gaussian distribution over both the mean and the precision. I know that the prior I'm looking for is the normal-gamma distribution, but the ...
2
votes
1answer
263 views

ML estimate of exponential distribution (with censored data)

In Survival Analysis, you assume the survival time of a r.v. $X_i$ to be exponentially distributed. Considering now that I have $x_1,\dots,x_n$ "outcomes" of i.i.d r.v.'s $X_i$. Only some proportion ...
4
votes
1answer
73 views

Exponential family distribution with high-order statistics

An exponential family distribution in its simplest form is given by $p(x|\theta) = \exp(\theta^\top T(x) - A(\theta))$ where $T(x)$ is a vector of sufficient statistics, $\theta$ is its natural ...
1
vote
0answers
29 views

Families of distributions with bounded variations

For what family of probability distributions $f(x)$ do we have the following property? $$ \forall x, \quad \int f(u) - f(u+x) du \leq L\| x \| $$ for some $L$. Can we say anything about the ...
0
votes
0answers
29 views

Scaling of a Weibull variate

I am currently working on an inter-event time dataset. These times are Weibull distributed (I verified my fit with a KS-Test). However, I need to subdivide them into different phases, that are in turn ...
7
votes
1answer
217 views

Why the mixtures of conjugate priors is important?

I have a questions about the mixture of conjugate priors. I learnt and say the mixture of conjugate priors a couple of times when I am learning bayesian. I am wondering why this theorem is such ...
2
votes
1answer
78 views

How to prove Bernoulli distribution belongs to the exponential family

According to a book, a distribution belongs to the exponential family if it can be written in the form of I wrote the Bernoulli distribution as $\exp\Big(y \log\,[{\mu}/{(1-\mu)}] + ...
0
votes
1answer
178 views

MLE for two-parameter exponential distribution

I have to find the parameters of a two-parameter exponential distribution using the MLE. But imposing first order conditions, I do not find enough conditions to found both the paramenters. Any hint? ...
6
votes
3answers
95 views

Are log-linear models exponential models?

What is usually referred to as "log-linear models"? Is a log-linear model an exponential model where the normalization constant is 1 (since its logarithm needs to be a linear function)? Or is there ...
11
votes
1answer
330 views

When if ever is a median statistic a sufficient statistic?

I came across a remark on The Chemical Statistician that a sample median could often be a choice for a sufficient statistic but, besides the obvious case of one or two observations where it equals the ...
2
votes
0answers
230 views

“weight” input in glm.nb function in R. How exactly does the weight affect the likelihood?

I would like to understand how the weight argument of glm.nb is affecting the likelihood function. I understand that glm.nb find the MLE in an alternating iteration process where for a given theta the ...
1
vote
0answers
184 views

What is relationship between Fisher Information and Variance in natural exponential Family?

I know that $Var(\hat\theta)\geq 1/I(\theta)$ where $I(\theta)$ is Fisher information. Let take an example of natural exponential family with density $f(x)=\lambda\exp(-\lambda x)$. In this case we ...
0
votes
0answers
14 views

Is $A(\eta)$ function in Canonical exponential family differentiable?

I'm studying canonical exponential family, and I'm stuck in differentiability of $A(\eta)$. There's no assumption or comments about $A(\eta)$ in canonical exponential family. (I struggled to search ...
0
votes
0answers
24 views

What is a convex support? (Bickel&Doksum, Mathematical Statistics, Basic ideas…Vol1)

Bickel&Doksum, Mathematical Statistics, Basic ideas...Vol1 page 122, just above Cor2.3.1, it says: Define the convex support of a probability P to be the smallest convex set C such that ...
8
votes
1answer
109 views

Is the negative binomial not expressible as in the exponential family if there are 2 unknowns?

I had a homework assignment to express the negative binomial distribution as an exponential family of distributions given that the dispersion parameter was a known constant. This was fairly easy, but ...
1
vote
2answers
383 views

Best Fit for Exponential Data

I'm trying to better understand some of the theory behind fitting models that have a nonlinear link between the response and the predictors. ...
8
votes
2answers
494 views

What is meant by the term 'exponential family'? Why it is named so?

I have come across the term exponential family. The Bernoulli, Gaussian and many more distributions come under this exponential family. What would be the commonalities between them?
5
votes
1answer
389 views

Exponential Family: Observed vs. Expected Sufficient Statistics

My question arises from reading reading Minka's "Estimating a Dirichlet Distribution", which states the following without proof in the context of deriving a maximum-likelihood estimator for a ...
6
votes
1answer
72 views

The distribution of a sufficient statistic

If I understand correctly, a distribution in the exponential family... $$\underline X\sim f_{\underline\theta}(\underline x) = exp\{\sum\limits_{i}\eta_i(\underline\theta)T_i(\underline ...
0
votes
1answer
71 views

How to show linear model corresponds to exponential family?

I am confronted with the exercise below. I have no given solution, so I hope someone can tell me whether my solution is right or wrong. I want to show the deterministic linear model $\quad ...
2
votes
1answer
83 views

What the dimension of an exponential family tell us about that family?

In Wikipedia it is stated that: A vector exponential family is said to be curved if the dimension of $$ {\boldsymbol \theta} = \left (\theta_1, \theta_2, \ldots, \theta_d \right )^T$$ is ...
0
votes
0answers
56 views

sufficient statistic and KL-divergence: Confusion with an equation

I am reading a paper, which talks about minimising KL-divergence of any arbitrary distribution over a family of exponential distribution. So, given a distribution $p$, we want to compute its ...
8
votes
2answers
253 views

A stochastically increasing exponential family for which $\lim_{\theta\rightarrow\inf\Theta}\mbox{P}_\theta(X\leq x)\neq 1$

Question A little something that I've been wondering about for a while: Let $P_\theta$ be a stochastically increasing (one-parameter) exponential family on the sample space $\mathcal{X}$ with ...
0
votes
0answers
224 views

Sample size for a non-normal distribution

I'm quite new in this field. I hope my question makes sense. I have a database that stores information for around 10.000.000 projects. Each project has several features (let's call them X) like number ...
7
votes
1answer
299 views

Do the mean and the variance always exist for exponential family distributions?

Assume a scalar random variable $X$ belongs to a vector-parameter exponential family with p.d.f. $$ f_X(x|\boldsymbol \theta) = h(x) \exp\left(\sum_{i=1}^s \eta_i({\boldsymbol \theta}) T_i(x) - ...
1
vote
0answers
44 views

Sequence of Bernouilli trials with diminishing returns

Say I have a sequence of yes-no trials where the probability of winning decays over time as the supply of wins is gradually exhausted. Assume that for every trial the probability of replacement is not ...
8
votes
1answer
623 views

Does log likelihood in GLM have guaranteed convergence to global maxima?

My questions are: Are generalized linear models (GLMs) guaranteed to converge to a global maximum? If so, why? Furthermore, what constraints are there on the link function to insure convexity? ...
4
votes
1answer
458 views

Show that Weibull distribution can be transformed to exponential family

How do I show the Weibull distribution $f:(y; \lambda, \rho)$ can be transformed to the exponential family using the transformation $z=y^\lambda$? I know the form I need to express it in is ...
10
votes
1answer
270 views

Does a canonical link function always exist for a Generalized Linear Model (GLM)?

In GLM, assuming a scalar $Y$ and $\theta$ for the underlying distribution with p.d.f. $$f_Y(y | \theta, \tau) = h(y,\tau) \exp{\left(\frac{\theta y - A(\theta)}{d(\tau)} \right)}$$ It can be shown ...
2
votes
1answer
2k views

How to find the expected Fisher information for a parameter of exponential distribution in an iid sample?

I am unclear to on how to find the Fisher information of $\lambda$ in the iid sample $X_1,...,X_n$. Attempt: I know that the fisher information for $X$ (not in a sample) would be: $I(\lambda) = - ...