The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
22 views

Exponential Families

I have a problem of estimating moments of an exponential family by integration by parts. Lets consider the exponential family in its canonical form. $f(x)=e^{\theta x-\psi(\theta)}h(x)$. The ...
0
votes
0answers
63 views

Expected values

Consider the posterior distribution $g(\delta|z)=e^{z\delta-\psi(z)}\big(e^{-\delta^2/2}g(\delta)\big).$ Then the posterior mean is given by: $E(\delta|z)=e^{-\psi(z)}\int_{-\infty}^{\infty}\delta ...
-1
votes
1answer
26 views
3
votes
1answer
57 views

Exponential distribution: How to avoid negative predictor of $\lambda$?

I have a joint distribution $P(X, Y_{1}, Y_{2}, ....)$ which contains one univariate exponential distribution ($X$) and several univariate gaussian distributions ($Y_{1}, ...$). For details regarding ...
1
vote
0answers
44 views

Best goodness-of-fit measure to determine: is my dataset power law or exponentially distributed?

I would like to determine whether a discrete dataset that I'm modeling would be better fitted by an exponential function or a power law function. I'm aware that a chi-squared test may be a suitable ...
1
vote
1answer
45 views

Conditional distribution for Exponential family

We have a random variable $X$ that belongs to the exponential family with p.d.f. $$ P_X(x|\boldsymbol \theta) = h(x) \exp\left(\eta({\boldsymbol \theta}) . T(x) - A({\boldsymbol \theta}) \right) $$ ...
3
votes
1answer
69 views

Proving a distribution is a member of the simple exponential family

Does anyone have any tips/ideas/method for proving that a distribution is a member of the simple exponential family (SEF)? Or is the process unique to each distribution? For example, I am trying to ...
4
votes
1answer
54 views

How to derive the conjugate prior of an exponential family distribution

I am trying to derive the conjugate prior of the univariate Gaussian distribution over both the mean and the precision. I know that the prior I'm looking for is the normal-gamma distribution, but the ...
2
votes
1answer
73 views

ML estimate of exponential distribution (with censored data)

In Survival Analysis, you assume the survival time of a r.v. $X_i$ to be exponentially distributed. Considering now that I have $x_1,\dots,x_n$ "outcomes" of i.i.d r.v.'s $X_i$. Only some proportion ...
4
votes
1answer
62 views

Exponential family distribution with high-order statistics

An exponential family distribution in its simplest form is given by $p(x|\theta) = \exp(\theta^\top T(x) - A(\theta))$ where $T(x)$ is a vector of sufficient statistics, $\theta$ is its natural ...
1
vote
0answers
26 views

Families of distributions with bounded variations

For what family of probability distributions $f(x)$ do we have the following property? $$ \forall x, \quad \int f(u) - f(u+x) du \leq L\| x \| $$ for some $L$. Can we say anything about the ...
0
votes
0answers
24 views

Scaling of a Weibull variate

I am currently working on an inter-event time dataset. These times are Weibull distributed (I verified my fit with a KS-Test). However, I need to subdivide them into different phases, that are in turn ...
5
votes
1answer
150 views

Why the mixtures of conjugate priors is important?

I have a questions about the mixture of conjugate priors. I learnt and say the mixture of conjugate priors a couple of times when I am learning bayesian. I am wondering why this theorem is such ...
1
vote
1answer
46 views

How to prove Bernoulli distribution belongs to the exponential family

According to a book, a distribution belongs to the exponential family if it can be written in the form of I wrote the Bernoulli distribution as $\exp\Big(y \log\,[{\mu}/{(1-\mu)}] + ...
0
votes
1answer
64 views

MLE for two-parameter exponential distribution

I have to find the parameters of a two-parameter exponential distribution using the MLE. But imposing FOC I do not find enough conditions to found both the paramenters. Any hint? Thanks!
0
votes
0answers
23 views

log-linear models and exponential models?

What is usually referred to as "log-linear models"? Is a log-linear model an exponential model where the normalization constant is 1? (since its logarithm needs to be a linear function.) Or is there ...
10
votes
1answer
296 views

When if ever is a median statistic a sufficient statistic?

I came across a remark on The Chemical Statistician that a sample median could often be a choice for a sufficient statistic but, besides the obvious case of one or two observations where it equals the ...
2
votes
0answers
112 views

“weight” input in glm.nb function in R. How exactly does the weight affect the likelihood?

I would like to understand how the weight argument of glm.nb is affecting the likelihood function. I understand that glm.nb find the MLE in an alternating iteration process where for a given theta the ...
1
vote
0answers
114 views

What is relationship between Fisher Information and Variance in natural exponential Family?

I know that $Var(\hat\theta)\geq 1/I(\theta)$ where $I(\theta)$ is Fisher information. Let take an example of natural exponential family with density $f(x)=\lambda\exp(-\lambda x)$. In this case we ...
0
votes
0answers
9 views

Is $A(\eta)$ function in Canonical exponential family differentiable?

I'm studying canonical exponential family, and I'm stuck in differentiability of $A(\eta)$. There's no assumption or comments about $A(\eta)$ in canonical exponential family. (I struggled to search ...
0
votes
0answers
21 views

What is a convex support? (Bickel&Doksum, Mathematical Statistics, Basic ideas…Vol1)

Bickel&Doksum, Mathematical Statistics, Basic ideas...Vol1 page 122, just above Cor2.3.1, it says: Define the convex support of a probability P to be the smallest convex set C such that ...
8
votes
1answer
73 views

Is the negative binomial not expressible as in the exponential family if there are 2 unknowns?

I had a homework assignment to express the negative binomial distribution as an exponential family of distributions given that the dispersion parameter was a known constant. This was fairly easy, but ...
1
vote
2answers
127 views

Best Fit for Exponential Data

I'm trying to better understand some of the theory behind fitting models that have a nonlinear link between the response and the predictors. ...
8
votes
2answers
479 views

What is meant by the term 'exponential family'? Why it is named so?

I have come across the term exponential family. The Bernoulli, Gaussian and many more distributions come under this exponential family. What would be the commonalities between them?
5
votes
1answer
238 views

Exponential Family: Observed vs. Expected Sufficient Statistics

My question arises from reading reading Minka's "Estimating a Dirichlet Distribution", which states the following without proof in the context of deriving a maximum-likelihood estimator for a ...
6
votes
1answer
67 views

The distribution of a sufficient statistic

If I understand correctly, a distribution in the exponential family... $$\underline X\sim f_{\underline\theta}(\underline x) = exp\{\sum\limits_{i}\eta_i(\underline\theta)T_i(\underline ...
0
votes
1answer
66 views

How to show linear model corresponds to exponential family?

I am confronted with the exercise below. I have no given solution, so I hope someone can tell me whether my solution is right or wrong. I want to show the deterministic linear model $\quad ...
0
votes
0answers
24 views

Understanding Conjugate families

I'm trying to understand conjugate families of distributions, and am hoping someone could give me a hint or direct to a good paper (or book) covering basics. Or perhaps, simple example would help? ...
2
votes
1answer
76 views

What the dimension of an exponential family tell us about that family?

In Wikipedia it is stated that: A vector exponential family is said to be curved if the dimension of $$ {\boldsymbol \theta} = \left (\theta_1, \theta_2, \ldots, \theta_d \right )^T$$ is ...
0
votes
0answers
46 views

sufficient statistic and KL-divergence: Confusion with an equation

I am reading a paper, which talks about minimising KL-divergence of any arbitrary distribution over a family of exponential distribution. So, given a distribution $p$, we want to compute its ...
8
votes
2answers
227 views

A stochastically increasing exponential family for which $\lim_{\theta\rightarrow\inf\Theta}\mbox{P}_\theta(X\leq x)\neq 1$

Question A little something that I've been wondering about for a while: Let $P_\theta$ be a stochastically increasing (one-parameter) exponential family on the sample space $\mathcal{X}$ with ...
0
votes
0answers
175 views

Sample size for a non-normal distribution

I'm quite new in this field. I hope my question makes sense. I have a database that stores information for around 10.000.000 projects. Each project has several features (let's call them X) like number ...
7
votes
1answer
258 views

Do the mean and the variance always exist for exponential family distributions?

Assume a scalar random variable $X$ belongs to a vector-parameter exponential family with p.d.f. $$ f_X(x|\boldsymbol \theta) = h(x) \exp\left(\sum_{i=1}^s \eta_i({\boldsymbol \theta}) T_i(x) - ...
1
vote
0answers
41 views

Sequence of Bernouilli trials with diminishing returns

Say I have a sequence of yes-no trials where the probability of winning decays over time as the supply of wins is gradually exhausted. Assume that for every trial the probability of replacement is not ...
8
votes
1answer
432 views

Does log likelihood in GLM have guaranteed convergence to global maxima?

My questions are: Are generalized linear models (GLMs) guaranteed to converge to a global maximum? If so, why? Furthermore, what constraints are there on the link function to insure convexity? ...
4
votes
1answer
288 views

Show that Weibull distribution can be transformed to exponential family

How do I show the Weibull distribution $f:(y; \lambda, \rho)$ can be transformed to the exponential family using the transformation $z=y^\lambda$? I know the form I need to express it in is ...
10
votes
1answer
244 views

Does a canonical link function always exist for a Generalized Linear Model (GLM)?

In GLM, assuming a scalar $Y$ and $\theta$ for the underlying distribution with p.d.f. $$f_Y(y | \theta, \tau) = h(y,\tau) \exp{\left(\frac{\theta y - A(\theta)}{d(\tau)} \right)}$$ It can be shown ...
2
votes
1answer
1k views

How to find the expected Fisher information for a parameter of exponential distribution in an iid sample?

I am unclear to on how to find the Fisher information of $\lambda$ in the iid sample $X_1,...,X_n$. Attempt: I know that the fisher information for $X$ (not in a sample) would be: $I(\lambda) = - ...
3
votes
0answers
56 views

Exponential family where set of natural parameters has empty interior

In my math-stat class we have a theorem that goes: Let $\{P_\theta : \theta \in \Theta\}$ be a $k$ parameter exponential family (i.e. the density of a member of this family can be written as ...
1
vote
0answers
147 views

Most Powerful Test; Two-Parameter Normal Distribution

Is it possible to show that the two-parameter Normal distribution has monotone likelihood ratio? EDIT: This is actually part of a larger problem. We have a random sample from $\mathcal N(\mu, ...
0
votes
0answers
91 views

Hypothesis testing with exponential family

I'm interested in running hypothesis tests for a variety of members of the exponential family with continuous support, for different values of the parameter/s, for a sample of n i.i.d random variables ...
1
vote
0answers
62 views

Computing non-central moments and normalizer of a quartic exponential distribution

Consider a random variable $X$ which has quartic exponential distribution: $$X \sim P(x)=\frac{1}{Z}e^{ax + bx^2 + cx^3 + dx^4}$$ How can one compute $Z$ or non-central moments $E X^k$ given that they ...
0
votes
0answers
92 views

Normalising Constant for exponentiated function

What would the normalising constant be of the following, or atleast an approximation? I would like to avoid sampling. $$f(\theta)=\exp(-k_1e^{-k_2\theta^2}-\theta^2)\qquad\theta\in(-\infty,\infty), ...
0
votes
0answers
24 views

normalising constant on exponential of exponential [duplicate]

I have a distribution of the form $\exp(-\exp(-x^2))$. Is this a known family of distributions. Otherwise how would you find/approximate the normalising constant. The domain is $x \in (-\infty,\infty) ...
3
votes
0answers
60 views

Exponential family parameter estimation and fitting, references

First of all, I want to express my apologies if the question is too broad or wrong, but I am in need of references and I have no idea whom I can ask. If you are interested, the question comes from a ...
6
votes
1answer
125 views

Checking whether a density is exponential family

Trying to prove that this doesn't belong to exponential family. $f(y|a)=4\frac{(y+a)}{(1+4a)} ; 0 < y < 1 , a>0$ Here is my approach: $$f(y|a) = 4(y+a)e^{-log(1+4a)}$$ $$f(y|a) = ...
2
votes
0answers
129 views

3 parameter Exponential Family and sufficient statistics

This is a homework problem. I've derived the following distribution from an earlier part in the problem $$ f_{X_1,X_2}(x_1,x_2) = ...
3
votes
0answers
302 views

Is a normalized version of an exponential family distribution still an exponential family distribution?

Is a normalized version of an exponential family distribution still an exponential family distribution? Here "normalized" means making its mean zero and variance one. According to the following ...
3
votes
2answers
102 views

What to do in logistic regression if you have a huge amount of variables?

I am dealing with logistic regression, trying to identify variables which have a causal relationship with a binary response. The way I usually do it is to try variables one by one and visualize the ...
2
votes
1answer
2k views

How should you express a negative binomial distribution in an exponential family form?

When $X$ is $X_1$,...$X_n$, how do you express the following negative binomial distribution in an exponential family form? $$ ...