All Questions
712 questions with no upvoted or accepted answers
3
votes
1
answer
59
views
Is this definition of likelihood presented in an arxived article correct?
In a recently arxived article the following definition of the likelihood is given:
The Bayes rule provides the posterior probability $p(h_i|o)$ for each hypothesis given the observation:
$$p(...
- 4,362
3
votes
0
answers
286
views
Likelihood ratio tests for quasi- models
I have been playing around with over-dispersion in binomial data and looking into qausi-binomial models as a solution. When comparing binomial models through the change in deviance, I can reproduce ...
- 31
3
votes
0
answers
104
views
Combining "unbalanced" likelihoods in a "process-based" model
I have a "process-based" water quality model, which is essentially a black-box full of differential equations describing various chemical and hydrological processes. The model is ...
- 478
3
votes
0
answers
153
views
Computing the CDF of the minimum of particular dependent random variables
For each $i=1,\dots,n$ let $Z_i\sim\text{Poisson}(\lambda_i)$, and suppose $\{Z_i\}$ are independent. Also for each $i=1,\dots,n$, let $\{Y_{ij}\}_{j\in\mathbb{N}}$ be an infinite sequence of iid ...
- 133
3
votes
0
answers
684
views
If the AIC is an estimate of Kullback-Leibler Divergence, then why can AIC be negative when KL divergence is always positive?
I have read many times that the AIC serves as an estimate of the KL divergence, and I know that AIC can be a negative value (and have seen that myself). Yet, the KL divergence must always be positive. ...
- 420
3
votes
0
answers
221
views
Likelihood with random censoring
Suppose to observe a random sample from a r.v. $Y_i=\min(T_i,C_i)$ where $T$ and $C$ are iid absolutely continuous distribution.
I would like to inference about a parameter of $T$ (for example, $\...
- 125
3
votes
1
answer
93
views
Minimum of Poissons
Let $X_i\sim\text{Pois}(\lambda_i)$ for $i=1,2,\ldots,n$ and $Y = \min X_i$. Can we show that, for example $\mathbb{E}[Y] \leq f(\lambda,n)\min\lambda_i$ for some $f : (\mathbb{R}^n,\mathbb{N}) \to [0,...
3
votes
0
answers
811
views
Likelihood Ratio Test for one-sided hypothesis
Let $X_1,\dots, X_n$ be a random sample from $N(\theta,\sigma^2)$, and we would like to test $H_0:\theta \le \theta_0 \text{ vs } H_1:\theta>\theta_0$. Assume $\sigma^2$ is known and derive a ...
- 31
3
votes
0
answers
418
views
Expected value of maxima of dependent random variables
I don't know if such theorem exists, but what I am looking for is a closed-form solution for $$E[\max(X_1, ..., X_N)]$$
where $X_1, ..., X_N$ is a sequence of dependent identically distributed ...
- 492
3
votes
0
answers
685
views
What's the use of the expected fisher information matrix over the hessian in the Newton Raphson approach to finding the MLE?
This may be a naive question, but I'm looking at the Newton Raphson iterative approach ( i.e. using the formula $\boldsymbol{\theta }^{(j+1)} = \boldsymbol{\theta }^{(j)} + \textrm{Hess}_{-\ell}(\...
- 31
3
votes
0
answers
118
views
Marginal likelihood for a half-normal posterior?
So I know if we have a normal likelihood $P(\mathbf{y|b}) = \mathcal{N}(\mathbf{y}|\mathbf{Gb}, \mathbf{\Sigma}_y)$ and a normal prior $P(\mathbf{b}|\mathbf{\theta}) = \mathcal{N}(\mathbf{b} | \mu_p, \...
- 613
3
votes
0
answers
394
views
Covariance term in Gradient of Gaussian Process marginal likelihood
log marginal likelihood for Gaussian Process as given by Rasmussen's: Gaussian Processes for Machine Learning equation 5.8 is
$$\log p(y|X, \theta) = -\frac{1}{2}y^{T} K_y^{-1}y - \frac{1}{2}\log|...
- 573
3
votes
0
answers
85
views
An arithmetic mean preserves normal distributions, maximum preserves Frechet/Gumbel/Extreme Value distributions, but what about all other power means?
Let the $k$-power mean of two numbers $x$ and $y$ be defined as $M^k(x,y) = \left(\frac{x^k+y^k}{2}\right)^{1/k}$.
For the case $k=1$, we have that if $X,Y$ are independently normally distributed, ...
- 1,594
3
votes
1
answer
439
views
Nested sampling integral on a previously obtained MCMC sample
This question concerns the calculation of the evidence, or marginal likelihood, from an existing MCMC sample without having to resample. There is an exhaustive and very helpful answer here:
[...
- 31
3
votes
0
answers
2k
views
Why do we use prediction error decomposition for the derivation of the likelihood function for AR(p)
A good example of deriving a likelihood function is the normal distribution:
The PDF of the normal distribution is:
$$
f(x;\mu, \sigma) = \frac{1}{\sigma\sqrt{2\pi}}\exp\left[\frac{(x - \mu)^2}{2\...
user124589
3
votes
0
answers
194
views
Distribution of the minimum of the squared Euclidean norm of a $N(\mu,\Sigma)$ random variable
Suppose that $X^n := \{x_1, x_2, \ldots, x_n\}$ is a sample of $n$ i.i.d $p$-dimensional points, where $X \sim N(\mu, \Sigma)$.
What is known about the distribution of $\min_{x_i \in X^n} \|x_i\|^2_2$?...
- 802
3
votes
0
answers
228
views
Log likelihhood ratio with soft information
The log likelihood ratio of two binary random variables $x$ and $y$ (only taking values $0$ or $1$) can be defined as
$LLR_x=\frac{P(x=0|y)}{P(x=1|y)}$
The above definition is OK if $y=0$ or $1$.
...
- 31
3
votes
0
answers
177
views
What is the likelihood of drawing a sample with standard deviation $s$ from a normal distribution?
I was reading Think Bayes book and chapter about Approximate Bayesian Computation where the author uses a bayesian approach to calculate mean $\mu$ and standard deviation $\sigma$ of height in US ...
- 133
3
votes
0
answers
151
views
Deviance in a distribution that has a shape parameter
Consider the following log-likelihood function
$$
\ell (\vec{\mu}, \rho \mid \vec{y}) =
\sum_{i=1}^I \log f(\mu_{i1}, \dotsc, \mu_{iJ}, \rho \mid y_{i1}, \dotsc, y_{iJ})
$$
where $\vec{y}$ ($I \...
- 2,392
3
votes
1
answer
52
views
How to make a confidence interval in a binomial model with fakes?
I'm writing a concept recognition self-test. (This is less weird than it sounds, but only slightly. Don't worry about it.) Partly to prevent people from clicking everything in sight, but mostly just ...
- 1,248
3
votes
0
answers
128
views
Extreme value theory: GPD larger expected value than average
We're using extreme value theory to model tail risks on our portfolio. After we choose the threshold, we fit generalized Pareto distribution to our data over the threshold. The expected value of GPD ...
- 371
3
votes
0
answers
149
views
Predictive modeling of an complex panel of heavy-tailed data
I am struggling to develop a sensible strategy or protocol for the predictive modeling of a complex set of data. Apologies in advance for the indeterminate nature of some of this description but it’s ...
- 10.9k
3
votes
0
answers
123
views
Likelihood function for spatial Poisson
The dependent variable in our data is the integer number of trips made in a time period. We can regress on socioeconomic attributes of the individual $X$. The most logical model for this is a Poisson ...
- 3,472
3
votes
0
answers
96
views
Properties of Average Multinomial Likelihood
I am trying to understand the Kullback-Leibler Information:
I read in http://arxiv.org/pdf/1404.2000v1.pdf the following:
Ideally, we want the probability to be invariant to the number of ...
- 1,824
3
votes
0
answers
156
views
How to optimize the choice of cointegration coefficient?
Testing for cointegration on the linear combination of time series vectors can be done by testing the error term for a unit root. $$Y_t - \gamma X_t = \varepsilon_t$$
In the bivariate case the ...
- 1,967
3
votes
0
answers
285
views
Derivation of likelihood function for latent variable model made explicit
I am trying to make the steps deriving the likelihood function for the following latent variable model as explicit as possible: $$Y^0=X\beta + u$$ where $$u \sim NID(0,\sigma^2).$$ The observed data ...
- 502
3
votes
0
answers
120
views
Problem with Finding Likelihood: Bayesian
I am really unfamiliar with Bayesian methods particularly parameter estimation.
Suppose I have a test to find a parameter, theta which is the number of packaged bag for retail sale that could contain ...
- 131
3
votes
0
answers
219
views
Marginal Likelihood Latent Variable Model
I am trying to apply the method proposed by Chib in Marginal Likelihood from the Metropolis Hastings Output to calculate the marginal likelihood of a logit model the includes latent variables. ...
- 31
3
votes
0
answers
249
views
Extremal serial dependence
As part of my analysis of heavy-tailed time series of company returns, I would like to check whether extreme returns exhibit serial dependence, i.e. if extreme events are followed by extreme events.
...
- 161
3
votes
0
answers
770
views
GEV of Normal Distribution and relationship of the parameters
My question goes on Extreme Value Theory for the Normal distribution (www.math.ethz.ch/~embrecht/RM/chap7.pdf):
Which type of GEV (Generalized Extreme Value) distribution does the Normal distribution ...
- 1,271
3
votes
0
answers
167
views
The probability that one bernoulli process has a higher p than another?
I have two data generating processes that are independent Bernoulli processes with probabilities of success $p_A$ and $p_B$. I am taking repeated samples from these two data generating processes, so ...
- 151
3
votes
0
answers
107
views
Repairable system and the sum of GEV random variables
We know that $X\sim {\mathrm {GEV}}(\alpha ,\beta ,0)$ and $Y\sim {\mathrm {GEV}}(\alpha ,\beta ,0)$ then $X+Y\sim {\mathrm {Logistic}}(2\alpha ,\beta )$.
I am wondering, what will be $X+Y+Z$ ...
- 328
3
votes
0
answers
671
views
Conceptual or mathematical motivation for the three extreme value distribution types?
What motivates, justifies, gives rise to the differences between the Gumbel, Fréchet, and Weibull distributions? Glen_b's comment indicates that they are distributions for extreme values generated by ...
- 1,108
3
votes
0
answers
207
views
Formal statistical test for comparing likelihood distributions obtained via MCMC
I am trying to formally compare the distribution of the likelihood values generated using two different models with marginal posterior values of the parameters obtained using MCMC in order to assess ...
- 31
3
votes
0
answers
284
views
Difference between Maximum a posteriori and penalized likelihood
Can anyone explain to me the difference between penalised likelihood and maximum a posteriori?
I read a paper where the likelihood function is
$$L(\theta_1, \theta_2,\theta_3 ; x)=f(x|\theta_1, \...
- 51
3
votes
0
answers
1k
views
Confidence intervals for extreme value distributions
I have wind data that i'm using to perform extreme value analysis (calculate return levels). I'm using R with packages 'evd', 'extRemes' and 'ismev'.
I'm fitting GEV, Gumbel and Weibull distributions,...
- 951
3
votes
0
answers
142
views
Distribution of variable
How to find the distribution of $$\sum_{i=1}^n (X_i - X_{1:n}),$$ where $X_i$ are i.i.d. random variables and $X_{1:n} = \min(X_1,X_2,...,X_n)$?
I need to find the distribution in a particular case, ...
- 31
3
votes
0
answers
116
views
Confusion related to calculation of likelihood
I was reading this paper related to Learning from multiple annotator using Gaussian processes. The idea is if we don't have the actual ground truth of a certain data, but only the labels from some ...
- 6,847
3
votes
0
answers
352
views
Correct variance for minimum detectable difference
I have a question regarding variance, paired testing and minimum detectable difference (MDD).
Paired samples:
$$
MDD (δ) = \sqrt{ \frac{σ^2}{n} (t_{(α/2,n-1)} + t_{(1-β, n-1)})}
$$
I have a set of ...
- 31
3
votes
1
answer
361
views
Difference between likelihood functions for pmf vs pdf
Can someone explain the intuition behind how the likelihood function for a specific value of $\theta$ is different if $f_\theta$ is a pmf vs a pdf?
I thought that it was simply the probability that a ...
- 31
3
votes
1
answer
1k
views
How to minimize Chi-Square using the CDF instead of the PDF?
Suppose one has data that is suspected to obey a normal distribution. One computes a histogram of the data, and performs Pearson's Chi-Squared Test. To perform this test, one must compare the observed ...
user146123
2
votes
0
answers
70
views
Can an outcome variable be used twice in the same model?
When is it appropriate to use the same outcome variable in two likelihoods in the same model framework?
Here is a specific example:
...
2
votes
0
answers
37
views
To what extent can likelihood methods be used for functional responses?
Let's suppose that we are working with a functional data set, $Y_i(t)$, $Y_i\in L^2[0,1]$, $1\le i\le n$. If we were working with univariate or even multivariate data set, likelihood methods would ...
- 1,413
2
votes
0
answers
164
views
Random effects model and log likelihood
Hello I am struggling a bit understanding the random effects model.
What I understood is that we fit a model but allow for sysematic differences of the variance of residuals per group.
I would ...
- 1,783
2
votes
0
answers
56
views
Unbiased estimate of log-likelihood of Markov bridge
Note: I have cross-posted this question to MathSE.
I have the following problem I am trying to solve. I have a parametric family of "transition" distributions $p_\theta(x_{i+1}\mid x_i)$ and ...
2
votes
1
answer
38
views
Relation between sample standard deviation from data and maximum likelihood estimates
This is my data:-
c(3164, 3362, 4435, 3542, 3578, 4529)
I estimated its sample mean and standard deviation via mean & ...
2
votes
0
answers
65
views
Likelihood function of VAR-MGARCH-BEKK model?
I am doing my dissertation on the spillover effect between countries' markets and looking to use VAR-MGARCH model to do it. For example how would a change/shock of US market index affect Thailand ...
- 21
2
votes
0
answers
158
views
Definition of exponent measure (extreme value theory)
Let $F$ be a distribution function on $\mathbb{R}^2$, and let $U_i$ be the left continuous inverse of $\frac{1}{1-F_i}$, where $F_i$ is the marginal distribution of $F$.
In my textbook, there is the ...
- 656
2
votes
0
answers
102
views
Implementing a 2-PL Dichotomous IRT Module in Python from scratch
I am trying to implement a 2-PL dichotomous IRT Model for my dataset from scratch in Python. Here is my code so far:
...
- 121
2
votes
0
answers
448
views
Zero-inflated Poisson - Implementing INLA with two likelihoods
I am trying to implement a zero inflated model in INLA.
I know a basic zero inflated Poisson can be implemented with "zeroinflatedpoisson1" as the family ...
- 121