Questions tagged [likelihood]

Given a random variable $X$ which arise from a parameterized distribution $F(X;θ)$, the likelihood is defined as the probability of observed data as a function of $θ: \text{L}(θ)=\text{P}(θ;X=x)$

Filter by
Sorted by
Tagged with
1
vote
0answers
21 views

How to deal with 'division by zero' and 'logarithm of zero' in simulations for finding the maximum log likelihood

I am trying to run a program which generates data from various covariate distributions, and finds the maximum likelihood estimator by explicitly maximising the log likelihood function. I am ...
0
votes
0answers
6 views

Finding likelihood of distribution of events across categories

Things I have : Categories: [A, B, C,....., Z, A1, B1, C1......., Z5] Likelihood of co-occurence of 2 categories : ...
1
vote
0answers
39 views

Estimating datapoint from log-likelihood function evaluations

Imagine that we are given multiple evaluations of a likelihood function on a datapoint for several samples of model parameters (coming from their prior), and this datapoint is hidden from us. Under ...
0
votes
0answers
16 views

Behaviour of likelihood ratio test as sample size increases

I know that the log likelihood statistic, i.e. $X^2=-2(LL_1 - LL_2)$ where $LL_1$ and $LL_2$ are the maximum log likelihoods of two models, one nested in the other, is asymptotically distributed as a ...
1
vote
1answer
49 views

Why is the likelihood a product of pdf terms $f(\theta; x_1, x_2, …)$ [duplicate]

Before anyone says this has been answered elsewhere I don't think it has. The likelihood is given by: $$ L(θ;x_1,\cdots, x_n) = \prod^n_i f(x_i\mid\theta)$$ where $f$ is the probability density ...
1
vote
1answer
64 views

Reducing dataset size in likelihood-free inference

What difference does it make working with a big or small dataset in ABC? Do we get any computational benefits by reducing a very big dataset when doing inference using ABC methods? My understanding is ...
0
votes
0answers
20 views

AIC / BIC for Model Seleciton in Copula Model

I'm trying to select the distributional model of 30 marginals (which are restricted to have the same distributional family) in a copula model. However, I therefore get 30 Likelihood/AIC/BIC values for ...
0
votes
0answers
15 views

Interpretation of likelihood function of a continuous random variable in the context of Bayesian Information Criterion

I am confused about the interpretation of the likelihood function of continuously distributed random variable. It's my understanding that the likelihood function is defined to be $L(\theta|x) = f_\...
1
vote
1answer
41 views

State Space models: rewriting the Likelihood to estimate the covariance matrix

I have a State Space model $ \begin{matrix} Y_t & = & FX_t +R_{t}^{1/2}\epsilon_t \\ X_{t+1} & = & GX_{t}+Du_t \end{matrix} $ where $\epsilon_t$ and $u_{t}$ distributed ...
0
votes
1answer
45 views

What happens to the log likelihood when the maximum likelihood estimate does not exist?

What happens to the log likelihood (or indeed the likelihood) function, when the MLE does not exist? The log likelihood is defined (for independent observations) as $$l(\boldsymbol{\theta}) = \...
2
votes
1answer
109 views

Why should there be two solutions for each parameter of likelihood ratio equation for Weibull-distribution?

I try to calculate the confidence ínterval of a Weibull distribution by means of the Likelihood method as described in ReliaWiki. In order to find the confidence intervals, I have to solve the ...
1
vote
2answers
92 views

Conceptual explanation of Maximum Likelihood Estimation

Given a generic time-series $$y_{t+1}= \alpha y_{t} + \Sigma_{t+1}^{1/2}\varepsilon_{t+1} \quad \text{with} \quad \varepsilon_{t+1}=N(0, I)$$ where $\Sigma_{t+1}^{1/2}$ indicates the conditional ...
1
vote
0answers
20 views

How to simultaneously maximize multiple likelihoods?

This is a more generic question. Assuming I have closed-form likelihood functions of 2 related observations|distributions (for e.g. likelihood of observations from 2 MVN dists). However, because the ...
2
votes
0answers
13 views

Identifiability of parameters in a linear model when covariates are random

Suppose we have a linear model (in $\mathbb{R}^n$, say), $$y = X\beta + \epsilon $$ where $\bf{\epsilon}$ is Gaussian with mean $0$ and covariance matrix $\Sigma(\theta)$ where $\theta$ is an unknown ...
0
votes
0answers
18 views

MLE vs Expectation Maximization to estimate time-changing parameters in state space model

Suppose I have a generic model in state-space form described as $$x_{t+1}=\phi_{t} x_{t}+w_{t+1}\epsilon_{t+1}$$ $$y_{t}=H_{t}x_{t}+v_{t}e_{t+1}$$ where both $e_{t+1}$ and $\epsilon_{t+1}$ are iid ...
0
votes
0answers
7 views

How to deduct the complete likelihood of mixture normal in EM algorithm

We have the well known complete likelihood of mixture normal in EM algorithm: Here $Z$ is a random variable that it has probability $\pi_k$ to choose k-th normal variable $X_k:N(\mu_k,\sigma_k).$ We ...
0
votes
0answers
44 views

Noncentral t-distribution — relationship to shifted/scaled normal distribution

Let $x$ be 100 random samples from a $N(10,4)$ distribution. Suppose that I want to calculate the likelihood of these data, given my knowledge that $\mu=10,\sigma=4$. For the normal distribution, this ...
1
vote
1answer
23 views

Clarification on Akaike's IC (AIC) and BIC for Expectation Maximization with time-changing parameters

I apologize in advance for the trivial question, but I need a clarification on the following issue. Suppose I have a generic model in state-space form described as $$x_{t+1}=\phi_{t} x_{t}+w_{t+1}$$ $...
0
votes
1answer
20 views

Likelihood of (multivariate) normal distribution

Given a data point $x$ and a possibly multivariate normal distribution $N_1$ with known mean and variance-covariance matrix, it is trivial to compute the likelihood of the data point $x$ given the ...
1
vote
3answers
69 views

Bayesian Inference: Feeding Posterior back in as Prior

I've just started reading about Bayesian Inference, and one thing I've wondered about is if it's possible to feed the posterior in as a new prior for a new model, using the same data. Or would that ...
0
votes
0answers
14 views

How to infer the number of unique parameters when fitting data to a multinomial model?

Suppose I have data I think follow a multinomial distribution, where I have N observations, and at each observation, some number of elements are observed to be distributed among some fixed number of ...
1
vote
1answer
46 views

Computing the accuracy of an answer

Let's assume I have three experts providing an answer. Expert 1 is 95% accurate (The likelihood of providing a correct answer is 95%). Expert 2 is 90% accurate Expert 3 is 85% accurate They are ...
1
vote
1answer
49 views

Is there a name for the concept that is the generalisation of probability and likelihood? [closed]

Given some model parameter $\theta$ and some data $x$, the model distribution is written as $p(x\vert \theta)$ and the likelihood as $L(\theta\vert x)$. Is there some formalised more general concept $...
2
votes
1answer
44 views

Degenerate multivariate normal in Maximum Likelihood Estimator (Akaike's Info Criterion, BIC, LR Test usage)

Let's suppose that the considered set of random variable has a covariance matrix which is psd. Therefore the Gaussian pdf must be written in its degenerate form, where the determninat of the ...
2
votes
2answers
102 views

Expectation of the log-likelihood under the posterior

Suppose $L(X \mid \theta)$ is a likelihood function, i.e., a probability distribution over $X \in \mathcal{X}$ indexed by a parameter $\theta \in \Theta$. Suppose further we have a prior $\pi(\theta)$,...
1
vote
1answer
63 views

Bayesian inference for Beta distribution after an uncertain outcome

Normally, when we have $$p\sim Beta(a,b)$$ and an observation of $x=1$ (''success'') from a Bernoulli trial with ''success'' probability $p$, the Bayesian inference on the parameter value $p$ is $$p|x\...
0
votes
1answer
23 views

Gaussian process classification / non-Gaussian likelihood / causes for that

according to chapter 3 from 1-book, in case if there are discrete class labels, then the Gaussian likelihood is inappropriate. What are the exact reason for that and what would I reach if I use ...
0
votes
0answers
19 views

What is a “surface” and the “likelihood”?

On Neyman & Pearson, 1933, page 302, Then the family of surfaces of constant likelihood, $\lambda$, appropriate for testing a simple hypothesis $H_0$ is defined by $$ p_0 = \lambda p(\...
2
votes
1answer
63 views

Does the probability density function remain normalized after marginalization?

Let's say, we have data $D$ and two parameters $A$ and $B$ in a model. I want to marginalize the likelihood $p(D \mid A, B)$ over $B$ as $$ p(D \mid A) = \int p(D \mid A, B) p(B) {\rm d}B. $$ I know ...
0
votes
0answers
39 views

Why is missing data likelihood a likelihood with a smaller joint distribution?

Suppose that the full likelihood for a given multidimensional data point is $L(\theta; y_1,..,y_n,..y_m) = f(y_1,..,y_n,...,y_m | \theta)$ If for the data point $(y_{n+1},...,y_m)$ is missing, then ...
2
votes
0answers
32 views

What is the likelihood function of the starting time of diffusion?

I need to find the likelihood that a set of molecules was instantaneously released at time $t_0$, say $t_0=0$. Toy System Example: Let $N$ be the set of molecules released from a specific point in a ...
0
votes
0answers
21 views

Solving an equation for linear local likelihood density estimation

I get the following system of non-linear equations by using a Guassian kernel and differentiating the local likelihood equation for the case where the "global distribution" is approximated using a ...
8
votes
2answers
187 views

Why not to use Bayes theorem in the form $p(\theta | x) = \frac{L(\theta | x) p(\theta)}{p(x)}$?

There are a lot of questions (like this) about some ambiguity with Bayesian formula in continuous case. $$p(\theta | x) = \frac{p(x | \theta) \cdot p(\theta)}{p(x)}$$ Oftentimes, confusion arises ...
0
votes
1answer
22 views

Order of observations in calculation of likelihood

I am afraid this will be quite obvious once you tell me, but I'm just confused right now. Say we have an i.i.d. RV $N\sim Poisson(\mu)$. Now, I want to calculate the likelihood $L$ of an observation ...
2
votes
1answer
57 views

When to use the full and the conditional likelihood

In the context of estimating parameters of a time series model, we may consider either the full likelihood or the conditional likelihood. I was wondering in what situations the full ...
2
votes
1answer
47 views

Is this definition of likelihood presented in an arxived article correct?

In a recently arxived article the following definition of the likelihood is given: The Bayes rule provides the posterior probability $p(h_i|o)$ for each hypothesis given the observation: $$p(...
3
votes
4answers
51 views

How to make sense out of integration over discrete data points?

Looking for a proof of the expected value of the score function equating zero, I came to this document that was recommended in another answer. Considering that we have a sample of n x_i values, I ...
1
vote
0answers
17 views

Total variance on binomial statistic with uncertain probability

Consider an experiment to measure a random occurrence with a probability $p$ to pass some selection criteria. If I have $N$ independent trials and $p$ is known, this is a standard Binomial ...
0
votes
0answers
24 views

How to deal with very small probabilities in likelihood? [duplicate]

Similar as in this thread, I have a problems with small probabilities in likelihoods, but I believe the thread does not apply to me. The likelihood is a product of probabilities. The log-likelihood ...
0
votes
0answers
14 views

Maximum likelihood second derivative test

Can anyone explain what to do if the maximum likelihood second derivative test comes back positive such that the M is a saddle point instead of a maximum value? What do I do after that to figure out ...
1
vote
0answers
52 views

Likelihood Ratio Test for Equality of two proportions

Suppose we have two i.i.d samples from $X_1,...,X_n$ and $Y_1,...,Y_n$ where $$X_1,...,X_n \sim B(n,p_0)$$ $$Y_1,...,Y_n \sim B(n,p_1)$$ I want to show that we get the familiar test of equality of ...
1
vote
1answer
53 views

Likelihood ratio test for GENLINMIXED in SPSS?

I am analyzing some linguistic data using SPSS 25, and have encountered some problems. Please note that I am not a statistician, so excuse me if my question is trivial. Basically, I am using the Mixed ...
0
votes
0answers
26 views

Differentiation with respect to beta j?

I'm not sure how to differentiate the log-likelihood with respect to beta j, so I can find the MLE? Can anyone help me?
0
votes
0answers
18 views

Prove that $T(\textbf{X}) = \hat{\sigma}^{2}$ reaches the Cramer-Rao bound

Let $X_{1},X_{2},\ldots,X_{n}$ be a random sample whose distribution is given by $\mathcal{N}(\mu,\sigma^{2})$, where both parameters are unknown. (a) Prove the normal probability density function ...
1
vote
1answer
34 views

deriving likelihood function for hierarchical bayesian model

I'm struggling with hierarchical bayesian modeling. I need to derive a full likelihood function for the given hierarchical structure of the model. $a_{it}|\lambda_i\sim TN(\lambda_i,\beta)$ $x_{it}|\...
0
votes
0answers
24 views

Likelihood ratio statistic lognormal

I want to determine the LRS lambda of a lognormal distribution under H0: the variances are equal. What I have so far is the following: $H_0: \sigma^2 = \sigma_{o}^{2}$ $H_1: \sigma^2 \neq \sigma_{o}^...
0
votes
1answer
68 views

How to prove or disprove that $T(X_{1},X_{2}) = X_{1} + X_{2}$ is a sufficient statistic

Let $X_{1},X_{2},\ldots,X_{n}$ be random sample from a population whose distribution is given by $X\sim\text{Bernoulli}(\theta)$, $0 < \theta < 1$. a. Show that $T(x) = \displaystyle\sum_{i=1}^{...
2
votes
0answers
23 views

Likelihood ratio tests for quasi- models

I have been playing around with over-dispersion in binomial data and looking into qausi-binomial models as a solution. When comparing binomial models through the change in deviance, I can reproduce ...
0
votes
0answers
18 views

How to calculate Kernel Density for Bootstrap Likelihood

I am attempting to write R code to generate bootstrap likelihood as described in section 3 of this paper https://arxiv.org/pdf/1510.07287.pdf. I am confident that I performed the bootstraps correct, ...
1
vote
0answers
26 views

Homogeneity test in survival meta-analysis with adjusted HR

I was wondering what was the way to measure heterogeneity in a survival meta-analysis. I have for each study, an hazard ratio calculated with a penalized cox model (...