Questions tagged [log-likelihood]

the logarithm of the likelihood function.

Filter by
Sorted by
Tagged with
2
votes
1answer
35 views

Difference between binary and count data of same data on logistic regression in R

I confuse that the difference of Residuals deviance between binary and count data of the same data, by logistic regression in R. I'd like to know the way to calculate the both Residual deviance. ...
0
votes
1answer
14 views

Comparing log likelihood & AIC for two spatial error regression models with the same dependent but different independent variables

I have two spatial lag models using the same dependent variable (average income) and different independent variables (1- living environment deprivation; 2- education deprivation) for towns in the UK. ...
1
vote
2answers
85 views

Why can't regression via Maximum Likelihood shrink coefficients to zero?

Why can't regression via Maximum Likelihood shrink regression coefficients to zero as in LASSO? Does shrinking coefficients to zero not give higher L-likelihood? Does the answer to my question have to ...
0
votes
2answers
29 views

Is likelihood ratio test comparing apples to oranges?

I don't believe that likelihood ratio tests work in the context of regression because the likelihood functions for model A versus B aren't the same thing so we're using different standards to judge ...
0
votes
0answers
18 views

Does the newton-raphson either find the maximum of the loglikelihood function or estimate the MLE and likelihood function? [closed]

Does using newton-raphson method or some other optimization method used in nlme package or mixed effects models actually find the maximum of the log-likelihood function's height or does it simply find ...
0
votes
0answers
23 views

Meaning of “log-likelihood” in terms of models in nlme [closed]

When R says that the log-likelihood of a model using nlme is some number, what does that mean? The log-likelihood is an abbreiation for the log of the likelihood function dependent on parameters $x_1,...
0
votes
1answer
16 views

Computing Gradients for a [-1, 1]-valued RBM

The gradient derivation for a binary-valued RBM with values $\in\{0,1\}$ is well-documented, for example in Goodfellow, et al and here on Cross Validated. However, in some works (e.g., associative ...
0
votes
0answers
19 views

Profile log-likelihood and confidence intervals

I wanted to ask how to compute the profile log-likelihood. I will take this exercise as an example to explain my trouble: If I've understood, The procedure is the following: 1)Consider the ...
1
vote
1answer
30 views

Log-likelihood using the link identity for poisson?

I understood the Log-likelihood using the link “log” for poisson, λ=exp(α+βx). But I can’t get the Log-likelihood in the case of “identity”, λ=α+βx. How do I get it?. The example is the following data....
0
votes
0answers
33 views

Likelihood-ratio based interval

I'm dealing with this exercise: I did the first two parts of the exercise without particular problems, so I differentiated the log-lik function with respect to p and eventually I maximized it. Then I ...
0
votes
0answers
23 views

Expected value and Maximum Likelihood Estimation

I'm doing this exercise about Poisson distribution and maximum likelihood estimation: I have had no problem with points a and b, but I'm struggling with the correct answers of the C part. From my ...
4
votes
0answers
36 views

Expectation Maximisation (EM) Algorithm

Some of my parameters do not have a closed form solution. Thus, for these parameters the M-step is implemented via a one-step Newton-Raphson update, i.e., \begin{equation} \theta^{t+1} = \theta^t - \...
1
vote
0answers
14 views

ERGM with larger log-likelihood but worse fit than reference model?

I'm using the R package statnet to fit some ERGMs to the Faux Dixon High simulated network data provided with the package. The first model I fit is almost identical ...
1
vote
0answers
22 views

Optimize Log Likelihood Model based on Gaussian Process involving Matrix Calculus

Given \begin{equation} \text{temperature(t, y)} = a_0 + a_1t + X(t) \end{equation} where temperature(t, year) is the dataset temperature at day $t$ in year $y$. $a_0, a_1 \in \mathbb{R}$, and $X(t)$ ...
2
votes
0answers
56 views

Can the likelihood ratio estimate multivariate confidence levels?

Wilks' theorem describes the log-ratio between the highest likelihood of a distribution $\mathcal{L}$ (aka the dominant mode, given at $\vec{x}_{m}$) and the likelihood of a distribution at a given ...
3
votes
1answer
34 views

Likelihood of linear mixed effects model

Consider the following model $$\left \{ \begin{array}{l} y_i = x_i\beta + z_ib + \varepsilon_i,\\\\ b_i \sim \mathcal N(0, \Sigma), \quad \varepsilon_i \sim \mathcal N(0, \sigma^2), \end{array} \right....
0
votes
1answer
24 views

Binary logit modelling with R - Issue finding same results

I need your help to figure out something about the estimation of simple binary logit model in R. As nicely explained on the following website (https://stats.idre.ucla.edu/r/dae/logit-regression/) the ...
0
votes
0answers
27 views

log-likelihood function

There are 3 classified groups with the numbers being $n_{1}, n_{2}, n_{3}$. According to the genetic model, the probabilities for each group are proportional to: $p_{1}(\theta):p_{2}(\theta):p_{3}(\...
0
votes
0answers
13 views

Calculating the log-likelihood score from densities

I've seen the log-likelihood score defined in terms of a density: $S_T(\theta) = p_T^{-1}(\theta)\left(\frac{\partial{p}_T(\theta)}{\partial \theta}\right)'$ where $'$ indicates transpose. A few ...
0
votes
0answers
62 views

Deriving Gradient from negative log-likelihood function

I have been having some difficulty deriving a gradient of an equation. I have a Negative log likelihood function, from which i have to derive its gradient function. Negative log likelihood function ...
1
vote
0answers
39 views

A trivial question about EM algorithm theory

In "The EM Algorithm and Extensions", second edition, from Geoffrey J. McLachlan and Thriyambakam Krishnan, X is the latent variable, and Y is de observed (incomplete) variable I'm little confuse ...
1
vote
0answers
19 views

How to compute statistical significance using a likelihood ratio?

The title really says it all. Suppose I have a change in log-likelihood (i.e., $\Delta LL = LL_{fitted} - LL_{null}$), and I would like to compute the $1\sigma$, $2\sigma$, etc. confidence region from ...
3
votes
2answers
90 views

Why is the expected gradient of a density not parallel to the expected gradient of the log density?

I'm confused by a seemingly counter-intuitive property of the interaction between distributions, log transforms, expectations and gradients. Suppose I have some distribution over random variable $x$ ...
0
votes
0answers
25 views

Maximum Likelihood Ratio with restriction

I'm trying to code the likelihood ratio test about two categories for observations from a multinomial experiment with twelve categories; e.g., $H_0: p_1 = p2$ versus $H_1: p_1\neq p_2$. As a test of ...
0
votes
0answers
19 views

Log-likehood : ommited value

I searched the second form of the log-likelihood equation which appear in A Probabilistic Perspective of Kevin Murphy because I've tried to understand why $y_{ic}$ disappear in the second part of the ...
0
votes
0answers
25 views

In log likelihood ratio test why do we always take model with fewer parameters as null hypothesis?

To compare two models using the log-likelihood ratio test, I found that the model with fewer parameters is taken as the null hypothesis and the other as the alternative. I was wondering if the reverse ...
0
votes
0answers
81 views

Interpreting this results: Log likelihood ratio and log likelihood

I am currently working a Survival model, for which I am using a Time Varying Cox regression and I have had the following results: Log likelihood: -999.76 ...
1
vote
1answer
45 views

The expected log-Likelihood in Kullback Leibler Divergence

Given a true normal distribution $g(x)$ with mean $\mu_G$ and variance $\sigma_G$, and a model $f(x)$, the KL divergence involves the expected log-likelihood $\mathbb{E}_G[log f(x|\theta]$. The ...
0
votes
1answer
248 views

Fisher information from MLE in R?

Reworded the question: I have read "The Fisher information I(p) is this negative second derivative of the log-likelihood function, averaged over all possible X = {h, N–h}, when we assume some value ...
1
vote
0answers
42 views

Expectation of Log Likelihood Function with given parameters Proof

I have been looking for AICc value derivation employing Kullback-Leibler distance but as a result of my search I got stuck with expectation of loglikelihood. In the link loglikelihood is given as $InL(...
0
votes
1answer
658 views

What is bits per dimension (bits/dim) exactly (in pixel CNN papers)?

If it is for the lack of my effort to search, I apologize in advance but I couldn't find a explicit definition of bits per dimension (bits/dim). The first mention of its definition I found was from ...
2
votes
1answer
44 views

Should the log-likelihood value be reported?

Just a quick question. I have used maximum likelihood estimation to find the best-fit parameters of a model. In doing so I get a log likelihood value, which obviously is the highest log likelihood ...
-1
votes
1answer
43 views

MCMC with slowly varying Log-Likelihood

I am using MCMC (Metropolis-Hastings) to simulate values of $\theta$: I have a Log-likelihood (using 10 inputs $x_i$) $$L=-\frac{n}{2}\ln(2\pi)-\frac{1}{2}\sum_{i=1}^n(x_i-\theta)^2$$ The variation ...
0
votes
0answers
50 views

calculating log likelihood for multivariate linear regression using R

I want to calculate a loglikelihood for multivariate linear regression. I’ve been calculated the log likelihood using multiple linear regression. ...
2
votes
1answer
59 views

When to use the full and the conditional likelihood

In the context of estimating parameters of a time series model, we may consider either the full likelihood or the conditional likelihood. I was wondering in what situations the full ...
0
votes
0answers
10 views

Reporting difference in distributions for text

I am working on a problem that can be translated to determining which of two corpora generated a given sample text. I have a hypothesis for before some processing and other for after processing, and ...
0
votes
0answers
45 views

Gradient Boosting using Log-Likelihood Loss Function

I would love some help with gradient boosting using the negative log-likelihood as a loss function. According to a few sources, this should easily be possible. How would this gradient be calculated? ...
1
vote
0answers
42 views

How to write the log likelihood for a full model of a negative binomial distribution

I would like to know how to write the log likelihood for a full model of a negative binomial distribution to test a null hypothesis that population mean group1 = population mean group2 vs not equal (i....
0
votes
0answers
37 views

Confidence regions after fitting a 2 parameter gaussian mixture model?

Suppose I have a gaussian mixture model with 2 parameters $(u,v)$ and 2 parts. The model is $P({x_i}|u,v)=uN(x_i|\mu_1^{i} = x_i^2/v,\sigma_1^{i}) + (1-u)N(x_i|\mu_2^{i} = x_i^3/2v^2,\sigma_2^i)$. ...
0
votes
1answer
62 views

BIC in practice with gaussian distribution

I am considering a gaussian distribution: \begin{equation} y \sim N(net(x,w), \sigma^2). \end{equation} $net()$ is just the output of some neural net with weights $w$ and input $x$. The log-...
1
vote
0answers
17 views

Can (log-)likelihood be used to compare a binomial model to its beta-binomial equivalent?

In this article the author talks about fitting beta-binomial models to data when the there data is over-dispersed relative to the assumptions of a model with binomial errors. Near the end they present ...
0
votes
0answers
18 views

Is there a difference in finding p-value using likelihood ratio vs minimum deviance statistic?

I am trying to fit my data to a distribution and find the fit parameters and associated p-value. If I use the -2-log likelihood ratio, or G-test, vs the minimum deviance method, will I get different p-...
4
votes
1answer
312 views

Does the Jensen-Shannon divergence maximise likelihood?

Minimising the KL divergence between your model distribution and the true data distribution is equivalent to maximising the (log-) likelihood. In machine learning, we often want to create a model ...
2
votes
1answer
140 views

Having trouble figuring out how loss was calculated for SQuAD task in BERT paper

The BERT Paper https://arxiv.org/pdf/1810.04805.pdf Section 4.2 covers the SQuAD training. So from my understanding, there are two extra parameters trained, they are two vectors with the same ...
1
vote
0answers
22 views

log-likelihood of normal distributed fitted using MLE

Suppose we fit a normal using MLE which means we have parameters $$\mu = \frac{1}{n}\sum_{i=1}^n x_i$$ and $$\sigma^2 = \frac{1}{n}\sum_{i=1}^n(x_i - \mu)^2$$ Then we calculate the log-likelihood as ...
0
votes
0answers
22 views

Comparing log-likelihoods for different datasets of the same size?

Does it make sense to compare log likelihood values from different datasets if the datasets are of the same nature and size? Like let's say I fit a model on daily S&P 500 returns from 2000-2010 ...
1
vote
1answer
56 views

Binomial distributed random sample: find the least variance from the set of all unbiased estimators of $\theta$

Let $X_{1},X_{2},\ldots,X_{n}$ be random sample from $X\sim\text{Binomial}(2,\theta)$. (a) Find the least variance from the set of all unbiased estimators of $\theta$. (b) Find a sufficient ...
1
vote
1answer
53 views

Normal distributed random sample: find the least variance from the set of all unbiased estimators of $\theta$

Let $X_{1},X_{2},\ldots,X_{n}$ be a random sample from $X\sim\mathcal{N}(0,\sigma^{2})$. (a) Find the least variance from the set of all unbiased estimators of $\sigma^{2}$. (b) Find a sufficient ...
3
votes
1answer
160 views

AIC for increasing sample size

I am using AIC as a model selection criteria in one of my projects. However, since AIC isn't dependent on the number of points sampled, for large n the log likelihood term rapidly outscales the ...
2
votes
1answer
85 views

trouble creating a negative log likelihood for a linear model in R [closed]

I am new to stats and R and I am having trouble figuring out how to calculate a function that gives me the NLL of a linear regression.