# Questions tagged [log-likelihood]

the logarithm of the likelihood function.

219 questions
Filter by
Sorted by
Tagged with
57 views

### Negative infinity produced when computing log-likelihood in Poisson Regression R

I am trying to compute the log-likelihood in a Poisson regression in R. However, my computation produces negative infinity values for some observations. This is my code: ...
34 views

### Calculate log likelihood of mixture of gaussians “by hand” in R

I'm trying to ensure the the method of calculating log likelihood for a model produced using mixtools vs a model produced using MLE estimates of mu and sigma are the same. The best way I can think of ...
8 views

### how to evaluate results with glmmPQL

First of all im thankfull for your attention. I have to evaluate the effect of 3 fixed effects in the vegetal coberture and i must use glmmPQL because my data has lineality condition problems and ...
24 views

### How do you calculate log likelihood p(x) for a VAE?

I was reading the Importance Weighted Autoencoders paper and its experiment section compares the density estimation result on MNIST for IWAE vs VAE. I know that density estimation estimating log p(x) ...
11 views

### question about the paper “ADDITIVE LOGISTIC REGRESSION: A STATISTICAL VIEW OF BOOSTING”

I am reading the paper "additive logistic regression:a statistical view of boosting". I am confused about how to get equation (31),(32),(33) in the paper. Could anyone explain a bit please? ...
30 views

### List Experiment “ictreg.joint” error: “log-likelihood is not monotonically increasing”

I'm trying to use "ictreg.joint" function in "list" package in R by Imai and Blair to use the predicted responses from the list experiment as the explanatory variable to analyze an outcome variable. ...
64 views

### why in Gaussian linear regression they put $y_i-\theta^Tx_i$ instead of $x$?

I am new to the machine learning area. Then I am studying a paper that considers the problem of logistic regression in Bayesian. In general, in regression or logistic regression when they assume the ...
13 views

### likelihood ratio test statistic on data

I have a dataset with variables date sold, prices, type (house/unit) and number of bedrooms. I have fitted the data with weibull distribution on the variable 'prices' with fitdist(). I would like to ...
93 views

### Machine Learning: Negative Log Likelihood vs Cross-Entropy

Im developing some machine learning code, and I'm using the softmax function in the output layer. My loss function is trying to minimize the Negative Log Likelihood (NLL) of the network's output. ...
32 views

### Measuring the “flatness” of the likelihood function

I would like to know if there is a way to measure the flatness of the likelihood for a distribution with a large number of parameters (at least 4 in this case). I am exploring the Generalized ...
18 views

### Akaike Information Criterion and distribution of residuals

TOY EXAMPLE. Suppose we are interested in the relationship between two quantities, $X$ and $Y$. We set $X = 1, 2, 3$ with certainty and measure $Y$. Our measurements are subject to iid measurement ...
35 views

### Does log-likelihood cost function in a multinomial classification consider only the output at the neuron that should be active for that class?

Consider a neural network with an output layer of softmax neurons and a log likelihood cost function. For easiness consider one wants to train a MNIST classifier. The output layer will have 9 neurons ...
24 views

### A particular method for estimating the gradient of a log-density from samples

Suppose I have $N$ samples $x^1, \ldots, x^N$ which were drawn iid from an unknown density $P(x)$. Suppose I am interested in estimating the vector-valued function $g(x) = \nabla \log P (x)$. One ...
15 views

### Define log likelihood for CJS model in Stan

FYI, I'm new to Stan and this is my first question here. I'm unsure how to calculate the log likelihood for a Cormack-Jolley-Seber model in Stan. Can anyone help me with this? Background: I've made ...
6 views

### hypothesis test to chose between two linear models assuming additive Gaussian error

I was trying to follow certain procedure that was done by some paper, and I don't fully understand how to do it. The paper describes as follow: The description is talking about 4 models and nested ...
55 views

### Intuitively, why is the expectation of the score 0? [duplicate]

Given a probability model $p(x|\theta)$, the score function is typically defined as $\frac{d}{d\theta} \log p(x|\theta)$. The expectation $\mathbb{E}[\frac{d}{d\theta} \log p(X|\theta)]$ is zero ...
36 views

### Negative Log-Likelihood Loss with Gibbs distribution for beta approaching infinity

This might be more of a math question, but since it concerns ML, I'll ask it here. In "A tutorial on energy-based learning" (LeCun et al., 2006), on page 15, section 2.2.4 about the Negative Log-...
26 views

### Maximum Likelihood Estimation (log-likelihood) Mistake search [closed]

I wanted to ask if my way was right because the end function looks quite complicated: iid X1, ... , Xn with f(x) = 1/2σ * exp(−|x|/σ), x∈R, σ > 0 with σ being our searched parameter. My try: L(σ) &...
33 views

### Maximum Likelihood of a sum of products - any way to transform?

I have a likelihood function in this form: $$L(\lambda) = \sum_i \prod_j f(i,j, \lambda)$$ $f(i,j, \lambda)$ are probabilities, so that their product gets very small. If I try and calculate it in ...
76 views

### Why logLik works for PLM models in R?

I'm estimating linear panel models with fixed effects in R using plm of the plm-package. As I was looking for ways to compare different models, I came across AIC (Akaike's ‘An Information Criterion’). ...
18 views

18 views

I want to fit a survival model using a proportional hazards assumption $$h(t) = h_0(t)\exp(x^T\beta),$$ where $$h_0(t) = \dfrac{\frac{1}{\sigma t} \phi \left(\frac{\log(t) - \mu}{\sigma}\right)}{\... 1answer 47 views ### Why Fisher Information uses Log Likelihood and not Plain Likelihood [duplicate] I would like to know that to determine Fisher information from the Likelihood model, why do we take the log of the likelihood first instead of using normal likelihood ? 1answer 22 views ### Log-likelihood function for a filtered Fourier spectrum I have time series data from which I am trying to infer parameters using MCMC. I normally infer parameters about the data in the time domain, using a Normal log-likelihood. However, I now have to ... 0answers 28 views ### Fisher Information | Second Derivative of Likelihood Vs Second Derivative of Log Likelihood I watched this video on Fisher Information and it is mentioned that in Taylor series expansion of the likelihood function the second derivative is parabola which is not a good approximation and a ... 1answer 72 views ### How to compute variance of Cox model coefficient estimate using Fisher information? We have Cox proportional hazards model:$$ \lambda(t,x) = \lambda_0(t)exp(\boldsymbol \beta'\boldsymbol x),$$where \boldsymbol \beta and \boldsymbol x are vectors. To make it simple, lets say ... 1answer 25 views ### Negative Log Likelihood cost For multiclass classification, does the negative log likelihood loss function only take the loss for the classification group? i.e$$ C(\theta) \equiv \sum{}{}y_ilog(\hat{y}_i) $$Doesn't y_i just ... 0answers 18 views ### Four different ways to deal with the log-likelihood of a probability density function (Python code included) This is not really a question but more of a discussion. Please correct me where wrong and share your thoughts and past experience with regards to computing the likelihoods for continuous data models. ... 0answers 34 views ### AIC Comparison for MLM with Different Distributions Thank you in advance for your time and consideration! I am a non-mathematically-inclined graduate student in communication just learning multilevel modeling. We are running different models - some ... 0answers 15 views ### How to interpret -2LL change in multilevel linear model in SPSS output? I ran a multilevel linear model and compared a fixed-effects-only model with another model with random intercept. I've been referring to resources by Prof Andy Field and it was mentioned that in order ... 1answer 36 views ### Gaussian Mixture model - Penalized log-likelihood in EM algorithm not monotone increasing I am working on a multivariate Gaussian Mixture Model in R. The goal is to do regularized clustering on the data, where each component represents a cluster. I wrote an EM algorithm to maximize a ... 3answers 73 views ### Prove that the likelihood function L(θ|x) is equivalent to maximizing log L(θ|x) where log is the natural logarithm [closed] In other words, why \text{argmax} \text{ } L(\theta) = \text{argmax} \text{ } \text{log} \text{ } L(\theta) ? 0answers 24 views ### What 'log-liklihood' tells in Multilple linear regression? I am doing multiple regression and I got this number 'Log-likelihood = -2320.1'. What this number actually indicates and what is the procedure to calculate this number? Is there any significance ... 0answers 37 views ### Difference between with and without “weight” option of the same data on logistic regression in R I still keep checking from my previous question here. The next, I tried the case of proportion(=yes/yes+no), using previous best answer. Yes, I got it. But, I couldn’t understand the case without “... 2answers 75 views ### Difference between binary and count data of same data on logistic regression in R [duplicate] I confuse that the difference of Residuals deviance between binary and count data of the same data, by logistic regression in R. I'd like to know the way to calculate the both Residual deviance. ... 1answer 22 views ### Comparing log likelihood & AIC for two spatial error regression models with the same dependent but different independent variables I have two spatial lag models using the same dependent variable (average income) and different independent variables (1- living environment deprivation; 2- education deprivation) for towns in the UK. ... 2answers 91 views ### Why can't regression via Maximum Likelihood shrink coefficients to zero? Why can't regression via Maximum Likelihood shrink regression coefficients to zero as in LASSO? Does shrinking coefficients to zero not give higher L-likelihood? Does the answer to my question have to ... 2answers 36 views ### Is likelihood ratio test comparing apples to oranges? I don't believe that likelihood ratio tests work in the context of regression because the likelihood functions for model A versus B aren't the same thing so we're using different standards to judge ... 1answer 20 views ### Computing Gradients for a [-1, 1]-valued RBM The gradient derivation for a binary-valued RBM with values \in\{0,1\} is well-documented, for example in Goodfellow, et al and here on Cross Validated. However, in some works (e.g., associative ... 1answer 36 views ### Log-likelihood using the link identity for poisson? I understood the Log-likelihood using the link “log” for poisson, λ=exp(α+βx). But I can’t get the Log-likelihood in the case of “identity”, λ=α+βx. How do I get it?. The example is the following data.... 0answers 37 views ### Likelihood-ratio based interval I'm dealing with this exercise: I did the first two parts of the exercise without particular problems, so I differentiated the log-lik function with respect to p and eventually I maximized it. Then I ... 0answers 36 views ### Expected value and Maximum Likelihood Estimation I'm doing this exercise about Poisson distribution and maximum likelihood estimation: I have had no problem with points a and b, but I'm struggling with the correct answers of the C part. From my ... 0answers 41 views ### Expectation Maximisation (EM) Algorithm Some of my parameters do not have a closed form solution. Thus, for these parameters the M-step is implemented via a one-step Newton-Raphson update, i.e., \begin{equation} \theta^{t+1} = \theta^t - \... 1answer 31 views ### ERGM with larger log-likelihood but worse fit than reference model? I'm using the R package statnet to fit some ERGMs to the Faux Dixon High simulated network data provided with the package. The first model I fit is almost identical ... 0answers 24 views ### Optimize Log Likelihood Model based on Gaussian Process involving Matrix Calculus Given \begin{equation} \text{temperature(t, y)} = a_0 + a_1t + X(t) \end{equation} where temperature(t, year) is the dataset temperature at day t in year y. a_0, a_1 \in \mathbb{R}, and X(t) ... 0answers 59 views ### Can the likelihood ratio estimate multivariate confidence levels? Wilks' theorem describes the log-ratio between the highest likelihood of a distribution \mathcal{L} (aka the dominant mode, given at \vec{x}_{m}) and the likelihood of a distribution at a given ... 1answer 56 views ### Likelihood of linear mixed effects model Consider the following model$$\left \{ \begin{array}{l} y_i = x_i\beta + z_ib + \varepsilon_i,\\\\ b_i \sim \mathcal N(0, \Sigma), \quad \varepsilon_i \sim \mathcal N(0, \sigma^2), \end{array} \right....
There are 3 classified groups with the numbers being $n_{1}, n_{2}, n_{3}$. According to the genetic model, the probabilities for each group are proportional to: \$p_{1}(\theta):p_{2}(\theta):p_{3}(\...