Questions tagged [log-likelihood]

the logarithm of the likelihood function.

Filter by
Sorted by
Tagged with
4
votes
1answer
57 views

Negative infinity produced when computing log-likelihood in Poisson Regression R

I am trying to compute the log-likelihood in a Poisson regression in R. However, my computation produces negative infinity values for some observations. This is my code: ...
1
vote
1answer
34 views

Calculate log likelihood of mixture of gaussians “by hand” in R

I'm trying to ensure the the method of calculating log likelihood for a model produced using mixtools vs a model produced using MLE estimates of mu and sigma are the same. The best way I can think of ...
0
votes
0answers
8 views

how to evaluate results with glmmPQL

First of all im thankfull for your attention. I have to evaluate the effect of 3 fixed effects in the vegetal coberture and i must use glmmPQL because my data has lineality condition problems and ...
2
votes
0answers
24 views

How do you calculate log likelihood p(x) for a VAE?

I was reading the Importance Weighted Autoencoders paper and its experiment section compares the density estimation result on MNIST for IWAE vs VAE. I know that density estimation estimating log p(x) ...
0
votes
0answers
11 views

question about the paper “ADDITIVE LOGISTIC REGRESSION: A STATISTICAL VIEW OF BOOSTING”

I am reading the paper "additive logistic regression:a statistical view of boosting". I am confused about how to get equation (31),(32),(33) in the paper. Could anyone explain a bit please? ...
0
votes
0answers
30 views

List Experiment “ictreg.joint” error: “log-likelihood is not monotonically increasing”

I'm trying to use "ictreg.joint" function in "list" package in R by Imai and Blair to use the predicted responses from the list experiment as the explanatory variable to analyze an outcome variable. ...
0
votes
1answer
64 views

why in Gaussian linear regression they put $y_i-\theta^Tx_i$ instead of $x$?

I am new to the machine learning area. Then I am studying a paper that considers the problem of logistic regression in Bayesian. In general, in regression or logistic regression when they assume the ...
0
votes
0answers
13 views

likelihood ratio test statistic on data

I have a dataset with variables date sold, prices, type (house/unit) and number of bedrooms. I have fitted the data with weibull distribution on the variable 'prices' with fitdist(). I would like to ...
1
vote
1answer
93 views

Machine Learning: Negative Log Likelihood vs Cross-Entropy

Im developing some machine learning code, and I'm using the softmax function in the output layer. My loss function is trying to minimize the Negative Log Likelihood (NLL) of the network's output. ...
0
votes
0answers
32 views

Measuring the “flatness” of the likelihood function

I would like to know if there is a way to measure the flatness of the likelihood for a distribution with a large number of parameters (at least 4 in this case). I am exploring the Generalized ...
0
votes
0answers
18 views

Akaike Information Criterion and distribution of residuals

TOY EXAMPLE. Suppose we are interested in the relationship between two quantities, $X$ and $Y$. We set $X = 1, 2, 3$ with certainty and measure $Y$. Our measurements are subject to iid measurement ...
2
votes
1answer
35 views

Does log-likelihood cost function in a multinomial classification consider only the output at the neuron that should be active for that class?

Consider a neural network with an output layer of softmax neurons and a log likelihood cost function. For easiness consider one wants to train a MNIST classifier. The output layer will have 9 neurons ...
2
votes
1answer
24 views

A particular method for estimating the gradient of a log-density from samples

Suppose I have $N$ samples $x^1, \ldots, x^N$ which were drawn iid from an unknown density $P(x)$. Suppose I am interested in estimating the vector-valued function $g(x) = \nabla \log P (x)$. One ...
0
votes
1answer
15 views

Define log likelihood for CJS model in Stan

FYI, I'm new to Stan and this is my first question here. I'm unsure how to calculate the log likelihood for a Cormack-Jolley-Seber model in Stan. Can anyone help me with this? Background: I've made ...
0
votes
0answers
6 views

hypothesis test to chose between two linear models assuming additive Gaussian error

I was trying to follow certain procedure that was done by some paper, and I don't fully understand how to do it. The paper describes as follow: The description is talking about 4 models and nested ...
3
votes
0answers
55 views

Intuitively, why is the expectation of the score 0? [duplicate]

Given a probability model $p(x|\theta)$, the score function is typically defined as $\frac{d}{d\theta} \log p(x|\theta)$. The expectation $\mathbb{E}[\frac{d}{d\theta} \log p(X|\theta)]$ is zero ...
0
votes
1answer
36 views

Negative Log-Likelihood Loss with Gibbs distribution for beta approaching infinity

This might be more of a math question, but since it concerns ML, I'll ask it here. In "A tutorial on energy-based learning" (LeCun et al., 2006), on page 15, section 2.2.4 about the Negative Log-...
0
votes
1answer
26 views

Maximum Likelihood Estimation (log-likelihood) Mistake search [closed]

I wanted to ask if my way was right because the end function looks quite complicated: iid X1, ... , Xn with f(x) = 1/2σ * exp(−|x|/σ), x∈R, σ > 0 with σ being our searched parameter. My try: L(σ) &...
2
votes
1answer
33 views

Maximum Likelihood of a sum of products - any way to transform?

I have a likelihood function in this form: $$L(\lambda) = \sum_i \prod_j f(i,j, \lambda)$$ $f(i,j, \lambda)$ are probabilities, so that their product gets very small. If I try and calculate it in ...
0
votes
0answers
76 views

Why logLik works for PLM models in R?

I'm estimating linear panel models with fixed effects in R using plm of the plm-package. As I was looking for ways to compare different models, I came across AIC (Akaike's ‘An Information Criterion’). ...
1
vote
0answers
18 views

Distance measure to be minimized to get MLE for linear regression coefficients

How do we decide that distance measure to be minimized for normal distribution in linear regression is: $||y−Xβ||^2$ from log-likelihood function: $l(θ) = −\frac{1}{2}nlog(σ^{2})− \frac{1}{2σ^{2}}||...
0
votes
0answers
22 views

Likelihood of three dimensional data

I'm having a lot of trouble finding the likelihood and log-likelihood of $\tau$ that corresponds to the following equation: $a_i = y(t_i, \tau) + \epsilon_i$ where $\epsilon_i \sim \mathcal{N}(0, \...
1
vote
0answers
18 views

How to prove that the likelihood of a proportional hazards with lognormal baseline model is log concave?

I want to fit a survival model using a proportional hazards assumption $$h(t) = h_0(t)\exp(x^T\beta),$$ where $$h_0(t) = \dfrac{\frac{1}{\sigma t} \phi \left(\frac{\log(t) - \mu}{\sigma}\right)}{\...
-1
votes
1answer
47 views

Why Fisher Information uses Log Likelihood and not Plain Likelihood [duplicate]

I would like to know that to determine Fisher information from the Likelihood model, why do we take the log of the likelihood first instead of using normal likelihood ?
1
vote
1answer
22 views

Log-likelihood function for a filtered Fourier spectrum

I have time series data from which I am trying to infer parameters using MCMC. I normally infer parameters about the data in the time domain, using a Normal log-likelihood. However, I now have to ...
1
vote
0answers
28 views

Fisher Information | Second Derivative of Likelihood Vs Second Derivative of Log Likelihood

I watched this video on Fisher Information and it is mentioned that in Taylor series expansion of the likelihood function the second derivative is parabola which is not a good approximation and a ...
1
vote
1answer
72 views

How to compute variance of Cox model coefficient estimate using Fisher information?

We have Cox proportional hazards model: $$ \lambda(t,x) = \lambda_0(t)exp(\boldsymbol \beta'\boldsymbol x),$$ where $\boldsymbol \beta$ and $\boldsymbol x$ are vectors. To make it simple, lets say ...
3
votes
1answer
25 views

Negative Log Likelihood cost

For multiclass classification, does the negative log likelihood loss function only take the loss for the classification group? i.e $$ C(\theta) \equiv \sum{}{}y_ilog(\hat{y}_i) $$ Doesn't $y_i$ just ...
0
votes
0answers
18 views

Four different ways to deal with the log-likelihood of a probability density function (Python code included)

This is not really a question but more of a discussion. Please correct me where wrong and share your thoughts and past experience with regards to computing the likelihoods for continuous data models. ...
4
votes
0answers
34 views

AIC Comparison for MLM with Different Distributions

Thank you in advance for your time and consideration! I am a non-mathematically-inclined graduate student in communication just learning multilevel modeling. We are running different models - some ...
0
votes
0answers
15 views

How to interpret -2LL change in multilevel linear model in SPSS output?

I ran a multilevel linear model and compared a fixed-effects-only model with another model with random intercept. I've been referring to resources by Prof Andy Field and it was mentioned that in order ...
1
vote
1answer
36 views

Gaussian Mixture model - Penalized log-likelihood in EM algorithm not monotone increasing

I am working on a multivariate Gaussian Mixture Model in R. The goal is to do regularized clustering on the data, where each component represents a cluster. I wrote an EM algorithm to maximize a ...
2
votes
3answers
73 views

Prove that the likelihood function L(θ|x) is equivalent to maximizing log L(θ|x) where log is the natural logarithm [closed]

In other words, why $\text{argmax} \text{ } L(\theta) = \text{argmax} \text{ } \text{log} \text{ } L(\theta)$ ?
0
votes
0answers
24 views

What 'log-liklihood' tells in Multilple linear regression?

I am doing multiple regression and I got this number 'Log-likelihood = -2320.1'. What this number actually indicates and what is the procedure to calculate this number? Is there any significance ...
1
vote
0answers
37 views

Difference between with and without “weight” option of the same data on logistic regression in R

I still keep checking from my previous question here. The next, I tried the case of proportion(=yes/yes+no), using previous best answer. Yes, I got it. But, I couldn’t understand the case without “...
2
votes
2answers
75 views

Difference between binary and count data of same data on logistic regression in R [duplicate]

I confuse that the difference of Residuals deviance between binary and count data of the same data, by logistic regression in R. I'd like to know the way to calculate the both Residual deviance. ...
0
votes
1answer
22 views

Comparing log likelihood & AIC for two spatial error regression models with the same dependent but different independent variables

I have two spatial lag models using the same dependent variable (average income) and different independent variables (1- living environment deprivation; 2- education deprivation) for towns in the UK. ...
1
vote
2answers
91 views

Why can't regression via Maximum Likelihood shrink coefficients to zero?

Why can't regression via Maximum Likelihood shrink regression coefficients to zero as in LASSO? Does shrinking coefficients to zero not give higher L-likelihood? Does the answer to my question have to ...
0
votes
2answers
36 views

Is likelihood ratio test comparing apples to oranges?

I don't believe that likelihood ratio tests work in the context of regression because the likelihood functions for model A versus B aren't the same thing so we're using different standards to judge ...
0
votes
1answer
20 views

Computing Gradients for a [-1, 1]-valued RBM

The gradient derivation for a binary-valued RBM with values $\in\{0,1\}$ is well-documented, for example in Goodfellow, et al and here on Cross Validated. However, in some works (e.g., associative ...
1
vote
1answer
36 views

Log-likelihood using the link identity for poisson?

I understood the Log-likelihood using the link “log” for poisson, λ=exp(α+βx). But I can’t get the Log-likelihood in the case of “identity”, λ=α+βx. How do I get it?. The example is the following data....
0
votes
0answers
37 views

Likelihood-ratio based interval

I'm dealing with this exercise: I did the first two parts of the exercise without particular problems, so I differentiated the log-lik function with respect to p and eventually I maximized it. Then I ...
0
votes
0answers
36 views

Expected value and Maximum Likelihood Estimation

I'm doing this exercise about Poisson distribution and maximum likelihood estimation: I have had no problem with points a and b, but I'm struggling with the correct answers of the C part. From my ...
4
votes
0answers
41 views

Expectation Maximisation (EM) Algorithm

Some of my parameters do not have a closed form solution. Thus, for these parameters the M-step is implemented via a one-step Newton-Raphson update, i.e., \begin{equation} \theta^{t+1} = \theta^t - \...
1
vote
1answer
31 views

ERGM with larger log-likelihood but worse fit than reference model?

I'm using the R package statnet to fit some ERGMs to the Faux Dixon High simulated network data provided with the package. The first model I fit is almost identical ...
1
vote
0answers
24 views

Optimize Log Likelihood Model based on Gaussian Process involving Matrix Calculus

Given \begin{equation} \text{temperature(t, y)} = a_0 + a_1t + X(t) \end{equation} where temperature(t, year) is the dataset temperature at day $t$ in year $y$. $a_0, a_1 \in \mathbb{R}$, and $X(t)$ ...
2
votes
0answers
59 views

Can the likelihood ratio estimate multivariate confidence levels?

Wilks' theorem describes the log-ratio between the highest likelihood of a distribution $\mathcal{L}$ (aka the dominant mode, given at $\vec{x}_{m}$) and the likelihood of a distribution at a given ...
4
votes
1answer
56 views

Likelihood of linear mixed effects model

Consider the following model $$\left \{ \begin{array}{l} y_i = x_i\beta + z_ib + \varepsilon_i,\\\\ b_i \sim \mathcal N(0, \Sigma), \quad \varepsilon_i \sim \mathcal N(0, \sigma^2), \end{array} \right....
0
votes
1answer
36 views

Binary logit modelling with R - Issue finding same results

I need your help to figure out something about the estimation of simple binary logit model in R. As nicely explained on the following website (https://stats.idre.ucla.edu/r/dae/logit-regression/) the ...
0
votes
0answers
30 views

log-likelihood function

There are 3 classified groups with the numbers being $n_{1}, n_{2}, n_{3}$. According to the genetic model, the probabilities for each group are proportional to: $p_{1}(\theta):p_{2}(\theta):p_{3}(\...

1
2 3 4 5