Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [log-likelihood]

the logarithm of the likelihood function.

1
vote
0answers
41 views

Maximum likelihood estimator of the parameter of randomness in Watts and Strogatz's model (1998)

According to the paper Menezes, M. B., Kim, S., & Huang, R. (2017). Constructing a Watts-Strogatz network from a small-world network with symmetric degree distribution. PloS one, 12(6), e0179120,...
1
vote
0answers
21 views

Using unnormalized log likelihood for model comparison with different features?

Is it possible to directly use the log likelihood from fitting a model, for the purpose of model comparison? For example, if I'm using a logistic regression model, and I want to see if adding ...
0
votes
1answer
28 views

how does the loss function work in word2vec?

I was watching CS224n and I Came across this equation for word2vec loss function. As in the blue box, "for each document\training example t we are calculating the probability of context words given ...
1
vote
0answers
7 views

Tensorflow InvalidArgumentError: The determinant is not finite [closed]

I'm trying to fit a Mixture of Gaussians to a data set. First the data is clustered using K-Means Clustering. Each cluster is then fitted with a Gaussian.To avoid inversion of large covariance matrix, ...
1
vote
2answers
38 views

Understand a statement about likelihood function

I'm reading Agresti - Categorical Data Analysis and it says Consider two models, $M_0$ with fitted values $\hat{\mu}_0$ and $M_1$ with fitted values $\hat{\mu}_1$ with $M_0$ a special case of $M_1$....
0
votes
0answers
12 views

estimation conditional logit

I'm creating a code to estimate a conditional logit model. Comparing with the code that I adjust with the package "mlogit" I get very different coefficients, could you help me determine where I'm ...
3
votes
1answer
62 views

The form of the Log-Likelihood Function in Mixed Linear Models

Let us assume the following mixed effects model: $y = X\beta+Zu+e$ where $y$ is a vector of n observable random variables, $\beta$ is a vector of $p$ fixed effects, $X$ and $Z$ are known matrices, ...
0
votes
1answer
87 views

AIC Calculation using log likelihood

I have a dataset that has 40 experimental observations of cells' activity, $n=40$, I tested several models using each of these samples. The model can only explain one cell at a time due to variability ...
0
votes
0answers
50 views

Compute log-likelihood from sum of squares?

I have fit a 2D Gaussian to a surface in Matlab and need to compute the log-likelihood of this fit. Can One use the sum of squares between the Gaussian model and the actual surface to compute the log-...
1
vote
0answers
22 views

How to infer the number of states in a Hidden Markov Model with Gaussian mixture emissions

I have a time series made up of an unknown number of hidden states. Each state contains a set of values unique to that state. I am trying to use a GMM HMM (as implemented in Python's ...
1
vote
1answer
32 views

log in the M-step of the EM algorithm

In the M-step of the EM algorithm, you have to maximize the expected log-likelihood of X with respect to z which is: $ \int d z P(Z \mid X, \theta^{old}) \ln P(X \mid Z, \theta)$. Why do we maximize ...
0
votes
1answer
50 views

Calculating Log-Likelihood of Logistic Adaptive-Quadrature GLMM for Comparison with Fixed Model

Fitting a binary logistic GLMM here, with ungrouped data (all responses either 0 or 1). It says in this thread and in the documentation of anova.merMod that the ...
2
votes
1answer
68 views

Check if log-likelihood function is correctly derived

This question is a continuation of this one. By guesswork, I found out that $\vec{\theta}=(5.2,5.3,1.0)=$ $(A,B,C)$ was a good guess that made my model $$y_i=A\sin\left(\frac{x_i}{B}\right)+C\...
0
votes
1answer
31 views

$2D$ Maximum Likelihood Fit

I have read a couple of places that it is possible to do a $2D$ (or $3D$) maximum likelihood fit, but I can't seem to understand how this would work. Suppose I'm considering a probability distribution ...
0
votes
0answers
31 views

Fitting an ARMA-GARCH using AIC

I am trying to fit an ARFIMA(p,d,q)-GARCH(1,1) model to an asset returns time series. I start with an ARFIMA(0,0,0)-GARCH(1,1). The diagnostics tests like persistence requirement, Ljung Box test for ...
2
votes
1answer
33 views

Is it ever convenient to maximize different functions of the likelihood than the logarithm?

We all know that it's often much more convenient to maximize the log-likelihood rather than the likelihood to get a parameter estimate, since it amounts to the same thing by the fact that the ...
0
votes
0answers
33 views

Difference between sentence log-likelihood objective in Collobert at al. 2011 and the CRF objective function?

I'm a bit confused while trying to understand what is the difference between the sentence log-likelihood objective described in "Natural Language Processing (Almost) from Scratch" (Collobert at al. ...
2
votes
0answers
33 views

Log-likelihood calculation on separate test set

I'm looking for a "hack" in R that would allow me to calculate the log-likelihood of a GLM fit on a separate test set easily regardless of the distribution. For instance for a Gamma GLM, this is how ...
1
vote
0answers
34 views

Estimation Multinomial Logit [closed]

I need to create a code manually corresponding to the likelihood of the multinomial logit model in R. I have not been able to get the same results from some packages (mlogit, multinom). My database ...
0
votes
0answers
56 views

EM algorithm and AIC criteria

I am using EM algorithm to estimate the model parameters. EM-algorithm iterates until the loglikelihood is converged. After that, I need to compute AIC criteria. As known, AIC is a loglikelihood ...
0
votes
0answers
20 views

log likelihood computation by hand in R for time series data

I have tried to compute the loglikelihood function for time series data. ...
2
votes
1answer
45 views

Cox Proportional Hazard models for more than 2 treatments and covariates

I am trying to figure out how to properly interpret the results of this cox proportional hazard model, represented by a forest plot. I have looked into a lot of references, but almost all of them ...
0
votes
0answers
8 views

Nonparametric classification of a sample of values — is my approach correct?

Suppose I have a machine with a number of different labelled settings. The labels go from $1$ up to $L$. When I choose a setting on the dial, let's say setting $j$, I can have it output i.i.d. samples ...
2
votes
2answers
289 views

python computing likelihood causing exp overflow

I am using numpy to compute the likelihood of a variable $Z$ using numpy. $Z$ is a Bernoulli random variable which has two outcomes $[0,1]$. I compute the log likelihood of observing $Z$ given the ...
0
votes
0answers
8 views

Fisher information matrix of the mean of a circularly symmetric complex Gaussian distribution

Does the FIM always exist for the mean vector of a complex Gaussian distribution? The log-likelihood function of a circularly symmetric complex Gaussian distribution for a $K\times1$ vector of ...
0
votes
0answers
123 views

How to calculate WAIC from a JAGS model, and fix p_waic issue?

I am running a logistic regression type model in JAGS, and I noticed that I was getting different DIC scores (more than just a few points difference) between runs of the same model. I have a suspicion ...
0
votes
1answer
42 views

Why does the log-likelihood ratio test change so much with sample size, and what can I do about it?

I am doing a log-likelihood ratio test between seven models fitting a set of data with N=2 000 000. The models are nested; each one contains the same parameters as the last, and some more parameters ...
2
votes
2answers
46 views

Simple Log-likelihood question

I've got a simple question about deriving log-likelihoods. I am stumped by the following--> If the log-likelihood is: 𝑙(𝜆1,𝜆2) = 𝑦1 log(𝜆1𝐹1)−𝜆1𝐹1 −log((𝑦1)!)+𝑦2 log(𝜆2𝐹2) −𝜆2𝐹2 −log⁡ (...
0
votes
0answers
26 views

Derivation of a log-likelihood function for AR(1) process

The question is: "Suposse that: y$_t$=$\beta$y$_t$$_-$$_1$+s$_t$e$_t$; e$_t$~N(0, $\sigma$$^2$) s$_t$=exp{$\beta$y$_t$$_-$$_1$} Derivate the log-likelihood function for y$_0$=0 Assume that $\sigma^2$...
0
votes
0answers
24 views

Calculating deviance on validation data

Using R, I'd like to compare three nested logistic models with a binary outcome: one with just the covariates, one with weak predictors, and one with what I think is a strong predictor. I'm using glm ...
2
votes
1answer
37 views

Hyperparameter value while computing the test log-likelihood

I have a very basic machine learning question. My likelihood function includes a parameter $\alpha$ which I set to a fixed value and do not learn from the model, which makes it a "hyperparameter". ...
0
votes
0answers
38 views

MLE when the likelihood function itself contains a random variable — do I just integrate?

If I have a set of i.i.d. observations $\{x_1, x_2, \dots x_n \}$ drawn from a distribution $f(x ; \theta)$, I can form the MLE estimate $\hat{\theta}$ by finding the argmax of $\sum_i^n \ln\left[ L(\...
2
votes
1answer
53 views

Help solving for log likelihood

I need help solving the log-likelihood for the following problem: The solution is below: I'm curious about the steps to take in the process. I understand that we multiply the entire pmf n times, ...
1
vote
0answers
29 views

Interpreting score function in Cox model

Several sources state that the score function for the likelihood of a cox model is $$ \dfrac{\partial{}l(\beta)}{\partial\beta}=\Big(X_{i}\delta_i^T-\sum\limits_{i=1}^{n}\delta_i\dfrac{\sum\limits_{j\...
0
votes
0answers
13 views

Bayesian Inversion - choice of likelihood function and whether to invert for standard deviation

Good evening, There are my main questions before a brief explanation of my work: 1. Should I be inverting for multiple standard deviations (for different portions of the data, or even at each data ...
5
votes
1answer
167 views

Does a quadratic log-likehood mean the MLE is (approximately) normally distributed?

So, in the usual case, one can prove from the asymptotic normality of a maximum likelihood estimator that the corresponding log-likehood surface is quadratic near the MLE (e.g. in the proof of the ...
1
vote
0answers
74 views

Exchange expectation and gradient in likelihood function with matrix functions

Let us consider $L$ to be a random matrix, i.e. a matrix of random variables $x$, each of them with the probability mass function $f(x;\theta)$ where $\theta$ are the parameters of the pmf. For ...
0
votes
0answers
78 views

R optim (): finding confidence intervals [duplicate]

I am using R optim() function with log-likelihood to estimate parameter P of Binomial distribution. I want to know how can I estimate confidence interval of estimated parameter value, P.
0
votes
0answers
34 views

Gower vs Log Likelihood distance measure for mixed variables

I am working on distance measures and have used gone through the theory for two distance measure i.e. Gower distance measure and the log-likelihood distance measure. Am I correct to say that the ...
0
votes
0answers
166 views

Gradient function for log-likelihood

We are trying to calculate the gradient for a logistic regression where the log-likelihood function is: $$ll(x,y,\beta)=ln\left(\frac{\Pi {(e^{x_i\beta}})^{y_i}}{\Sigma {e^{x_i\beta}}}\right)$$ $x$ ...
0
votes
1answer
2k views

Can log likelihood function be negative

I do some optimization problem in R. I minimize the loglikelihood function. I found that the log-likelihood has a negative value. For example, I have this: -34.5. Then, when I count the AIC, I will ...
1
vote
0answers
162 views

Maximizing likelihood for noise-free Gaussian process regression

In "Machine Learning: A Probabilistic Perspective" the maximum marginal likelihood optimization for the kernel hyperparameters is explained for the noisy observation case. I am dealing with a noise-...
1
vote
0answers
43 views

Cox model with frailty (partial loglikelihood and df)

I am fitting a Cox propotional hazard model with a frailty term using the survival package in R. When I remove a fixed efect, I ...
4
votes
1answer
89 views

Determination of maximum log-likelihood of nonlinear model for calculation of Aikaike IC

I set up a kinetic model consisting of a system of ODE's. These ODE's are solved with an ode-solver (ode45) for varying parameters $k_1$ & $k_2$. The parameters are estimated using lsqnonlin (for ...
1
vote
0answers
47 views

Log Likelihood vs Chi Squared [duplicate]

Let's start with the preface that I am a linguist, not a mathematican. My knowledge of stats stopped after A-level oh-so-many years ago. So if answers could be written in simpleton terms, that would ...
0
votes
0answers
107 views

How to cross-validate a markov model

What is the process for validating the Markov model? Suppose I have 38 files which contain sequences(i.e A, B, B, C, E, D, E......). all the files contain different numbers of sequence (i.e. 1st file ...
0
votes
0answers
35 views

Simulation of Markov chain from Transition probability

I have 38 files of sequence (i.e., A, B, C, C, B, A...), from which I have calculated transition probability matrix. Now I want to cross-validate (leave one out CV) it. How can I do it? More precisely,...
1
vote
0answers
30 views

How to compare two modelled time courses using the log likelihood ratio test in R?

I have two different species in three replicates of which I measured 24 metrics during five days. I'm trying to decide when there is a significant difference between the two species for a specific ...
0
votes
1answer
64 views

Find maximum likelihood estimators of PDF

I am quite stumped by the following problem. The usual log-likelihood route with differentiation doesn't work. The problem is as follows: Let $X_1,...,X_n$ be i.i.d. random variables from a ...
0
votes
0answers
12 views

Calculating the likelihood ratio of two non linear models

This is probably a naive question - I've used the least squares method to fit both an exponential function and a hyperbolic function to my data. How do i now calculate the likelihood ratio?