Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [log-likelihood]

the logarithm of the likelihood function.

1
vote
1answer
8 views

Having trouble figuring out how loss was calculated for SQuAD task in BERT paper

The BERT Paper https://arxiv.org/pdf/1810.04805.pdf Section 4.2 covers the SQuAD training. So from my understanding, there are two extra parameters trained, they are two vectors with the same ...
1
vote
0answers
18 views

log-likelihood of normal distributed fitted using MLE

Suppose we fit a normal using MLE which means we have parameters $$\mu = \frac{1}{n}\sum_{i=1}^n x_i$$ and $$\sigma^2 = \frac{1}{n}\sum_{i=1}^n(x_i - \mu)^2$$ Then we calculate the log-likelihood as ...
0
votes
0answers
12 views

Comparing log-likelihoods for different datasets of the same size?

Does it make sense to compare log likelihood values from different datasets if the datasets are of the same nature and size? Like let's say I fit a model on daily S&P 500 returns from 2000-2010 ...
1
vote
1answer
53 views

Binomial distributed random sample: find the least variance from the set of all unbiased estimators of $\theta$

Let $X_{1},X_{2},\ldots,X_{n}$ be random sample from $X\sim\text{Binomial}(2,\theta)$. (a) Find the least variance from the set of all unbiased estimators of $\theta$. (b) Find a sufficient ...
1
vote
1answer
48 views

Normal distributed random sample: find the least variance from the set of all unbiased estimators of $\theta$

Let $X_{1},X_{2},\ldots,X_{n}$ be a random sample from $X\sim\mathcal{N}(0,\sigma^{2})$. (a) Find the least variance from the set of all unbiased estimators of $\sigma^{2}$. (b) Find a sufficient ...
3
votes
1answer
49 views

AIC for increasing sample size

I am using AIC as a model selection criteria in one of my projects. However, since AIC isn't dependent on the number of points sampled, for large n the log likelihood term rapidly outscales the ...
2
votes
1answer
39 views

trouble creating a negative log likelihood for a linear model in R [closed]

I am new to stats and R and I am having trouble figuring out how to calculate a function that gives me the NLL of a linear regression.
1
vote
0answers
26 views

How do I calculate the negative-log likelihood of a linear regression model in r? [closed]

I need to calculate the NLL of the regression and minimize it. How I can do this in R?
1
vote
1answer
127 views

Understanding the log-likelihood (score) in scikit-learn GMM

I have been training a GMM (Gaussian Mixture, clustering / unsupervised) on two version of the same dataset: one training with all its features and one training after a PCA truncated to its 2 first ...
0
votes
1answer
38 views

Gaussian process likelihood function

I'm trying to understand the likelihood function in Gaussian Process. The book by Rasmussen et al. defined Gaussian Process lml as $$log~p(y|X) = -\frac{1}{2}y^T\alpha-\sum log L_{ii} - \frac{N}{2}...
1
vote
1answer
21 views

Definition of curvature

Kay (Fundamentals of Statistical Signal Processing) defines the curvature of a log-likelihood function to be the "negative of the second derivative of the logarithm of the likelihood function at its ...
1
vote
1answer
104 views

Find (or calculate) log-likelihood value, AIC, and BIC for SUR model (for each equation) with systemfit

I have estimated SUR model with systemfit (R package). With the estimated results, I am trying to get logLik, AIC and BIC for ...
0
votes
0answers
17 views

Minimizing numerically a nondifferentiable function

Many (likelihood) functions are not differentiable at the optimal point. Does this cause problems a) in the numerical methods used to minimize the function based on the gradient? b) in the statistical ...
1
vote
0answers
13 views

Log or MSE loss for hyperparameter tuning of probabilistic NN

I am building a predictive model of a dynamical system using a NN whose output neurons enconde the mean and diagonal covariance of a Gaussian distribution. For training, the negative log prediction ...
2
votes
0answers
29 views

R - Fitting a Coxian phase-type distribution to data

Using R, I would like to fit a Coxian phase-type distribution to a vector of waiting times ($t_i$) where the $\mu$'s and $\lambda$'s are unknown i.e. I want to find ...
1
vote
0answers
54 views

Maximum likelihood estimator of the parameter of randomness in Watts and Strogatz's model (1998)

According to the paper Menezes, M. B., Kim, S., & Huang, R. (2017). Constructing a Watts-Strogatz network from a small-world network with symmetric degree distribution. PloS one, 12(6), e0179120,...
1
vote
0answers
29 views

Using unnormalized log likelihood for model comparison with different features?

Is it possible to directly use the log likelihood from fitting a model, for the purpose of model comparison? For example, if I'm using a logistic regression model, and I want to see if adding ...
0
votes
1answer
111 views

how does the loss function work in word2vec?

I was watching CS224n and I Came across this equation for word2vec loss function. As in the blue box, "for each document\training example t we are calculating the probability of context words given ...
1
vote
0answers
21 views

Tensorflow InvalidArgumentError: The determinant is not finite [closed]

I'm trying to fit a Mixture of Gaussians to a data set. First the data is clustered using K-Means Clustering. Each cluster is then fitted with a Gaussian.To avoid inversion of large covariance matrix, ...
1
vote
2answers
43 views

Understand a statement about likelihood function

I'm reading Agresti - Categorical Data Analysis and it says Consider two models, $M_0$ with fitted values $\hat{\mu}_0$ and $M_1$ with fitted values $\hat{\mu}_1$ with $M_0$ a special case of $M_1$....
0
votes
0answers
21 views

estimation conditional logit

I'm creating a code to estimate a conditional logit model. Comparing with the code that I adjust with the package "mlogit" I get very different coefficients, could you help me determine where I'm ...
3
votes
1answer
68 views

The form of the Log-Likelihood Function in Mixed Linear Models

Let us assume the following mixed effects model: $y = X\beta+Zu+e$ where $y$ is a vector of n observable random variables, $\beta$ is a vector of $p$ fixed effects, $X$ and $Z$ are known matrices, ...
0
votes
1answer
162 views

AIC Calculation using log likelihood

I have a dataset that has 40 experimental observations of cells' activity, $n=40$, I tested several models using each of these samples. The model can only explain one cell at a time due to variability ...
0
votes
0answers
50 views

Compute log-likelihood from sum of squares?

I have fit a 2D Gaussian to a surface in Matlab and need to compute the log-likelihood of this fit. Can One use the sum of squares between the Gaussian model and the actual surface to compute the log-...
1
vote
0answers
38 views

How to infer the number of states in a Hidden Markov Model with Gaussian mixture emissions

I have a time series made up of an unknown number of hidden states. Each state contains a set of values unique to that state. I am trying to use a GMM HMM (as implemented in Python's ...
1
vote
1answer
48 views

log in the M-step of the EM algorithm

In the M-step of the EM algorithm, you have to maximize the expected log-likelihood of X with respect to z which is: $ \int d z P(Z \mid X, \theta^{old}) \ln P(X \mid Z, \theta)$. Why do we maximize ...
0
votes
1answer
63 views

Calculating Log-Likelihood of Logistic Adaptive-Quadrature GLMM for Comparison with Fixed Model

Fitting a binary logistic GLMM here, with ungrouped data (all responses either 0 or 1). It says in this thread and in the documentation of anova.merMod that the ...
2
votes
1answer
70 views

Check if log-likelihood function is correctly derived

This question is a continuation of this one. By guesswork, I found out that $\vec{\theta}=(5.2,5.3,1.0)=$ $(A,B,C)$ was a good guess that made my model $$y_i=A\sin\left(\frac{x_i}{B}\right)+C\...
0
votes
1answer
31 views

$2D$ Maximum Likelihood Fit

I have read a couple of places that it is possible to do a $2D$ (or $3D$) maximum likelihood fit, but I can't seem to understand how this would work. Suppose I'm considering a probability distribution ...
0
votes
0answers
46 views

Fitting an ARMA-GARCH using AIC

I am trying to fit an ARFIMA(p,d,q)-GARCH(1,1) model to an asset returns time series. I start with an ARFIMA(0,0,0)-GARCH(1,1). The diagnostics tests like persistence requirement, Ljung Box test for ...
2
votes
1answer
34 views

Is it ever convenient to maximize different functions of the likelihood than the logarithm?

We all know that it's often much more convenient to maximize the log-likelihood rather than the likelihood to get a parameter estimate, since it amounts to the same thing by the fact that the ...
0
votes
0answers
39 views

Difference between sentence log-likelihood objective in Collobert at al. 2011 and the CRF objective function?

I'm a bit confused while trying to understand what is the difference between the sentence log-likelihood objective described in "Natural Language Processing (Almost) from Scratch" (Collobert at al. ...
2
votes
0answers
43 views

Log-likelihood calculation on separate test set

I'm looking for a "hack" in R that would allow me to calculate the log-likelihood of a GLM fit on a separate test set easily regardless of the distribution. For instance for a Gamma GLM, this is how ...
1
vote
0answers
34 views

Estimation Multinomial Logit [closed]

I need to create a code manually corresponding to the likelihood of the multinomial logit model in R. I have not been able to get the same results from some packages (mlogit, multinom). My database ...
0
votes
0answers
126 views

EM algorithm and AIC criteria

I am using EM algorithm to estimate the model parameters. EM-algorithm iterates until the loglikelihood is converged. After that, I need to compute AIC criteria. As known, AIC is a loglikelihood ...
0
votes
0answers
23 views

log likelihood computation by hand in R for time series data

I have tried to compute the loglikelihood function for time series data. ...
2
votes
1answer
58 views

Cox Proportional Hazard models for more than 2 treatments and covariates

I am trying to figure out how to properly interpret the results of this cox proportional hazard model, represented by a forest plot. I have looked into a lot of references, but almost all of them ...
0
votes
0answers
8 views

Nonparametric classification of a sample of values — is my approach correct?

Suppose I have a machine with a number of different labelled settings. The labels go from $1$ up to $L$. When I choose a setting on the dial, let's say setting $j$, I can have it output i.i.d. samples ...
2
votes
2answers
392 views

python computing likelihood causing exp overflow

I am using numpy to compute the likelihood of a variable $Z$ using numpy. $Z$ is a Bernoulli random variable which has two outcomes $[0,1]$. I compute the log likelihood of observing $Z$ given the ...
0
votes
0answers
10 views

Fisher information matrix of the mean of a circularly symmetric complex Gaussian distribution

Does the FIM always exist for the mean vector of a complex Gaussian distribution? The log-likelihood function of a circularly symmetric complex Gaussian distribution for a $K\times1$ vector of ...
0
votes
0answers
199 views

How to calculate WAIC from a JAGS model, and fix p_waic issue?

I am running a logistic regression type model in JAGS, and I noticed that I was getting different DIC scores (more than just a few points difference) between runs of the same model. I have a suspicion ...
0
votes
1answer
54 views

Why does the log-likelihood ratio test change so much with sample size, and what can I do about it?

I am doing a log-likelihood ratio test between seven models fitting a set of data with N=2 000 000. The models are nested; each one contains the same parameters as the last, and some more parameters ...
2
votes
2answers
51 views

Simple Log-likelihood question

I've got a simple question about deriving log-likelihoods. I am stumped by the following--> If the log-likelihood is: 𝑙(𝜆1,𝜆2) = 𝑦1 log(𝜆1𝐹1)−𝜆1𝐹1 −log((𝑦1)!)+𝑦2 log(𝜆2𝐹2) −𝜆2𝐹2 −log⁡ (...
0
votes
0answers
29 views

Derivation of a log-likelihood function for AR(1) process

The question is: "Suposse that: y$_t$=$\beta$y$_t$$_-$$_1$+s$_t$e$_t$; e$_t$~N(0, $\sigma$$^2$) s$_t$=exp{$\beta$y$_t$$_-$$_1$} Derivate the log-likelihood function for y$_0$=0 Assume that $\sigma^2$...
0
votes
0answers
29 views

Calculating deviance on validation data

Using R, I'd like to compare three nested logistic models with a binary outcome: one with just the covariates, one with weak predictors, and one with what I think is a strong predictor. I'm using glm ...
2
votes
1answer
38 views

Hyperparameter value while computing the test log-likelihood

I have a very basic machine learning question. My likelihood function includes a parameter $\alpha$ which I set to a fixed value and do not learn from the model, which makes it a "hyperparameter". ...
0
votes
0answers
39 views

MLE when the likelihood function itself contains a random variable — do I just integrate?

If I have a set of i.i.d. observations $\{x_1, x_2, \dots x_n \}$ drawn from a distribution $f(x ; \theta)$, I can form the MLE estimate $\hat{\theta}$ by finding the argmax of $\sum_i^n \ln\left[ L(\...
2
votes
1answer
55 views

Help solving for log likelihood

I need help solving the log-likelihood for the following problem: The solution is below: I'm curious about the steps to take in the process. I understand that we multiply the entire pmf n times, ...
1
vote
0answers
30 views

Interpreting score function in Cox model

Several sources state that the score function for the likelihood of a cox model is $$ \dfrac{\partial{}l(\beta)}{\partial\beta}=\Big(X_{i}\delta_i^T-\sum\limits_{i=1}^{n}\delta_i\dfrac{\sum\limits_{j\...
0
votes
0answers
18 views

Bayesian Inversion - choice of likelihood function and whether to invert for standard deviation

Good evening, There are my main questions before a brief explanation of my work: 1. Should I be inverting for multiple standard deviations (for different portions of the data, or even at each data ...