Questions tagged [likelihood]

Given a random variable $X$ which arise from a parameterized distribution $F(X;θ)$, the likelihood is defined as the probability of observed data as a function of $θ$: $\operatorname{L}(θ | x)=\operatorname{P}(X=x \mid θ)$

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

Test for seasonality with LR-test?

I have an economic time series in monthly frequency. I want to test for seasonality using LR-Test. So the idea is to: Regress the time series y on a model with a time trend and 12 seasonal dummy ...
user avatar
-1 votes
0 answers
8 views

How to estimate the log-likelihood of 2 independent coins [closed]

Coin A has a probability p of landing on heads, and coin B has a probability q of landing on heads. Calculate log-likelihood of p and q given X
user avatar
-1 votes
0 answers
15 views

True or false and why true and why false [closed]

i) In the presence of multicollinearity, the variance of OLS estimators are quite fine only that t and F test are highly misleading. ii) Forecast is the quantitative estimation of the likelihood of ...
user avatar
1 vote
2 answers
41 views

Understanding the Evidence Lower Bound (ELBO)

I am reading this tutorial about Variational Inference, which includes the following depiction of ELBO as the lower bound on log-likelihood on the third page. In the tutorial, $x_i$ is the observed ...
user avatar
  • 1,427
3 votes
1 answer
63 views

The Likelihood Approach a.k.a. the 'third way' versus Bayesian

In his book "In All Likelihood" Yudi Pawitan writes that "the likelihood approach offers a distinct 'third way', a Bayesian-frequentist compromise. We might call it Fisherian as it owes ...
user avatar
1 vote
0 answers
34 views

Maximum Likelihood Estimator for Bernoulli distribution

Given a random sample $X_1, X_2,..., X_n$ from Bernoulli distribution. The log-likelihood function is: $\mathcal{L}(\theta) = \sum_1^n x_i^*\log{\theta} + (n - \sum_1^n x_i^*)\log{(1-\theta)}$ Score ...
user avatar
  • 129
2 votes
0 answers
28 views

Find the likelihood threshold for a Goodness-of-Fit test for multinomial data

Given a sample size $n \in \mathbb{N}$, a null hypothesis $H_0 = \langle p_1, p_2, \dots p_k\rangle$ which is an element of the $k$-dimensional probability simplex, and a significance threshold $\...
user avatar
4 votes
1 answer
250 views

Units for likelihoods and probabilities

In this discussion by comments Is the exact value of any likelihood meaningless?, it was suggested (firmly!) that likelihoods and probabilities calculated from continuous data not only have units, but ...
user avatar
  • 11.3k
1 vote
0 answers
34 views

Likelihood in mixture models

As per my understanding, normally, when we talk about Bayes rule, we write: p(z|x) = [p(x|z) * p(z)] / p(x) where, p(z|x) is called posterior p(x|z) is called likelihood p(z) is called prior p(x) is ...
user avatar
  • 161
11 votes
5 answers
1k views

Is the exact value of any likelihood meaningless?

While reading about likelihood, I have heard that "the exact value of any likelihood is meaningless" why? So, because of that we may use the likelihood ratio. So, my question is, why the ...
user avatar
  • 457
2 votes
1 answer
63 views

How to choose between mean squared error and likelihood?

I have a very simple data set with just one real valued feature ($x_i$) and a real valued target ($y_i$). My model assumes that the targets depend on the feature in a very simple way: for the features ...
user avatar
  • 407
2 votes
2 answers
47 views

How to rearrange exponent terms in a Gaussian likelihood?

I'm working out of the textbook "Bayesian Data Analysis for the Behavioral and Neural Sciences" by Todd Hudson, and on p. 105 (above) we see the preceding explanation for a Gaussian ...
user avatar
  • 23
2 votes
1 answer
49 views

Comparing log likelihood of "nested" GAM models (mgcv)

I am using the mgcv package to fit a GAM model to my data set to determine whether the trajectories along two (or more) different processes are different. More specifically I fit the following two ...
user avatar
  • 21
0 votes
0 answers
29 views

Likelihood of mixed discrete-continuous data

I'm struggling with the derivation of the likelihood with mixed continuous and discrete variables. Let us take this simple example: \begin{align*} X &= \begin{cases} 0 & \text{with ...
user avatar
  • 157
0 votes
0 answers
26 views

Weighing Maximum Likelihood Estimations

I'm trying to arrive at a time series of optimized parameter values $Z_t$ that maximizes the likelihood of occurrence of a specific time series $Y_t$. There is a subsample within the sample that ...
user avatar
0 votes
0 answers
17 views

How to interpret log likelihood [duplicate]

I am using negative binomial regression on panel data. I would appreciate help interpreting part of my regression table. The log likelihood is very low (~-7000) and doesn't change that much- does that ...
user avatar
0 votes
0 answers
29 views

Sample size affecting distribution of MLE estimator

I was working on an exercise that asked to describe the graph of the likelihood and log likelihood functions for a large sample size. Specifically it is a graph of boys = 6000 and girls = 4000. Prior ...
user avatar
0 votes
0 answers
16 views

Fisher Information usage [duplicate]

regarding fisher information in wikipedia, it is mentioned that fisher information is used in optimsl design of experiments. so an example is needed to illustrate how fisher information is used in ...
user avatar
0 votes
0 answers
17 views

How is the objective function of the different flavors of GARCH different?

How does the objective function/likelihood function of these different GARCH variations differ? Is it convex in all cases? Knowing convexity tells me whether some are not possible to find a globally ...
user avatar
  • 285
0 votes
1 answer
20 views

If a time-series achieves max-likelihood at GARCH(1,1), would EGARCH, or other GARCH variations achieve global maximum likelihood at p=1, q=1?

If I find that a time-series fits GARCH(1,1), would EGARCH, or other GARCH variations still be X-GARCH(1,1)?
user avatar
  • 285
0 votes
1 answer
16 views

Estimate distribution of variable from non-perfect predictions

Say I pull $n$ balls from a box while blinfolded. The balls can be either red or blue. I do not know the distribution of the balls. After that I receive a list predicting the color of every ball I ...
user avatar
3 votes
1 answer
120 views

How do location and scale parameters affect likelihoods?

Many statistical procedures rely on analyses of a likelihood function. Frequently, some (or all) of the parameters in that function are "location" and "scale" parameters. (See ...
user avatar
  • 290k
0 votes
0 answers
14 views

Expectation Maximization: How to perform "E" step in coin flipping example? [duplicate]

I recently read this primer that does a great job of explaining the principles of the Expectation-Maximization (EM) "algorithm." However, I'm confused about how they calculated the ...
user avatar
  • 211
2 votes
1 answer
73 views

How to get most likely value with confidence interval or expected value from a set of observations

lets say I have a set of values (R-code below for creating a vector) ...
user avatar
  • 153
0 votes
0 answers
15 views

Find fisher information matrix for optimization estimator

I have that $$f(x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}x^2}$$ I have the conditional distribution: $f_{\beta}(y|x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}(y-\beta_0-\beta_1x-\beta_2x^2)^2}$ and we have ...
user avatar
  • 253
0 votes
0 answers
7 views

How does the lifelines package calculate the CoxPHFitter log-likelihood? [duplicate]

I'm looking for the equation that how the lifelines package calculates the log-likelihood of CoxPHFitter. I read the lifelines documentation but cannot find where is it. Any help would be appreciated!
user avatar
0 votes
0 answers
40 views

Marginal Likelihood and Its Intractability [duplicate]

In case of VAE, it is said that the posterior distribution is intractable because the marginal likelihood is intractable. My understanding as to why marginal likelihood is intractable: z can have ...
user avatar
  • 161
2 votes
1 answer
33 views

Estimating Mixture Models with Maximum Likelihood

Suppose you have a Normal Mixture Model with 2 Components - you could write this model as follows: $\pi_1 N(\mu_1, \sigma_1) + \pi_2 N(\mu_2, \sigma_2)$ In the above model, there are 6 unknowns : $\...
user avatar
  • 5,884
1 vote
1 answer
53 views

Does approximating the likelihood function violate the likelihood principle in Bayesian Inference?

Suppose we have a prior $p(\theta)$ and a likelihood function $L(\theta|x)$, and that the likelihood $L(\theta|x)$ is intractable somehow (difficult or impossible to compute) and we instead replace it ...
user avatar
6 votes
1 answer
138 views

Expected Fisher information isn't positive definite for truncated normal with heteroskedasticity

This question is about having a non-positive-definite expected Fisher information in a normal model in which observations have different dispersions. Consider this simple normal model: $$Y_i \sim N(\...
user avatar
  • 3,501
0 votes
0 answers
70 views

Question about likelihood function in Bayes Rule and marginalization

The following is an instance of Bayes' Rule: $$P(\alpha, \beta, \gamma, \delta, \epsilon|\mathbf{X}) = \frac{P(\mathbf{X}|\alpha, \beta, \gamma, \delta, \epsilon)P(\alpha, \beta, \gamma, \delta, \...
user avatar
  • 389
1 vote
1 answer
41 views

How to sample and compute the likelihood from a Mollified Uniform distribution?

I want to draw samples from the mollified Uniform distribution presented in another Cross Validated thread, cf the answer from whuber. What is the best way to do so?...
user avatar
0 votes
0 answers
18 views

Equivalent ways of writing the log-likelihood of a sample of normal RVs

I am going through my econometrics textbook right now and the textbook writes the log-likelihood equation for a sample of normal random variables in a way I have never seen before. Specifically, for a ...
user avatar
  • 135
2 votes
2 answers
410 views

AIC, BIC and log likelihood which more important?

I am currently searching for the best ARMA(p,q) model for my conditional mean. When comparing the AIC, BIC and LL, I see that some model perform better in AIC, some in BIC and some in LL. The AC and ...
user avatar
  • 41
0 votes
0 answers
30 views

Log-likelihood Gaussian cox process

I have a question related to the slide below. How do you obtain the log-likelihood function? Isn't it equal to: $$ \displaystyle \sum_{x,t} \log\left( \int_{-\infty}^{+\infty} f(D_{x,t} = d_{x,t} \mid ...
user avatar
  • 1
2 votes
0 answers
27 views

Hidden Markov Model observing sequences

I have been trying to understand Hidden Markov Models but I often find myself confused. I have discussed with my tutor for further help however, he is often rude and does not help and so I have ...
user avatar
  • 133
0 votes
1 answer
21 views

Bayesian estimation under transformation on the paramater

Consider the classical model Normal-Normal-Inserse-Gamma model: $$ x=(x_1,...,x_n)|\mu,\sigma^2\sim N(\mu,\sigma^2)\,\,(iid),\,\,\mu\sim N(m_0,\tau),\sigma^2\sim IG(a,b), $$ where $m_0,\tau,a,b$ are ...
user avatar
  • 53
0 votes
1 answer
41 views

multilevel logistic regression via maximum likelihood in R

I want to develop a mixed effect logistic regression in R using likelihood function and compare the results (estimated parameters) with the output of glmer function. I couldn't find a good material ...
user avatar
0 votes
0 answers
46 views

Likelihood and cross-entropy: continuous case

I think it's pretty clear to me that average log-likelihood is equivalent to negative cross-entropy for discrete distributions, as shown here: $$\frac{1}{N}\log\mathcal{L}(\theta) = \frac{1}{N}\log \...
user avatar
0 votes
0 answers
15 views

Is the likelihood of a discrete binomial variable same as it probability? Like in the case of tossing a coin lets say 12 times

I am working on a probability project where we first generate random variables for a given Binomial experiment and then we generate a PMF for 10 coin tosses using the list of random variables we ...
user avatar
  • 1
0 votes
1 answer
40 views

fitting left censored models using software for right censored data

When analyzing a lognormal data with left-censored values using a regression model, I have read that you can use methods that fit right-censored data but “flip” the data by subtracting from some large ...
user avatar
  • 2,262
1 vote
1 answer
14 views

Impact of Laplace smoothing on likelihood in Naive Bayes

When 1 is added to word count in Laplace Smoothing in Naive Bayes, the new probabilities either increase or decrease as shown below. Though the problem of "zero" probability has been solved. ...
user avatar
0 votes
1 answer
59 views

How to use R to plot $f_n(\theta)$ based on following samples? [closed]

For iid random samples $X_1,\dots, X_n$ with mean value $E X_1=\theta$, take $X_{(1)}=\min \{X_1,\dots, X_n\}$ and $X_{(n)}=\max \{X_1,\dots, X_n\}$. If $\lambda$ solves equation $$ \sum_{i=1}^n \frac{...
user avatar
  • 274
1 vote
1 answer
61 views

How to interpret a logistic regression model with negative coefficients of varying magnitudes and odds ratios <1?

I'm doing logistic regression where X has four factors (1-4) and Y has two factors (0-1). I did: ...
user avatar
2 votes
1 answer
103 views

Does joint probability apply when calculating likelihood with the binomial distribution?

I am trying to understand this picture from Statquest in the light of the Wikipedia statement that Likelihood describes the joint probability of the observed data as a function of the parameters of ...
user avatar
  • 461
0 votes
1 answer
82 views

Why are mixed models robust to missing (at random) data in the response variable?

I have read that mixed effects models are well equipped to handle missing (at random) response data if estimated using likelihood methods. However, I am yet to find a clear (not overly technical) ...
user avatar
1 vote
0 answers
35 views

Likelihood for a three level linear mixed effect model

Consider the therapist data described in https://rpsychologist.com/r-guide-longitudinal-lme-lmer. We have patients ($j$) clustered with respect to therapists (i). Let $u_{0ij} $ be patient-specific ...
user avatar
0 votes
0 answers
17 views

Likelihood for left-censored data in one dimension and uncensored in another dimension

I need to maximize a likelihood for parameters $\vec{\theta}$ given a model $\vec{g}(x_i,\vec{\theta})$ (let's say this is a non-linear black box) and observed data $(x_i,\vec{y}_i)$. The dependent ...
user avatar
  • 1
0 votes
0 answers
32 views

Fisher Information and MLE

I know that the Fisher Information is defined as the variance of the score function: $$ I(\theta)=Var(\frac{d}{d\theta}\mathrm{log}L(x|\theta))=\int(\frac{d}{d\theta}\mathrm{log}f(x|\theta))^2p_\theta(...
user avatar
0 votes
0 answers
28 views

What is the number of free parameters in an n-component GMM?

I am trying to calculate BIC = -2logL + log(N)d where d is the number of free parameters or degrees of freedom. If I am fitting guassian mixture model to the data, ...
user avatar
  • 1

1
2 3 4 5
28