Questions tagged [likelihood]

Given a random variable $X$ which arise from a parameterized distribution $F(X;θ)$, the likelihood is defined as the probability of observed data as a function of $θ$: $\operatorname{L}(θ | x)=\operatorname{P}(X=x \mid θ)$

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

Posterior distribution when the domain of the likelihood depends on the parameter

I am trying to calculate a posterior density given distribution and a prior. And I am a bit confused about how I should act as the domain of the distribution depends on the parameter. I am talking ...
user avatar
0 votes
0 answers
10 views

Randomly choose between options with multiple criteria

Here's the problem: I have some options. Each is represented with three attributes or say criteria (with normalized values between 0 and 1). I want to randomly choose one of these options based on ...
user avatar
  • 1
0 votes
0 answers
14 views

A problem about making an approximation to the integral over parameters -- eq (3.70) of Bishop's Pattern Recognition and Machine Learning

The problem comes from the paragraph containing equation (3.70) at the bottom of page 162 of Bishop's "Pattern Recognition and Machine Learning" which talks about an approximation to the ...
user avatar
  • 45
1 vote
1 answer
23 views

Deriving the expression for $p(\mathcal{K})$ where $\mathcal{K} = \{(\mathbf{s}^k,\mathbf{d}^k), k = 1,..., K\}$

This is a follow up from this question. Consider a model of diseases and symptoms. $s_i\in\{0,1\}$ is a binary random variable indicating whether the patient is showing the $i$-th symptom and $d_j\in ...
user avatar
  • 99
0 votes
0 answers
33 views

How to update a prior probability distribution of hurricane occurrence based on absence of hurricanes to date?

For a forecasting tournament, I am trying to forecast the number of Atlantic basin hurricanes in the 2022 hurricane season. I have reason to believe that my prior distribution looks as follows: At ...
user avatar
2 votes
0 answers
28 views

Is it circular reasoning to compute the ELBO using MCMC?

Let's say we have a posterior distribution $q(\theta) = p(\theta \mid D, \mathcal{M})$ over parameters $\theta$ given data $D$ and a model $\mathcal{M}$. As is often the case, computing $q$ is hard, ...
user avatar
  • 21
0 votes
0 answers
16 views

Obtaining log-likelihood value of a Markov chain when probability transition matrix contains exact-zero entries

I have a $n$ sequences $\boldsymbol{X}_1, \dots, \boldsymbol{X}_n$ of varying lengths arising from a Markov chain with a large state space $\mathcal{S} = \{ 1, 2, \dots, s \}$. Suppose the initial ...
user avatar
1 vote
1 answer
25 views

Terminology question regarding a certain "partial maximum likelihood" which approximates the marginal likelihood

Suppose that we have a model with many parameters, which we'll partition into two subvectors called $\theta$ and $\lambda$. In this situation, $\lambda$ corresponds to those parameters that are really ...
user avatar
2 votes
0 answers
32 views

parallelizing log-sum-exp

I have some approximate likelihoods: $L_1, \ldots, L_n$. Each is quite expensive to calculate. They're approximate because they use random numbers. Each of them is being calculated on the same data ...
user avatar
  • 18.8k
0 votes
0 answers
10 views

Mixed Models: can restricted models have larger likelihood than complete model?

I have estimated a mixed model of the form $\underline{Y} = \mathbf{X}\underline{\alpha} + \mathbf{Z}\underline{\beta} + \underline{\varepsilon}$, which has a few interaction terms and individually ...
user avatar
0 votes
0 answers
8 views

Reparametrizing a Uniform Prior Distribution to Multivariate Standard Normal

Problem Description I have a posterior distribution $$ p(\theta\mid y) \propto p(y \mid \theta) p(\theta) $$ with a uniform prior $p(\theta)= \mathcal{U}([a, b]^n)$, which is bounded. However, for my ...
user avatar
  • 1,498
0 votes
0 answers
38 views

How to express a likelihood function for the following regression?

I have asked this question in entirely different forms a number of times on StackExchange, to no avail. Between each question, I investigated the literature thoroughly, but I have yet to find a ...
user avatar
  • 1,040
0 votes
1 answer
26 views

Is Hessian of neural nets with NLL loss positive semi-definite?

I learned that expected Hessian of negative log likelihood is the same as Fisher information matrix, which is known to be positive semi-definite $$ \begin{aligned} F(\theta) &= E_{x \sim p_\theta}...
user avatar
2 votes
2 answers
36 views

Find a likelihood to calculate a posterior probability

I am having trouble understanding a basic Bayesian inference exercise: Suppose we are interested in inferring the proportion $\theta$ of individuals in a given population suffering from a certain ...
user avatar
  • 121
1 vote
0 answers
17 views

Bayesian inference, likelihood on positive data

Suppose I have a parameter $\theta$, that I know is positive, and some data $(x_1,x_2,\dots,x_n)$ on noisy realisations of the $\theta$. I then assume a prior with positive support on $\theta$ (...
user avatar
  • 11
2 votes
0 answers
28 views

Can log-likelihood test be applied to test two models which are not nested but nested within a full model?

If we have a response variable y and three predictor variables x1, x2, and x3 and M1 and M2 are nested within M3 where ...
user avatar
  • 131
0 votes
1 answer
14 views

Mixtures vs Multi-level models?

I'm confused on how mixture models and multi-level models are different (if at all.) Are there general rules for when to use one and not the other, pros/cons, etc?
user avatar
  • 1,706
2 votes
1 answer
61 views

Compute the Likelihood of binomial data

Say we have to following data: p = 0.95 -> rate of true positive result of pcr test. q = 0.1 -> rate of false positive result of pcr test. s = 0.2 -> rate of total patients in the population ...
user avatar
1 vote
0 answers
30 views

Multinomial likelihood function with data for only 2 of 3 outcomes

Can/should I use a binomial likelihood function if the data were generated from a multinomial process (3 possible outcomes) but data were only collected for two of the possible outcomes? In each trial ...
user avatar
  • 11
2 votes
1 answer
46 views

Log-transformation in negative log-likelihood for negative binomial distribution

I am performing negative log-likelihood maximization for success probability parameter of the negative binomial distribution avoiding numerical errors. I am not 100% sure if this procedure is valid, ...
user avatar
0 votes
0 answers
28 views

References for the conjugate prior to the beta distribution?

The Wikipedia article about "Conjugate Prior" has a table containing information about Likelihood Distributions with their Conjugate Priors. In the "Continuous Likelihood" table, ...
user avatar
  • 1
0 votes
0 answers
20 views

Nested sampling: What does "uniform sampling over the prior" mean?

I'm reading up on Nested Sampling in the book "Data Analysis - A Bayesian Tutorial" (Sivia and Skilling, 2006), and I do not understand the following: What I understand: Given a prior $\pi(\...
user avatar
0 votes
0 answers
13 views

Test for seasonality with LR-test?

I have an economic time series in monthly frequency. I want to test for seasonality using LR-Test. So the idea is to: Regress the time series y on a model with a time trend and 12 seasonal dummy ...
user avatar
1 vote
2 answers
61 views

Understanding the Evidence Lower Bound (ELBO)

I am reading this tutorial about Variational Inference, which includes the following depiction of ELBO as the lower bound on log-likelihood on the third page. In the tutorial, $x_i$ is the observed ...
user avatar
  • 1,487
3 votes
1 answer
67 views

The Likelihood Approach a.k.a. the 'third way' versus Bayesian

In his book "In All Likelihood" Yudi Pawitan writes that "the likelihood approach offers a distinct 'third way', a Bayesian-frequentist compromise. We might call it Fisherian as it owes ...
user avatar
1 vote
0 answers
44 views

Maximum Likelihood Estimator for Bernoulli distribution

Given a random sample $X_1, X_2,..., X_n$ from Bernoulli distribution. The log-likelihood function is: $\mathcal{L}(\theta) = \sum_1^n x_i^*\log{\theta} + (n - \sum_1^n x_i^*)\log{(1-\theta)}$ Score ...
user avatar
  • 139
2 votes
0 answers
31 views

Find the likelihood threshold for a Goodness-of-Fit test for multinomial data

Given a sample size $n \in \mathbb{N}$, a null hypothesis $H_0 = \langle p_1, p_2, \dots p_k\rangle$ which is an element of the $k$-dimensional probability simplex, and a significance threshold $\...
user avatar
5 votes
1 answer
266 views

Units for likelihoods and probabilities

In this discussion by comments Is the exact value of any likelihood meaningless?, it was suggested (firmly!) that likelihoods and probabilities calculated from continuous data not only have units, but ...
user avatar
  • 11.3k
1 vote
0 answers
39 views

Likelihood in mixture models

As per my understanding, normally, when we talk about Bayes rule, we write: p(z|x) = [p(x|z) * p(z)] / p(x) where, p(z|x) is called posterior p(x|z) is called likelihood p(z) is called prior p(x) is ...
user avatar
  • 161
12 votes
5 answers
2k views

Is the exact value of any likelihood meaningless?

While reading about likelihood, I have heard that "the exact value of any likelihood is meaningless" why? So, because of that we may use the likelihood ratio. So, my question is, why the ...
user avatar
  • 507
2 votes
1 answer
64 views

How to choose between mean squared error and likelihood?

I have a very simple data set with just one real valued feature ($x_i$) and a real valued target ($y_i$). My model assumes that the targets depend on the feature in a very simple way: for the features ...
user avatar
  • 285
2 votes
2 answers
49 views

How to rearrange exponent terms in a Gaussian likelihood?

I'm working out of the textbook "Bayesian Data Analysis for the Behavioral and Neural Sciences" by Todd Hudson, and on p. 105 (above) we see the preceding explanation for a Gaussian ...
user avatar
  • 23
2 votes
1 answer
55 views

Comparing log likelihood of "nested" GAM models (mgcv)

I am using the mgcv package to fit a GAM model to my data set to determine whether the trajectories along two (or more) different processes are different. More specifically I fit the following two ...
user avatar
  • 21
0 votes
0 answers
32 views

Likelihood of mixed discrete-continuous data

I'm struggling with the derivation of the likelihood with mixed continuous and discrete variables. Let us take this simple example: \begin{align*} X &= \begin{cases} 0 & \text{with ...
user avatar
  • 157
0 votes
0 answers
26 views

Weighing Maximum Likelihood Estimations

I'm trying to arrive at a time series of optimized parameter values $Z_t$ that maximizes the likelihood of occurrence of a specific time series $Y_t$. There is a subsample within the sample that ...
user avatar
0 votes
0 answers
18 views

How to interpret log likelihood [duplicate]

I am using negative binomial regression on panel data. I would appreciate help interpreting part of my regression table. The log likelihood is very low (~-7000) and doesn't change that much- does that ...
user avatar
0 votes
0 answers
30 views

Sample size affecting distribution of MLE estimator

I was working on an exercise that asked to describe the graph of the likelihood and log likelihood functions for a large sample size. Specifically it is a graph of boys = 6000 and girls = 4000. Prior ...
user avatar
0 votes
0 answers
18 views

Fisher Information usage [duplicate]

regarding fisher information in wikipedia, it is mentioned that fisher information is used in optimsl design of experiments. so an example is needed to illustrate how fisher information is used in ...
user avatar
0 votes
0 answers
20 views

How is the objective function of the different flavors of GARCH different?

How does the objective function/likelihood function of these different GARCH variations differ? Is it convex in all cases? Knowing convexity tells me whether some are not possible to find a globally ...
user avatar
  • 503
0 votes
1 answer
20 views

If a time-series achieves max-likelihood at GARCH(1,1), would EGARCH, or other GARCH variations achieve global maximum likelihood at p=1, q=1?

If I find that a time-series fits GARCH(1,1), would EGARCH, or other GARCH variations still be X-GARCH(1,1)?
user avatar
  • 503
0 votes
1 answer
16 views

Estimate distribution of variable from non-perfect predictions

Say I pull $n$ balls from a box while blinfolded. The balls can be either red or blue. I do not know the distribution of the balls. After that I receive a list predicting the color of every ball I ...
user avatar
3 votes
1 answer
128 views

How do location and scale parameters affect likelihoods?

Many statistical procedures rely on analyses of a likelihood function. Frequently, some (or all) of the parameters in that function are "location" and "scale" parameters. (See ...
user avatar
  • 293k
0 votes
0 answers
14 views

Expectation Maximization: How to perform "E" step in coin flipping example? [duplicate]

I recently read this primer that does a great job of explaining the principles of the Expectation-Maximization (EM) "algorithm." However, I'm confused about how they calculated the ...
user avatar
  • 211
2 votes
1 answer
77 views

How to get most likely value with confidence interval or expected value from a set of observations

lets say I have a set of values (R-code below for creating a vector) ...
user avatar
  • 153
0 votes
0 answers
18 views

Find fisher information matrix for optimization estimator

I have that $$f(x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}x^2}$$ I have the conditional distribution: $f_{\beta}(y|x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}(y-\beta_0-\beta_1x-\beta_2x^2)^2}$ and we have ...
user avatar
  • 253
0 votes
0 answers
7 views

How does the lifelines package calculate the CoxPHFitter log-likelihood? [duplicate]

I'm looking for the equation that how the lifelines package calculates the log-likelihood of CoxPHFitter. I read the lifelines documentation but cannot find where is it. Any help would be appreciated!
user avatar
0 votes
0 answers
41 views

Marginal Likelihood and Its Intractability [duplicate]

In case of VAE, it is said that the posterior distribution is intractable because the marginal likelihood is intractable. My understanding as to why marginal likelihood is intractable: z can have ...
user avatar
  • 161
2 votes
1 answer
37 views

Estimating Mixture Models with Maximum Likelihood

Suppose you have a Normal Mixture Model with 2 Components - you could write this model as follows: $\pi_1 N(\mu_1, \sigma_1) + \pi_2 N(\mu_2, \sigma_2)$ In the above model, there are 6 unknowns : $\...
user avatar
  • 6,094
1 vote
1 answer
59 views

Does approximating the likelihood function violate the likelihood principle in Bayesian Inference?

Suppose we have a prior $p(\theta)$ and a likelihood function $L(\theta|x)$, and that the likelihood $L(\theta|x)$ is intractable somehow (difficult or impossible to compute) and we instead replace it ...
user avatar
6 votes
1 answer
144 views

Expected Fisher information isn't positive definite for truncated normal with heteroskedasticity

This question is about having a non-positive-definite expected Fisher information in a normal model in which observations have different dispersions. Consider this simple normal model: $$Y_i \sim N(\...
user avatar
  • 3,501

1
2 3 4 5
28