Questions tagged [likelihood]

Given a random variable $X$ which arise from a parameterized distribution $F(X;θ)$, the likelihood is defined as the probability of observed data as a function of $θ$: $\operatorname{L}(θ | x)=\operatorname{P}(X=x \mid θ)$

Filter by
Sorted by
Tagged with
1 vote
2 answers
27 views

Normalising likelihood for BIC/AIC calculation

I am running some model inference using AIC and BIC. My problem is that when I go and calculate the (maximum) loglikelihoods of my models, they are usually really high (range between 4700 and 1400 ...
  • 13
0 votes
1 answer
38 views

How to determine if the log likelihood of logistic regression is too large or not?

I am running a logistic regression on STATA with binary response variable, and 2 predictor variable that are discrete, as such one is in % (but takes only 2 values strictly i.e., 5% or 10%) and ...
  • 51
1 vote
2 answers
40 views

Is it practical to derive the prior distribution by dividing the posterior by the likelihood and multiplying by the "evidence"?

Is it practical to derive the optimal prior distribution by dividing the posterior by the likelihood and multiplying by the "evidence"? Suppose you assume a probability distribution. You ...
0 votes
0 answers
22 views

Marginal Likelihood Computation for Bayesian Linear Model

Given a simple Bayesian linear model with $N$ observations $y = X\beta + \varepsilon \quad \quad \varepsilon \sim \mathcal{N}(0, \Sigma)$ with known error variance-covariance matrix $\Sigma$ and ...
  • 651
2 votes
0 answers
32 views

Likelihood of a random vector with each component following a different distribution

How do you write down the likelihood for random vectors when each component follows a different distribution with a dependence structure? For example, Suppose there are n-random vectors, mutually ...
0 votes
0 answers
13 views

Which is the likelihood function of the logit model? [duplicate]

I'm wondering which is the likelihood function for a logit model and how I can derive it. Thanks!
1 vote
1 answer
30 views

Derivation of Box-Cox and Yeo-Johnson Log-Likelihood Functions

The scipy documention lists expressions for the Log-likelihood functions for the Box-Cox and Yeo-Johnson transformations here and here. I'm looking for a source ...
  • 314
0 votes
0 answers
43 views

How to find the regression model log likelihood of data $(x, z, b) $ where $b$ indicates whether $y > z? $

I have a dataset where I don't have the exact output labels $y$ but what I have is if $y$ is larger or smaller than another value $z.$ There is another binary parameter $b $ that decides if y is ...
  • 1
2 votes
1 answer
47 views

Coding the likelihood function for logistic regression

I would appreciate help in understanding if I made a correct interpretation and coding of the likelihood function for logistic regression. Background: For a task I am going to write a function in <...
  • 23
0 votes
0 answers
17 views

Does adding more covariates always increase the condition number of the Hessian, and can you have a high-condition number but higher log-likelihood?

Does adding more covariates always increase the condition number of the Hessian, and can you have a high-condition number but higher log-likelihood/more optimal model? Should we ever not use the model ...
2 votes
1 answer
39 views

Where information comes for Binomial Likelihood?

Suppose that we have replicates from a Binomial distribution, i.e. $$n_{1},n_{2},...,n_{R}\sim Bin(N,p)$$ Then the likelihood can be written as $L(p,N|n_{1},n_{2},...,n_{R})=\prod_{r=1}^{R}\binom{N}{...
  • 1,883
2 votes
2 answers
57 views

$L(\theta;x)=f(x;\theta)$ vs. $L(\theta;x)\propto f(x|\theta)$

My second-year notes in Statistical Inference and Modelling (unpublished) have a definition, The likelihood function of $X$, given the data $x$, is $L∶\Theta\rightarrow\mathbb{R}$ defined by $L(\...
  • 343
0 votes
0 answers
10 views

Variational Autoencoder Not Explicit Likelihood

I was reading the Wikipedia page for flow-based generative models. On the page, it says "In contrast, many alternative generative modeling methods such as variational autoencoder (VAE) and ...
0 votes
0 answers
19 views

Time series: averaging the log-likelihood to select the best lookback window

Let me have an example toy model like a cointegration regression between two time series. I follow the Engle-Granger two-steps procedure and model the residuals with an $AR(1)$ process, so: $y_t = \...
  • 595
1 vote
0 answers
25 views

Binomial distribution using MLE [closed]

I have a few points in this binomial distribution given. Problem is that we find that out of $n=130$ people, $x=75$ of them play Minecraft. We also find that in a population of $m_1 = 25$ men, $x_1 = ...
1 vote
0 answers
30 views

Binomial vs Bernoulli Likelihoods: Difference assumptions of independence across observations

In Bayesian statistics, logistic regression can be facilitated by priors, a link function, and a likelihood choice of either the Bernoulli or Binomial distributions. My question is whether this design ...
  • 1,728
0 votes
0 answers
10 views

Is the log-likelihood a good metric to determine the appropriate number of gauss-hermite quadratures for GLMM?

Is the log-likelihood a good metric to determine the appropriate number of gauss-hermite quadrature for GLMM? I saw a case where the estimation does not converge for some values of quadratures, and ...
1 vote
0 answers
28 views

Likelihood function-expectation

Given likelihood is a function of parameters, I cannot understand why the expectation of likelihood functions is not calculated with respect to the the parameter space but the sample space, as put ...
  • 11
1 vote
0 answers
17 views

Bayesian inference when distribution depends on unobserved outcome with known distribution

Let's say we have an observed outcome $Y_i$ for an object $i=1,\ldots,I$ that arises like this: For each object a coin is tossed (outcome $X_i$ = $H$ or $T$). We know the coin is fair, so $X_i \sim \...
  • 24.2k
1 vote
0 answers
32 views

Asserting parentage likelihood based on SNP data

Let's say I have a population of N specimen that I've succesfully genotyped by M SNPs. If I know allele frequencies for every SNP, how do I compute likelihood of parental link (e.g. mother-daughter) ...
1 vote
1 answer
48 views

Log Likelihood for a Gaussian process regression model

According to Bishop, the author from "Statistical Pattern Recognition", we can optimize the hyperparameters of a Gaussian process by maximizing the likelihood function $$p(\textbf{t}|\theta),...
  • 73
0 votes
0 answers
19 views

derivative for $-K\log(Z(W,\mathbf{a},\mathbf{b}))+ \sum_k(\mathbf{s^k})^T\mathbf{W}\mathbf{d}^k+\mathbf{a}^T\mathbf{s}^k+\mathbf{b}^T \mathbf{d}^k$?

This is a follow up from this question. Consider a model of diseases and symptoms. $s_i\in\{0,1\}$ is a binary random variable indicating whether the patient is showing the $i$-th symptom and $d_j\in ...
  • 71
0 votes
0 answers
21 views

Likelihood of multiple spells as repeated measurement from Tutz & Schmid (2016)

I am reading Tutz & Schmid "Modeling Discrete Time-to-Event Data" (2016) chapter 10 Multiple Spell Analysis section 10.2 Multiple Spells as Repeated Measurement. On p. 216 the hazard for ...
0 votes
1 answer
56 views

Likelihood of a multiple-spell survival model from Tutz & Schmid (2016)

I am reading Tutz & Schmid "Modeling Discrete Time-to-Event Data" (2016) chapter 10 Multiple Spell Analysis section 10.1.1 Estimation. On p. 215 the closed form of the total likelihood ...
0 votes
0 answers
29 views

Are two likelihoods statistically different?

Here is an example that will help me ask my question. Let's say, I have the red and blue distributions as below, both of which are normal distributions (n = number of observations): Then I introduce ...
  • 1
0 votes
1 answer
37 views

Posterior distribution when the domain of the likelihood depends on the parameter

I am trying to calculate a posterior density given distribution and a prior. And I am a bit confused about how I should act as the domain of the distribution depends on the parameter. I am talking ...
0 votes
1 answer
33 views

Randomly choose between options with multiple criteria

Here's the problem: I have some options. Each is represented with three attributes or say criteria (with normalized values between 0 and 1). I want to randomly choose one of these options based on ...
  • 3
0 votes
0 answers
29 views

A problem about making an approximation to the integral over parameters -- eq (3.70) of Bishop's Pattern Recognition and Machine Learning

The problem comes from the paragraph containing equation (3.70) at the bottom of page 162 of Bishop's "Pattern Recognition and Machine Learning" which talks about an approximation to the ...
  • 67
1 vote
1 answer
36 views

Deriving the expression for $p(\mathcal{K})$ where $\mathcal{K} = \{(\mathbf{s}^k,\mathbf{d}^k), k = 1,..., K\}$

This is a follow up from this question. Consider a model of diseases and symptoms. $s_i\in\{0,1\}$ is a binary random variable indicating whether the patient is showing the $i$-th symptom and $d_j\in ...
  • 71
0 votes
0 answers
40 views

How to update a prior probability distribution of hurricane occurrence based on absence of hurricanes to date?

For a forecasting tournament, I am trying to forecast the number of Atlantic basin hurricanes in the 2022 hurricane season. I have reason to believe that my prior distribution looks as follows: At ...
2 votes
0 answers
42 views

Is it circular reasoning to compute the ELBO using MCMC?

Let's say we have a posterior distribution $q(\theta) = p(\theta \mid D, \mathcal{M})$ over parameters $\theta$ given data $D$ and a model $\mathcal{M}$. As is often the case, computing $q$ is hard, ...
  • 21
0 votes
0 answers
23 views

Obtaining log-likelihood value of a Markov chain when probability transition matrix contains exact-zero entries

I have a $n$ sequences $\boldsymbol{X}_1, \dots, \boldsymbol{X}_n$ of varying lengths arising from a Markov chain with a large state space $\mathcal{S} = \{ 1, 2, \dots, s \}$. Suppose the initial ...
1 vote
1 answer
28 views

Terminology question regarding a certain "partial maximum likelihood" which approximates the marginal likelihood

Suppose that we have a model with many parameters, which we'll partition into two subvectors called $\theta$ and $\lambda$. In this situation, $\lambda$ corresponds to those parameters that are really ...
2 votes
0 answers
39 views

parallelizing log-sum-exp

I have some approximate likelihoods: $L_1, \ldots, L_n$. Each is quite expensive to calculate. They're approximate because they use random numbers. Each of them is being calculated on the same data ...
  • 18.9k
0 votes
0 answers
11 views

Mixed Models: can restricted models have larger likelihood than complete model?

I have estimated a mixed model of the form $\underline{Y} = \mathbf{X}\underline{\alpha} + \mathbf{Z}\underline{\beta} + \underline{\varepsilon}$, which has a few interaction terms and individually ...
0 votes
0 answers
13 views

Reparametrizing a Uniform Prior Distribution to Multivariate Standard Normal

Problem Description I have a posterior distribution $$ p(\theta\mid y) \propto p(y \mid \theta) p(\theta) $$ with a uniform prior $p(\theta)= \mathcal{U}([a, b]^n)$, which is bounded. However, for my ...
  • 1,560
0 votes
0 answers
45 views

How to express a likelihood function for the following regression?

I have asked this question in entirely different forms a number of times on StackExchange, to no avail. Between each question, I investigated the literature thoroughly, but I have yet to find a ...
  • 1,072
0 votes
1 answer
36 views

Is Hessian of neural nets with NLL loss positive semi-definite?

I learned that expected Hessian of negative log likelihood is the same as Fisher information matrix, which is known to be positive semi-definite $$ \begin{aligned} F(\theta) &= E_{x \sim p_\theta}...
2 votes
2 answers
39 views

Find a likelihood to calculate a posterior probability

I am having trouble understanding a basic Bayesian inference exercise: Suppose we are interested in inferring the proportion $\theta$ of individuals in a given population suffering from a certain ...
  • 121
1 vote
0 answers
19 views

Bayesian inference, likelihood on positive data

Suppose I have a parameter $\theta$, that I know is positive, and some data $(x_1,x_2,\dots,x_n)$ on noisy realisations of the $\theta$. I then assume a prior with positive support on $\theta$ (...
  • 11
2 votes
0 answers
30 views

Can log-likelihood test be applied to test two models which are not nested but nested within a full model?

If we have a response variable y and three predictor variables x1, x2, and x3 and M1 and M2 are nested within M3 where ...
  • 131
0 votes
1 answer
20 views

Mixtures vs Multi-level models?

I'm confused on how mixture models and multi-level models are different (if at all.) Are there general rules for when to use one and not the other, pros/cons, etc?
  • 1,728
2 votes
1 answer
64 views

Compute the Likelihood of binomial data

Say we have to following data: p = 0.95 -> rate of true positive result of pcr test. q = 0.1 -> rate of false positive result of pcr test. s = 0.2 -> rate of total patients in the population ...
1 vote
0 answers
34 views

Multinomial likelihood function with data for only 2 of 3 outcomes

Can/should I use a binomial likelihood function if the data were generated from a multinomial process (3 possible outcomes) but data were only collected for two of the possible outcomes? In each trial ...
  • 11
2 votes
1 answer
91 views

Log-transformation in negative log-likelihood for negative binomial distribution

I am performing negative log-likelihood maximization for success probability parameter of the negative binomial distribution avoiding numerical errors. I am not 100% sure if this procedure is valid, ...
0 votes
0 answers
33 views

References for the conjugate prior to the beta distribution?

The Wikipedia article about "Conjugate Prior" has a table containing information about Likelihood Distributions with their Conjugate Priors. In the "Continuous Likelihood" table, ...
  • 1
0 votes
1 answer
33 views

Nested sampling: What does "uniform sampling over the prior" mean?

I'm reading up on Nested Sampling in the book "Data Analysis - A Bayesian Tutorial" (Sivia and Skilling, 2006), and I do not understand the following: What I understand: Given a prior $\pi(\...
0 votes
0 answers
15 views

Test for seasonality with LR-test?

I have an economic time series in monthly frequency. I want to test for seasonality using LR-Test. So the idea is to: Regress the time series y on a model with a time trend and 12 seasonal dummy ...
1 vote
2 answers
223 views

Understanding the Evidence Lower Bound (ELBO)

I am reading this tutorial about Variational Inference, which includes the following depiction of ELBO as the lower bound on log-likelihood on the third page. In the tutorial, $x_i$ is the observed ...
  • 1,487
3 votes
1 answer
70 views

The Likelihood Approach a.k.a. the 'third way' versus Bayesian

In his book "In All Likelihood" Yudi Pawitan writes that "the likelihood approach offers a distinct 'third way', a Bayesian-frequentist compromise. We might call it Fisherian as it owes ...

1
2 3 4 5
29