Skip to main content

Questions tagged [likelihood]

Given a random variable $X$ which arise from a parameterized distribution $F(X;θ)$, the likelihood is defined as proportional to the probability of observed data as a function of $θ$: $\operatorname{L}(θ | x)=\operatorname{P}(X=x \mid θ)$

Filter by
Sorted by
Tagged with
1 vote
0 answers
26 views

Gaussian linear model marginal likelihood under g-prior

Consider a Gaussian linear model with an $ n \times 1 $ outcome vector $ y $ and an $ n \times p $ matrix of centered predictors $ X $: $ y = \iota\alpha + X\beta + \varepsilon \quad \quad \varepsilon ...
yrx1702's user avatar
  • 710
0 votes
1 answer
20 views

Monte Carlo method for likelihoods ratio density estimation

I recently started reading Stephen Kay's Fundamentals of Statistical Signal Processing - Detection Theory (Volume II) and there is something I do not fully understand about likelihoods and hypothesis ...
gangrene's user avatar
  • 103
1 vote
1 answer
53 views

Birnbaum's Theorem: Strong belief in a model $\implies$ the likelihood function must be used as a data reduction device?

Working through understanding section 6.3.2 (pg. 292-294) in Casella and Berger's Statistical Inference (2nd-ed). The following definitions and principles are given: Definition (Experiment): An ...
Aaron Hendrickson's user avatar
0 votes
1 answer
25 views

Why my Rho-square on Multinomial Logit Model (McFadden) so small?

When I'm using MNL, and try to find my rho square, it's found out to be so small. It is $0.0139$. For a good fit model, the rho square has to be between $0.2$-$0.4$. Is there any reason why it's so ...
Fajri's user avatar
  • 1
3 votes
1 answer
219 views

Basic question about deriving MAP estimator

Say we have a random process $X(t, u)$ parametrized by $t$ and $u$ that generates data $x$. We also have a prior on $u$, $p(u)$. Am I correct in stating that the expression to find the maximum a ...
DangerousTim's user avatar
3 votes
2 answers
59 views

Confusion over Fisher-scoring algorithm

Given a probability model $f(X;\theta)$ and a set of i.i.d. observations $x_1,\ldots,x_n$ which we assume to be drawn from some true parameter $f(X; \theta_0)$, we can perform maximum-likelihood ...
shem's user avatar
  • 186
0 votes
0 answers
20 views

Correlation Coefficent is higher when likelihood of an event is lower, how does this occur?

I have different variables that I am interested in if they influence pass/fail rates. To see what variables I might use as a leading indicator, I've pulled different variables such as "tutoring&...
helloyellobird's user avatar
0 votes
2 answers
45 views

Negative log-likelihood, high BIC, high R-squared, low error, using a difference-in-differences (DiD) methodology [closed]

I am trying to see the impact of Brexit on UK imports. My dependent variable are EU exports to the rest of world. I have monthly data from 2013 to 2023, also data is in billions of GBP. When I do ...
rea123's user avatar
  • 1
0 votes
0 answers
26 views

How to obtain likelihood ($P(B/R)$ given the prior $P(R)$ and the posterior $P(R/B)$

I am working on a topic related to multiple-choice response. I would like to measure the efficiency of the information source (or a student’s information search) and I believe Bayesian statistics is ...
Francisco 's user avatar
1 vote
0 answers
26 views

Closed-Form Lambda for Yeo-Johnson-Transformed Normal-Inverse-Gaussian-Distributed Random Variables

I would like to know whether there exists a closed-form solution for the $\lambda$-parameter that maximizes the log-likelihood function of Yeo-Johnson transformed random variables that (before the ...
Hiro's user avatar
  • 425
2 votes
0 answers
39 views

Likelihood from posterior [closed]

This question is strange and perhaps silly but it would be very useful for my research. Is there any method to find the likelihood given a prior distribution and its corresponding posterior ...
Francisco 's user avatar
0 votes
0 answers
12 views

Convergence of a Bayesian classifier

Background Let $y_k$ be a noisy measurement at time $k$ and let $\{p_{k-1}(i)\}_{i=1}^n$ be (a discrete) prior probability distribution. Using Bayes rule, one can update the prior in function of $y_k$ ...
matteogost's user avatar
0 votes
0 answers
13 views

Comparing GLMs with different fitted distributions

I have a scenario where I need to compare some generalized liner models (with same link function, target variable, but not necessarily nested) with k fold cross validation, using a cost function to ...
user101874's user avatar
2 votes
0 answers
65 views

Can an outcome variable be used twice in the same model?

When is it appropriate to use the same outcome variable in two likelihoods in the same model framework? Here is a specific example: ...
Benny Borremans's user avatar
0 votes
0 answers
17 views

How can uncertainties in a generative model be propagated to an overall log-likelihood?

I am trying to use a Bayesian approach to carry out model selection and estimate the posterior distributions for parameters in a peak fitting scenario (quasi elastic neutron scattering). The ...
Andrew Nelson's user avatar
2 votes
1 answer
48 views

Confused between Multiple Random Variables and Likelihood Function [closed]

I am confused between the two at a very fundamental level. Following is the problem: I take observations $\vec{x}$ and create a histogram $\mathbf{n} = (n_1,\ldots,n_N)$ out of it with $N$ bins. ...
Sid's user avatar
  • 69
2 votes
1 answer
41 views

Why does the sufficient statistic for the bivariate normal not imply a sufficient statistic for the correlation under bivariate normality?

This question links to a document by Jon Wellner that defines the sufficient statistic for the multivariate normal (p. 7, Example 2.7). The result follows from the factorization theorem and is proven ...
virtuolie's user avatar
  • 642
5 votes
1 answer
182 views

Rasmussen Equation 5.9

Can any one add the steps showing how Rasmussen (Gaussian Processes for Machine Learning, the MIT Press, 2006) got from line 1 to line 2 of equation 5.9. (pg 114)? It is calculating the gradient of ...
Snowy Baboon's user avatar
0 votes
0 answers
13 views

Specific Question about Deriving the Fisher Information for a Complex Multivariate Normal Distribution [duplicate]

I am starting with the following form for the likelihood function for a complex multivariate normal distribution for data with dimension $d$ and mean $\boldsymbol \mu$: $$ p(\mathbf x|\boldsymbol \...
StackMonkey's user avatar
0 votes
0 answers
83 views

What is the likelihood of a regression? [duplicate]

I understand linear regressions themselves have likelihoods. Is this simply the likelihood of the error? I thought it was the likelihood of the data for Y given X. In other words, $Lik(Y$~$X)=Lik(Y|...
A Friendly Fish's user avatar
0 votes
0 answers
21 views

Is the behavior of log-likelihood and number of parameters correct in probabilistic PCA?

I am studying the behavior of Probabilistic PCA as described by Tipping and Bishop (1999). I am using the R package "Rdimtools" to help. I am puzzled about the number of parameters in the ...
Daniel Caetano's user avatar
3 votes
1 answer
36 views

describing binomial data in likelihood models

Binomial data can be described in various ways. Suppose we flip a fair coin twice and get one head (success). One method to calculate its negative log-likelihood is ...
quibble's user avatar
  • 1,694
0 votes
0 answers
24 views

What is the Likelihood Formula for an Error in Variable Model?

I am comparing different models ability to explain my joint observation of (X,Y) with AIC for which I need the likelihood. How can I calculate the likelihood of (X,Y) for the error in variable model ...
A Friendly Fish's user avatar
1 vote
0 answers
37 views

Likelihood of logistic GLMs using R [duplicate]

I am currently working on understanding how GLMs work in R, and more precisely logistic regressions. The documentation of the glm() function states: A typical ...
FloLecorvaisier's user avatar
0 votes
1 answer
41 views

Likelihood function of paired conditional logistic regression

I found the information about the likelihood of conditional logistic regression for paired data is few. The Wiki gives this answer, but I think it is wrong because event Yi1 and Yi2 are dependent, ...
Tom Hsiung's user avatar
0 votes
0 answers
40 views

Likelihood function for data with random censoring

The following is from Klein and Moeschberger, p. 76. Let $(T,\delta)$ be a tuple with $T = \min(X,C_r)$ and $\delta = 0$ if the lifetime X is censored and $\delta = 1$ if it is not; $C_r$ denotes the ...
Montresor's user avatar
  • 101
3 votes
1 answer
78 views

Random effects Models vs Gaussian Log likelihood + explicit grouping features

Let us assume the Gaussian negative log likelihood (like e.g. here https://pytorch.org/docs/stable/generated/torch.nn.GaussianNLLLoss.html) $\text{Gaussian Negative Log Likelihood} = -\frac{1}{n} \...
Ggjj11's user avatar
  • 1,543
1 vote
1 answer
32 views

Derivative of structure matrices

I'm trying to follow 'Advanced Multivariate Statistics with Matrices', chapter 1.4. I know that this book is quite old, however I'm rather constrained with time and the papers I'm reading references ...
AyamGorengPedes's user avatar
0 votes
0 answers
26 views

How to make the profile likelihood model for estimation?

I tried to make the age estimation model using the chemical compound results from The soil. Initially, I used the multivariable regression model. However, the reviewer highly recommend using the ...
user21268575's user avatar
2 votes
0 answers
30 views

To what extent can likelihood methods be used for functional responses?

Let's suppose that we are working with a functional data set, $Y_i(t)$, $Y_i\in L^2[0,1]$, $1\le i\le n$. If we were working with univariate or even multivariate data set, likelihood methods would ...
cgmil's user avatar
  • 1,373
6 votes
3 answers
135 views

Is Pitman-Koopman-Darmois Theorem valid for discrete random variables?

I am interested in the Pitman-Koopman-Darmois theorem. I'm having a hard time finding a simple rigorous version of this theorem as I struggle finding sources. This helpful post provides three sources ...
Pohoua's user avatar
  • 2,628
0 votes
0 answers
13 views

Turning a list of cost into categorical probability mass distribution

Background Given a noisy dataset $D$, I have to solve a classification problem where the possible anserwer is $i\in\{1,\dots,N\}$. So far I can get pretty decent result with an algorithm that, based ...
matteogost's user avatar
1 vote
2 answers
81 views

Related to consistency of MLE: why $E_{\theta_0}\log\frac{f(X,\theta)}{f(X,\theta_0)}<0$?

Assumptions. $\theta_1\ne\theta_2\Rightarrow F_{\theta_1}\ne F_{\theta_2}$ The set $(x:f(x,\theta)>0)$ does not depend on $\theta$ For a.e. $x$, $f(x,\theta)$ is a differentiable function of $\...
reyna's user avatar
  • 385
0 votes
0 answers
31 views

Negative log likelihood cholesky decomposition

The code in question comes from Spatial Data Science by Pebesma and Bivand. I can pretty much understand what it says except for the last line. How does this give the negative log likelihood? I've ...
R Walser's user avatar
  • 101
-1 votes
1 answer
48 views

Maximum-LIkelihood Estimation with NLL using parameters in Logscale

Consider the following toy problem: I have a $\mathcal{C}^\infty$ function $f(t,\Theta):[0,t_{\max}] \times (0, \infty)^n \to \mathbb{R}$. I choose some "correct/ground-truth" parameter ...
Paul Joh's user avatar
  • 101
0 votes
0 answers
12 views

Interpretation question about the article "Finite mixture modeling of censored data using the multivariate Student-t distribution"

I am reading this article and I am struggling to understand the following passage from page 6: \begin{align*} L_{i}(\boldsymbol{\theta}|\textbf{V}_{i},\textbf{C}_{i}) = f(\textbf{V}_{i}|\textbf{C}_{i},...
user1234's user avatar
  • 245
5 votes
3 answers
165 views

Likelihood principle and inference

I've been reading Casella and Berger's Statistical Inference. In section 6.3 the author stated the likelihood principle: if the likelihood functions from two samples are proportional, then the ...
INvisibLE's user avatar
0 votes
1 answer
24 views

Trading card game poss

A trading card game that is called Flesh and Blood (description and rules here) has two players construct a 60 card deck and the hand limit is 4. Player 2's Deck color combination: 60 cards in deck; ...
Patrick's user avatar
0 votes
0 answers
43 views

Understanding the reasoning behind Tobit composite likelihood

The Tobit model defines a likelihood for what amounts to a censored normal distribution - i.e. the tails are clipped - which is meant to be applied to data that has undergone censoring and thus has an ...
anymous.asker's user avatar
11 votes
1 answer
284 views

Reconciling optimisation for log-likelihood and Brier score

Both log-likelihood and Brier score are proper scoring rules. As such, they reach the optimum when the predicted probabilities match the true ones. Since there is only one true probability for each ...
Igor F.'s user avatar
  • 9,418
0 votes
1 answer
77 views

How to derive marginal likelihood from prior and likelihood

Given the following: X|θ ~ N(θ , 1/k) model M0 stating that θ = θ0 model M1 stating that θ ~ N(u , 1/z) How to show that X|M1 follows a normal distribution with mean u and variance 1/k + 1/z? I ...
user avatar
0 votes
0 answers
40 views

Estimate the likelihood of two continuous samples of unknown distribution

Consider two continuous and unknown distributions $$X : {x_1, x_2, ..., x_n}$$ and $$Y : {y_1, y_2, ..., y_n}$$ both can be tagged as time series with $n > 8000$. I need to estimate the likelihood ...
joueswant's user avatar
1 vote
1 answer
28 views

Parameterization of Negative Binomial for Dynamical System Model Calibration/Fitting

I'm studying about the applications of bayesian inference to fitting dynamical systems to observations. So the model itself is a deterministic SIR model: $$ f(R_0,D_{inf})=\begin{cases} \frac{dS}{t}&...
Derf's user avatar
  • 295
30 votes
5 answers
33k views

Why is everything based on likelihoods even though likelihoods are so small?

Suppose I generate some random numbers from a specific normal distribution in R: set.seed(123) random_numbers <- rnorm(50, mean = 5, sd = 5) These numbers look ...
Uk rain troll's user avatar
0 votes
0 answers
19 views

Correct way to compute denominator in the bayes theorem

Say I have two data points and two Gaussians that these might be coming from. What is my evidence in this case? $P(X) = \prod_{i=1}^{2} (p(xi|\mu1)p(\mu1) + p(xi|\mu2)p(\mu2)) $ Because my samples ...
figs_and_nuts's user avatar
1 vote
1 answer
29 views

Interpreting AIC relative likelihoods ( qpcR::akaike.weights() )

I want to ensure that I am correctly interpreting AIC relative likelihood (RL) scores, specifically those returned by qpcR::akaike.weights$rel.LL. For example, I ...
PhelsumaFL's user avatar
1 vote
0 answers
30 views

Likelihood and Probability [duplicate]

So to my understanding, Probabilities are areas under fixed distribution, which can be expressed mathematically probability of data given distribution: $$P(Data|Distribution)$$ Whereas likelihoods ...
Rust32627's user avatar
1 vote
0 answers
44 views

Connection between Z-Score and Quasi Likelihood?

Consider the Quasi-Likelihood function (I wrote it in the GEE context): $$ Q(\beta; y) = \sum_{i=1}^{n} (y_i - \mu_i(\beta))^T V_i^{-1}(\beta, \alpha) (y_i - \mu_i(\beta))$$ where: $y_i$ is the ...
Uk rain troll's user avatar
0 votes
0 answers
6 views

Estimating a time series' likelihood of a missing observation

I have this time series of seasonal monthly data, which sometimes has missing observations. The likelihood of an observation being missing is mostly dependent on the month, but also of the value had ...
BlackNinja's user avatar
0 votes
0 answers
24 views

Fitting GLMM using lme4 package, fitting algorithm

I'm a little confused here. So I'm using glmm model to fit user/item interaction. If user #12 liked movie #15 it's 1 otherwise it's 0. Here is my model:  ...
Efecan Bahcivanoglu's user avatar

1
2 3 4 5
32