Questions tagged [likelihood]
Given a random variable $X$ which arise from a parameterized distribution $F(X;θ)$, the likelihood is defined as the probability of observed data as a function of $θ$: $\operatorname{L}(θ | x)=\operatorname{P}(X=x \mid θ)$
1,526
questions
2
votes
0
answers
23
views
Analogue of landscape conjecture in likelihood theory or Bayes?
The so-called landscape conjecture in machine learning says that in high dimensions, most critical points of the loss surface are saddle points rather than poor local minima.
Out of curiosity I was ...
- 951
0
votes
0
answers
31
views
Maximum likelihood in linear regression
My understanding is that when we do maximum likelihood we want to choose parameters $\theta$ such that the probability of observing the actual, fixed data is maximized. That's how I understood it ...
2
votes
3
answers
132
views
What exactly is likelihood? [duplicate]
My understanding about likelihood, given some reading, is that it is how likely we are to observe the actual data given a certain parameter or parameter values $\theta$. Like with the coin toss ...
2
votes
1
answer
60
views
pdf vs probability vs likelihood [duplicate]
How to compute the log likelihood?
Let's take a simple example using a normal distribution and scipy to do the work. Assuming X is the data, and the normal distribution as the model (...
- 123
4
votes
4
answers
180
views
Interpretation of Maximum Likelihood Value
I have a question about Maximum Likelihood values, and how to interpret them.
In order to explain the question, please see the Figure below. I will add explanation for how this figure has been created ...
- 266
2
votes
0
answers
50
views
Unbiased estimate of log-likelihood of Markov bridge
Note: I have cross-posted this question to MathSE.
I have the following problem I am trying to solve. I have a parametric family of "transition" distributions $p_\theta(x_{i+1}\mid x_i)$ and ...
1
vote
0
answers
39
views
Likelihood ratio as minimal sufficient statistics in infinite parameter space
I just read a question from here (Likelihood ratio minimal sufficient) and have some thoughts. Let me restate the question first:
Consider a family of density functions $f(x|\theta)$ where the ...
- 31
1
vote
0
answers
20
views
Likelihood of sum of log-normal and normal distribution [closed]
Given $y(x_t)=e^{f(x_t)}-\varepsilon _t$ with $\varepsilon _t\sim N(0,4e^{f(x_t)})$ and $f\sim GP(\mu,\sum)$. What is the likelihood $p(y|f)$? Is it $p(y|f)\sim N(e^{f(x_t)},4e^{f(x_t)})$?
Thanks a ...
- 11
0
votes
1
answer
26
views
What is the correct way to refer to the Likelihood $L(\theta, \alpha | X)$ where $\theta$ and $\alpha$ are parameters?
I am currently studying some latent variable models.
In many works, I found the following equation:
$L( \theta, \boldsymbol\pi | x ) = \sum_{c=1}^{C} f(x| \theta, \alpha = \alpha_c) \cdot \pi_c$
where ...
- 173
4
votes
1
answer
373
views
Where does Quasi-Likelihood formula come from?
In regular likelihood/log likelihood, if there is random variable "$Y$" with pdf (probability distribution functions) $f_Y(y)$... the likelihood of this can be written as: $\mathcal{L}(y_i) =...
1
vote
1
answer
46
views
Are these two equivalent forms for the likelihood of a Poisson point process?
I have a Poisson point process in a bounded region $W$. I'm trying to calculate the likelihood of observing a particular set of points within $W$. I'm told that there are two equivalent forms of ...
- 1,446
3
votes
1
answer
140
views
Is likelihood the y axis coordinate on the distribution curve?
Josh Starmer says it in here.
I have been searching for a simple way to understand likelihood and it's Bayesian and Frequentist use.
Josh's way seems simple to me.
Is he correct?
- 703
1
vote
0
answers
51
views
Python statsmodels GLM - log likelihood of null model
I have an issue when calculating log-likelihood for null model to double-check GLMResults.llnull parameter:
https://www.statsmodels.org/devel/generated/statsmodels.genmod.generalized_linear_model....
- 111
1
vote
1
answer
23
views
Why does the score test work for values longer in the tail that have a small log-likelihood derivative?
The score test says that we take the derivative of the log-likelihood at $H_0$ and divide it by the fisher information at $H_0$.
$U(\theta )={\frac {\partial \log L(\theta \mid x)}{\partial \theta }}.$...
0
votes
0
answers
65
views
MLE of weibull distribution with survival data
I would like to ask about estimating parameters of Weibull distribution (a, and b)
I am trying to code likelihood of weibull distribution with survival data $(T_i, \Delta_i),$ which I believe is: $(ab)...
- 1
0
votes
0
answers
40
views
Should we increase the number of samples when adding more classes?
Assume we are solving a $k$-class classification problem, $k \geq 2$, and we have a trained classifier $\phi$ from a family of generative or discriminative classifiers $\Phi$
minimizing an objective $\...
- 294
1
vote
0
answers
33
views
Which parameters optimise the weighted cross-entropy loss for a pre-specified categorical distribution?
Question:
Given a categorical distribution $C_q$ with parameters $q_1, \ldots, q_K$ with $K > 2$, $\sum_k q_k = 1$, which (new) categorical distribution $C_p$ with parameters $p_1, \ldots, p_K$ ...
- 51
1
vote
1
answer
51
views
Likelihood determination for a step-like pdf
Suppose that random numbers x are generated on the computer using the following procedure:
Generate two numbers $x_1$, $x_2$ from a uniform distribution $\mathcal{U}$([0,1])
If $x_1$ > f, take x =...
- 11
0
votes
1
answer
103
views
Why does higher dimensional data has higher likelihood?
I am reading about generative models. I came across an example a few times but I cannot come up with an explanation for it.
Imagine data is generated according to $p_\text{data}(x)$. It is often said ...
- 77
0
votes
0
answers
18
views
Predictive Diagnostic, Comparison of simulated data with observed data
The question is quite abstract, so I display it with only the essential information.
Suppose that we have three models $B_{1}, B_{2}$ and $F_{3}$. The $B_{1}, B_{2}$ are Bayesian models and the $F_{3}$...
- 2,152
2
votes
1
answer
28
views
Relation between sample standard deviation from data and maximum likelihood estimates
This is my data:-
c(3164, 3362, 4435, 3542, 3578, 4529)
I estimated its sample mean and standard deviation via mean & ...
4
votes
1
answer
71
views
Overlapping circular bearing distributions on a plane
I have some directional hydrophones capable of recognizing transient signals/sound and estimating the circular probability density function of the bearing, or direction, that the sound came from. I ...
- 43
0
votes
0
answers
25
views
How to calculate the likelihood for a normal distribution N(theta, 1) if we only know the maximum of a sample?
Assuming iid samples x ~ N(theta, 1), we have a sample of 5 observations with maximum value = 3. How to calculate the likelihood?
- 1
0
votes
0
answers
53
views
comparing 2 likelihood values
Are likelihood values (density values) comparable across different types of distributions?
For example, if you have a data point that has a likelihood value of .05 under a normal distribution and .025 ...
- 1
2
votes
1
answer
104
views
What are the undefined constants and functions in Stern's 2011 paper?
I'm reading the 2011 paper on ranking called Moderated Paired Comparisons by Steven E. Stern and there are no definitions given for some of the constants and functions in equation 1.
As you can see, ...
- 33
0
votes
0
answers
31
views
BIC to test good fitting of data to a model
I want to use the Bayesian Information Criterion in order to measure how well a gaussian and 0 order polynomial fit (using python), the one with the lowest BIC should then be the 'best fit' ?
My ...
- 1
5
votes
1
answer
252
views
Likelihood ratio test vs p value for Poisson regression
I have a Poisson regression model, from its summary table, I could see the p-value for a certain variable, e.g. gender. Since the p-value is testing the hypothesis whether the coefficient of gender ...
- 353
0
votes
0
answers
24
views
Probability of next flip being heads given I have seen h heads and t tails
I am currently attempting to understand "Question 2" at this link but having many difficulties. The problem is as follows:
A coin has a chance of landing heads with an unknown probability ...
- 173
2
votes
0
answers
24
views
Likelihood function of VAR-MGARCH-BEKK model?
I am doing my dissertation on the spillover effect between countries' markets and looking to use VAR-MGARCH model to do it. For example how would a change/shock of US market index affect Thailand ...
- 21
1
vote
0
answers
14
views
Conditional likelihood with missing values
I want to estimate a logistic regression model on a panel data (subject-time) with subject-fixed effects.
$$\log(p_{it}/(1-p_{it})) = \alpha_{i} + \beta x_{it} + \epsilon_{it}.$$
To do so, I want to ...
- 11
2
votes
1
answer
52
views
Trying to understand log-likelihood estimation for exponential smoothing models in R forecast function ets()
I am doing work on AIC comparisons. For this purpose, I am trying to understand how log-likelihood is calculated for exponential smoothing models (ETS models) in different R packages.
In particular, <...
1
vote
1
answer
23
views
Binomial vs product of binomial in likelihood for Bayesian inference
I am working through McElreath's book on statistical rethinking. One of the problems is the following:
Using grid approximation, compute the posterior distribution for the probability of a birth being ...
- 461
1
vote
1
answer
47
views
MLE for parametric binomial model
I have a model in which $p_i=f(\theta,Z_i)$, where $Z_i$ are iid latent variables distributed with CDF $F_\theta$, and $d_i\sim B(n_i,p_i)$, where $B$ is the binomial distribution. The likelihood ...
- 191
1
vote
0
answers
39
views
Why "likelihood is proportional to the probability of the data given the hypothesis" [duplicate]
In Etz, there is "likelihood is proportional to the probability of the data given the hypothesis" and "L(H) = K × P(D|H)", are there more detailed explanations to understand it? ...
- 111
0
votes
0
answers
23
views
Log likelihood decreases with posterior obtained after fitting GP
Possibly related to Log posterior probability in MCMC is decreasing but I do not have a MCMC process and the details there are not sufficient for me to understand fully (I'm a mathematician with basic ...
- 101
2
votes
0
answers
46
views
Implementing a 2-PL Dichotomous IRT Module in Python from scratch
I am trying to implement a 2-PL dichotomous IRT Model for my dataset from scratch in Python. Here is my code so far:
...
- 121
1
vote
0
answers
32
views
Likelihood function is a product of PDFs [duplicate]
I am learning about the likelihood function given iid random variables $X_i$ and realizations $x_i$: $\mathcal{L}(\theta | x) = \prod_{i=1}^n \mathbb{P}(X_i = x_i)$. One thing I am confused about is ...
- 173
0
votes
0
answers
20
views
Likelihood in a Bayesian interference problem
I'm currently reading some lecture notes in the field of statistical physics for optimization problems.
In there we are given a $N \times N$ symmetric matrix $Y$ as follows
$$Y = \sqrt{\frac{\lambda}{...
0
votes
0
answers
30
views
Computing odds ratios for multiple dichotomous DVs in a within-person, mixed-effects design?
I have an experiment where people participate in a series of tasks (say 4) and then are scored based on their performance (pass/fail). The order of the tasks is randomized. I want to predict ...
2
votes
0
answers
152
views
Zero-inflated Poisson - Implementing INLA with two likelihoods
I am trying to implement a zero inflated model in INLA.
I know a basic zero inflated Poisson can be implemented with "zeroinflatedpoisson1" as the family ...
- 121
0
votes
0
answers
12
views
How to calculate the increased likelihood of drawing a type of item from a supply of mixed item types after adding more of a specific items type?
There is a board game, The Castles of Burgundy, which has a supply of 40 "black market" tiles, of which 16 are beige, 8 green, 2 gray, 6 blue, 6 yellow, and 2 burgundy.
The game takes place ...
- 101
0
votes
0
answers
81
views
Sampling from a gamma distribution and computing its likelihood
I would like to conduct a model comparison analysis of a process that is modelled with a gamma distribution. To illustrate, let's consider the example of sampling the time of incidents in a factory. ...
- 101
1
vote
1
answer
93
views
What is "cohort likelihood" and where is it from?
this is my very first post and I would consider myself a beginner in statistics, but I couldn't find anything about this in other asks.
I am learning about the Self-controlled case series (SCCS) model ...
- 13
1
vote
0
answers
20
views
Minimizing the NLL of a t-distribution derived from a NIG prior
My question concerns this paper which is a little too succinct for me to understand. The context is the following. Suppose $y$ is Normal distributed, with a Normal-Inverse-Gamma prior,
$$
y \sim N(\mu,...
- 831
1
vote
1
answer
41
views
Conditional Maximum Likelihood Estimation with Subsample
Suppose that we have an $i.i.d$ sample, $\{Y_i,X_i\}_{i=1}^N$, and a correctly specified conditional density of $Y$ given $X$, $f(Y|X; \theta)$, where $\theta$ is the parameters of the density.
Then, ...
- 191
2
votes
1
answer
59
views
Log-likelihood skew-t
I am trying to write down the log-likelihood for the multivariate skewed student-t distribution, but I don't really get how to define it exactly. Could someone please tell me the definition as in how ...
- 21
2
votes
1
answer
48
views
If Likelihood is not a PDF then why is the PDF of Multivariate Normal equivalent to the likelihood of I.I.D. Normals?
I am understanding why likelihoods are not PDFs using links such as What is the reason that a likelihood function is not a pdf. However I am getting more confused.
For instance, the likelihood of I.I....
- 45
0
votes
0
answers
44
views
Why is it that we can talk about the probability of the data given the parameter in bayesian inference though the data is considered to be fixed?
Basically, the title of the question is all there is.
quoting from bishop's pattern recognition and machine learning:
In both the Bayesian and frequentist paradigms, the likelihood functions $p(D/w)$ ...
- 2,385
0
votes
0
answers
46
views
How to write the likelihood for a multivariate gaussian linear model
I have a lasso-like bayesian graphical model where we try to estimate precision matrices between two conditions (0 and 1), $\Sigma_0^{-1}$ and $\Sigma_1^{-1}$, respectively. The model can be ...
- 523
0
votes
0
answers
78
views
Confusion with the "lower bound"-term in diffusion models
I am trying to understand the maths of diffusion models following this video explanation on youtube and this blog post.
Here is what how I understood it so far:
The overall goal is, that we want to ...
- 1