Questions tagged [likelihood]
Given a random variable $X$ which arise from a parameterized distribution $F(X;θ)$, the likelihood is defined as the probability of observed data as a function of $θ$: $\operatorname{L}(θ | x)=\operatorname{P}(X=x \mid θ)$
1,523
questions
1
vote
1
answer
21
views
Why does the score test work for values longer in the tail that have a small log-likelihood derivative?
The score test says that we take the derivative of the log-likelihood at $H_0$ and divide it by the fisher information at $H_0$.
$U(\theta )={\frac {\partial \log L(\theta \mid x)}{\partial \theta }}.$...
0
votes
0
answers
28
views
MLE of weibull distribution with survival data
I would like to ask about estimating parameters of Weibull distribution (a, and b)
I am trying to code likelihood of weibull distribution with survival data $(T_i, \Delta_i),$ which I believe is: $(ab)...
- 1
0
votes
0
answers
31
views
Should we increase the number of samples when adding more classes?
Assume we are solving a $k$-class classification problem, $k \geq 2$, and we have a trained classifier $\phi$ from a family of generative or discriminative classifiers $\Phi$
minimizing an objective $\...
- 294
1
vote
0
answers
30
views
Which parameters optimise the weighted cross-entropy loss for a pre-specified categorical distribution?
Question:
Given a categorical distribution $C_q$ with parameters $q_1, \ldots, q_K$ with $K > 2$, $\sum_k q_k = 1$, which (new) categorical distribution $C_p$ with parameters $p_1, \ldots, p_K$ ...
- 51
1
vote
1
answer
50
views
Likelihood determination for a step-like pdf
Suppose that random numbers x are generated on the computer using the following procedure:
Generate two numbers $x_1$, $x_2$ from a uniform distribution $\mathcal{U}$([0,1])
If $x_1$ > f, take x =...
- 11
0
votes
1
answer
79
views
Why does higher dimensional data has higher likelihood?
I am reading about generative models. I came across an example a few times but I cannot come up with an explanation for it.
Imagine data is generated according to $p_\text{data}(x)$. It is often said ...
- 77
0
votes
0
answers
18
views
Predictive Diagnostic, Comparison of simulated data with observed data
The question is quite abstract, so I display it with only the essential information.
Suppose that we have three models $B_{1}, B_{2}$ and $F_{3}$. The $B_{1}, B_{2}$ are Bayesian models and the $F_{3}$...
- 2,142
2
votes
1
answer
26
views
Relation between sample standard deviation from data and maximum likelihood estimates
This is my data:-
c(3164, 3362, 4435, 3542, 3578, 4529)
I estimated its sample mean and standard deviation via mean & ...
3
votes
1
answer
53
views
Overlapping circular bearing distributions on a plane
I have some directional hydrophones capable of recognizing transient signals/sound and estimating the circular probability density function of the bearing, or direction, that the sound came from. I ...
- 33
0
votes
0
answers
21
views
How to calculate the likelihood for a normal distribution N(theta, 1) if we only know the maximum of a sample?
Assuming iid samples x ~ N(theta, 1), we have a sample of 5 observations with maximum value = 3. How to calculate the likelihood?
- 1
0
votes
0
answers
52
views
comparing 2 likelihood values
Are likelihood values (density values) comparable across different types of distributions?
For example, if you have a data point that has a likelihood value of .05 under a normal distribution and .025 ...
- 1
2
votes
1
answer
104
views
What are the undefined constants and functions in Stern's 2011 paper?
I'm reading the 2011 paper on ranking called Moderated Paired Comparisons by Steven E. Stern and there are no definitions given for some of the constants and functions in equation 1.
As you can see, ...
- 33
0
votes
0
answers
22
views
BIC to test good fitting of data to a model
I want to use the Bayesian Information Criterion in order to measure how well a gaussian and 0 order polynomial fit (using python), the one with the lowest BIC should then be the 'best fit' ?
My ...
- 1
5
votes
1
answer
149
views
Likelihood ratio test vs p value for Poisson regression
I have a Poisson regression model, from its summary table, I could see the p-value for a certain variable, e.g. gender. Since the p-value is testing the hypothesis whether the coefficient of gender ...
- 353
0
votes
0
answers
24
views
Probability of next flip being heads given I have seen h heads and t tails
I am currently attempting to understand "Question 2" at this link but having many difficulties. The problem is as follows:
A coin has a chance of landing heads with an unknown probability ...
- 173
2
votes
0
answers
18
views
Likelihood function of VAR-MGARCH-BEKK model?
I am doing my dissertation on the spillover effect between countries' markets and looking to use VAR-MGARCH model to do it. For example how would a change/shock of US market index affect Thailand ...
- 21
1
vote
0
answers
14
views
Conditional likelihood with missing values
I want to estimate a logistic regression model on a panel data (subject-time) with subject-fixed effects.
$$\log(p_{it}/(1-p_{it})) = \alpha_{i} + \beta x_{it} + \epsilon_{it}.$$
To do so, I want to ...
- 11
2
votes
1
answer
38
views
Trying to understand log-likelihood estimation for exponential smoothing models in R forecast function ets()
I am doing work on AIC comparisons. For this purpose, I am trying to understand how log-likelihood is calculated for exponential smoothing models (ETS models) in different R packages.
In particular, <...
1
vote
1
answer
22
views
Binomial vs product of binomial in likelihood for Bayesian inference
I am working through McElreath's book on statistical rethinking. One of the problems is the following:
Using grid approximation, compute the posterior distribution for the probability of a birth being ...
- 461
1
vote
1
answer
45
views
MLE for parametric binomial model
I have a model in which $p_i=f(\theta,Z_i)$, where $Z_i$ are iid latent variables distributed with CDF $F_\theta$, and $d_i\sim B(n_i,p_i)$, where $B$ is the binomial distribution. The likelihood ...
- 171
1
vote
0
answers
38
views
Why "likelihood is proportional to the probability of the data given the hypothesis" [duplicate]
In Etz, there is "likelihood is proportional to the probability of the data given the hypothesis" and "L(H) = K × P(D|H)", are there more detailed explanations to understand it? ...
- 111
0
votes
0
answers
20
views
Log likelihood decreases with posterior obtained after fitting GP
Possibly related to Log posterior probability in MCMC is decreasing but I do not have a MCMC process and the details there are not sufficient for me to understand fully (I'm a mathematician with basic ...
- 101
2
votes
0
answers
27
views
Implementing a 2-PL Dichotomous IRT Module in Python from scratch
I am trying to implement a 2-PL dichotomous IRT Model for my dataset from scratch in Python. Here is my code so far:
...
- 121
1
vote
0
answers
32
views
Likelihood function is a product of PDFs [duplicate]
I am learning about the likelihood function given iid random variables $X_i$ and realizations $x_i$: $\mathcal{L}(\theta | x) = \prod_{i=1}^n \mathbb{P}(X_i = x_i)$. One thing I am confused about is ...
- 173
0
votes
0
answers
20
views
Likelihood in a Bayesian interference problem
I'm currently reading some lecture notes in the field of statistical physics for optimization problems.
In there we are given a $N \times N$ symmetric matrix $Y$ as follows
$$Y = \sqrt{\frac{\lambda}{...
0
votes
0
answers
29
views
Computing odds ratios for multiple dichotomous DVs in a within-person, mixed-effects design?
I have an experiment where people participate in a series of tasks (say 4) and then are scored based on their performance (pass/fail). The order of the tasks is randomized. I want to predict ...
2
votes
0
answers
93
views
Zero-inflated Poisson - Implementing INLA with two likelihoods
I am trying to implement a zero inflated model in INLA.
I know a basic zero inflated Poisson can be implemented with "zeroinflatedpoisson1" as the family ...
- 121
0
votes
0
answers
12
views
How to calculate the increased likelihood of drawing a type of item from a supply of mixed item types after adding more of a specific items type?
There is a board game, The Castles of Burgundy, which has a supply of 40 "black market" tiles, of which 16 are beige, 8 green, 2 gray, 6 blue, 6 yellow, and 2 burgundy.
The game takes place ...
- 101
0
votes
0
answers
46
views
Sampling from a gamma distribution and computing its likelihood
I would like to conduct a model comparison analysis of a process that is modelled with a gamma distribution. To illustrate, let's consider the example of sampling the time of incidents in a factory. ...
- 101
1
vote
1
answer
81
views
What is "cohort likelihood" and where is it from?
this is my very first post and I would consider myself a beginner in statistics, but I couldn't find anything about this in other asks.
I am learning about the Self-controlled case series (SCCS) model ...
- 13
1
vote
0
answers
18
views
Minimizing the NLL of a t-distribution derived from a NIG prior
My question concerns this paper which is a little too succinct for me to understand. The context is the following. Suppose $y$ is Normal distributed, with a Normal-Inverse-Gamma prior,
$$
y \sim N(\mu,...
- 821
1
vote
1
answer
37
views
Conditional Maximum Likelihood Estimation with Subsample
Suppose that we have an $i.i.d$ sample, $\{Y_i,X_i\}_{i=1}^N$, and a correctly specified conditional density of $Y$ given $X$, $f(Y|X; \theta)$, where $\theta$ is the parameters of the density.
Then, ...
- 191
2
votes
1
answer
47
views
Log-likelihood skew-t
I am trying to write down the log-likelihood for the multivariate skewed student-t distribution, but I don't really get how to define it exactly. Could someone please tell me the definition as in how ...
- 21
2
votes
1
answer
37
views
If Likelihood is not a PDF then why is the PDF of Multivariate Normal equivalent to the likelihood of I.I.D. Normals?
I am understanding why likelihoods are not PDFs using links such as What is the reason that a likelihood function is not a pdf. However I am getting more confused.
For instance, the likelihood of I.I....
- 45
0
votes
0
answers
40
views
Why is it that we can talk about the probability of the data given the parameter in bayesian inference though the data is considered to be fixed?
Basically, the title of the question is all there is.
quoting from bishop's pattern recognition and machine learning:
In both the Bayesian and frequentist paradigms, the likelihood functions $p(D/w)$ ...
- 2,263
0
votes
0
answers
44
views
How to write the likelihood for a multivariate gaussian linear model
I have a lasso-like bayesian graphical model where we try to estimate precision matrices between two conditions (0 and 1), $\Sigma_0^{-1}$ and $\Sigma_1^{-1}$, respectively. The model can be ...
- 513
0
votes
0
answers
49
views
Confusion with the "lower bound"-term in diffusion models
I am trying to understand the maths of diffusion models following this video explanation on youtube and this blog post.
Here is what how I understood it so far:
The overall goal is, that we want to ...
- 1
0
votes
0
answers
13
views
Odds Ratio - how many times more likely?
Problem: I need to figure out how many times are women more likely to continue attending a class after a certain period of time.
Data:
Solution: I used odds ratio (see picture).
Is it correct to ...
- 1
0
votes
1
answer
40
views
Using a Normal Distribution to motivate the use of a Parameter Penalty in Negative Log-Likelihood
This is a question arose from an argument I had about a statistical explanation.
We are comparing two models using their negative log-likelihoods. Given that we are unable to split the data into a ...
user389839
0
votes
0
answers
70
views
Likelihood and log-likelihood for Weibull distribution
Question
Suppose there are two groups on treatments and an individual follows a Weibull distribution with the following probability density function.
\begin{align}
f(x; \alpha, \lambda) = \alpha \...
- 99
1
vote
1
answer
116
views
Calculate log-likelihood of logistic regression
I am trying, without success, to calculate the log-likelihood of the most basic logistic regression model - a constant probability model (i.e. only $\beta_0 \ne 0$).
For the simplest model with 1 ...
- 663
0
votes
1
answer
40
views
Model, Likelihood & ABC
I'm struggling to understand what likelihood free means in ABC, since ABC is using a model as simulator to produce $y_{simulated}$. However, to me is not clear the difference between model/simulator ...
- 337
2
votes
1
answer
72
views
Why is the likelihood the product of the conditional PDF and the PDF of the parameter
Please help me understand the following:
Suppose a tester recorded the quantity $Y=X_1+\cdots X_n$ where $X_i$ has a Poisson distribution with mean $\theta$.
Now, the tester lost all samples $X_i$ ...
- 601
3
votes
0
answers
118
views
Function proportional to the log likelihood for the Gaussian distribution
The following question is crossposted from MathStackExchange upon recommendation from the MSE community and a lack of responses on my post over there.
Consider the following problem from a course on ...
- 213
0
votes
0
answers
40
views
Why the log likelihood is positive in some cases
I have a log likelihood that looks like the following.
$$ \log(p(Z|\Theta)) = -\sum_{p = 1}^{N} \left[ L \log\left(\pi\left(A F(p, \Theta) + \sigma_n^2\right) \right) + \frac{\sum_{l = 1}^{L} Z_l(p) }{...
- 113
0
votes
0
answers
32
views
Likelihood being driven to zero because of large number of model-observation misfits
I am performing a Bayesian calibration of a computer model and wondering if I am setting up my likelihood correctly. I have model output generated using Monte Carlo sampling of prior distributions of ...
- 1
0
votes
1
answer
76
views
Numerically stable transformation of log-likelihoods to probability [closed]
I have the following problem: I have log-likelihoods that need to be transformed to probabilities. One thing I have attempted is the following. Define $\kappa_{s} := \log \int p (\theta | X_s) \text{d}...
2
votes
1
answer
87
views
Generating MLE in python - Problem witth the function [closed]
After my previous question (here) I tried to improve my work with this distribution. I'm using the parametrization $$f_X(x) = \frac{\theta^2 x^{\theta-1}(\gamma-\log(x))}{1+\theta\gamma} \mathbb{I}(0&...
- 177
2
votes
1
answer
83
views
Loss function for estimating the conditional variance by fitting $y_i^2$
I'm trying to detect anomolies in a dataset $i \in \{1,2,...,N\}$ where a random variable $y_i$ is expected to be drawn from a normal distribution with mean $\mu_i=0$ and variance $\sigma_i^2 (X_i)$ ...
- 187
0
votes
1
answer
49
views
Definition of cost function and likelihood: how one appears from the other
I was reading this paper and the book "Introduction to Quantum State Estimation" by Yong Siah Teo and I am facing some issues trying to understand how the definition of the cost function ...
- 15