Given a random variable $X$ which arise from a parameterized distribution $F(X;θ)$, the likelihood is defined as the probability of observed data as a function of $θ: \text{L}(θ)=\text{P}(θ;X=x)$

learn more… | top users | synonyms (1)

0
votes
1answer
39 views

Comparing two Gaussians with likelihood

Given a univariate Gaussian with mean $\mu_1$ and variance $\sigma_1$ and a second univariate Gaussian with $\mu_2, \sigma_2$. Compare the two using the likelihood in order to find out how similar ...
0
votes
0answers
18 views

Problem computing the Log Likelihood Ratio (LLR)

The LLR is the logarithm of the ratio of probabilities of a 0 bit being transmitted versus a 1 bit being transmitted for a received signal. We can define it as: Given that r is the received signal ...
3
votes
3answers
97 views

How well does a single data point fit a distribution?

I have to come up with a way to measure the 'quality' of a distribution for a research project. We collect data over a a period of time $t_0$ through $t_1$ and then estimate the distribution that ...
1
vote
0answers
13 views

Reference prior for a three-parameter model and likelihood factorization

Let a (regular) statistical model with three parameters $\phi_1$, $\lambda_2$, $\mu$, and three observations $x_1$, $x_2$, $y$. Assume the likelihood has form $$ L(\mu,\phi_1,\lambda_2 \mid y, x_1, ...
0
votes
1answer
16 views

Rationale for MCEM

From what I've read, the main advantage of the EM algorithm is that the expectation step can be expressed in closed form giving a deterministic answer and thus 0 variance. What's the rationale then ...
2
votes
0answers
40 views

Comparing likelihoods from non-nested models

Short: I have a series of joint probabilities (likelihoods) for how likely sample $Q$ belongs to group $K$. I need to compute a p-value describing how "significant" the "top" group is compared to ...
0
votes
0answers
7 views

Likelihood for image measurement

I'm developing a multi-target track-before-detect application in a Bayesian context (i.e. using prediction and update). The likelihood function (for entire image, as I treat the entire image as a ...
0
votes
0answers
23 views

How to compare orders of magnitude?

In Fan and Li's paper "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties", they provided a proof to Theorem 1. The very last part of the proof is as follows. Some ...
3
votes
1answer
38 views

How to prove the properties of penalized likelihood estimator in Fan and Li (2001) paper

I'm reading through Fan and Li's paper "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties". In Page 2 near bottom right corner, they proposed three properties that a ...
0
votes
0answers
24 views

Calculating the Log Likelihood of models in glmnet?

glmnet() returns a lambda sequence fitobj$lambda and I would like to calculate the log likelihood of the models (LL_model) defined by the lambda sequence. The obvious solution is to just take the ...
0
votes
1answer
21 views

How to estimate which distribution a new observation belongs to, when the distributions are given by a set of obervations

The question is that I have several probability distributions. For each distribution $P_i$, I don't know what it is. Instead, I have a set of observations $\{x_i^j\}_{j=1}^{n_i}$ drawn from each of ...
0
votes
0answers
8 views

Likelihood estimator for JMP decision trees

In the documentation for JMP decision trees- it details that the calculation for the "Entropy RSquared" (Actually McFadden's $R^2$) is calculated thus: $R^2 = -2\log(L_m/L_0)$ Where $L_m$ and $L_0$ ...
6
votes
2answers
794 views

Likelihood of 10000:1 probability happening exactly once in 10,000 tries

I am interested in understanding the difference between "likelihood" of a random event with a particular probability actually occurring the exact probability it is said to be likely. i.e. if an event ...
1
vote
0answers
44 views

Question about the calculation of likelihood function

I am looking at the answer on this thread: Why likelihood is not always a density function? Here as I understand that the likelihood function is given by: $$ L(\theta) = \frac{1}{\theta} \quad ...
1
vote
1answer
51 views

Understanding Likelihood Function

Just learning Bayesian techniques through Insua et.al.'s "Bayesian Analysis of Stochastic Process Models." On page 18 they give an example of a gambler estimating the parameter $p$ in a binomial ...
0
votes
0answers
12 views

Find a likelihood function from varying data for Bayes Theorem

I'm not sure how to model a likelihood function for the following problem: Assume I've got a sensor producing the following raw values (normalized to an interval $\pm$1): In reality there are more ...
1
vote
0answers
22 views

How to use a normal log-likelihood function to estimate the variance?

I have an array of data that is normally distributed, i.e. we're dealing with a multivariate Gaussian. We write the data as $X = \{x_1, x_2, \ldots , x_N\}$ So, there are unknown parameters $\mu$ and ...
0
votes
1answer
24 views

What is the dimension of the Gaussian log-likelihood function?

I am having trouble comprehending the log-likelihood of a multivariate normal distribution. For an n-dimensional vector $\mathbf{r}$ of N i.i.d. data points $\mathbf{r}=(r_1,...,r_N)$, the ...
0
votes
0answers
9 views

How to understand a Gaussian likelihood function vs. the variance of data points? [duplicate]

This is an elementary question, but I find myself very confused visualizing this (if there are errors in anything below, please correct me): The likelihood function describes the probability density ...
0
votes
1answer
28 views

How to form likelihood function of a variable that follows a quadratic function

I am having difficulty understanding likelihood functions. If we have a probability density function of a random variable $X$ like this: $f_{X} (x)=ax^2 + bx + c$ (i.e a simple quadratic polynomial), ...
3
votes
1answer
82 views

Bayes theorem: normalisation denominator and likelihood

I have been racking my brains trying to understand Bayes theorem. So, the way I have understood is that the likelihood is the probability of observing the particular outcome given a set of parameter ...
1
vote
0answers
24 views

What does this terminology mean in this introduction to likelihood ratios?

I am currently reading In All Likelihood by Yudi Pawitan (2013 edition) and I am making my way through the second chapter on the likelihood function. In part 2.4 which is where likelihood functions ...
0
votes
0answers
18 views

log-likelihood of regression model with exponential distribution as response

I would like to calculate the log-likelihood of a simple regression model, where the response variable $y$ is exponentially distributed. I thought I could just use least-squares (LS) to find the best ...
3
votes
1answer
89 views

Likelihood-based hypothesis testing

$N_A$ and $N_B$ are variables of the counts of the number of events 'A' and events 'B' respectively. Those variables follow Poisson distributions with parameters $\lambda_A$ and $\lambda_B$. In ...
1
vote
0answers
22 views

Combined likelihood of multiple counting experiments

I have two counting experiments, each experiment is an independent detector (A and B) which detects some events (photons). Detector A has frequency resolution - it measures that a count occurred at a ...
0
votes
0answers
15 views

Using Likelihood Ratio Test to deal with heteroscedastic data results in unreliable results

Suppose $Y$ and $x$ are not related. Therefore the linear regression analysis should not reject the null hypothesis ($H_0: b=0$) in $E(Y) = a+bx$. Suppose the variance in $Y$ increase with $x$ (i.e., ...
0
votes
0answers
18 views

Marginal Likelihood of a Non-Linear Mixed Effects Model

The marginal likelihood of a non-linear mixed effects model does not admit a closed-form expression, unless data is normally distributed... or so I was told. Does anyone know of any literature, ...
1
vote
0answers
40 views

Imposing a model on a pdf

(This question is an attempt to zoom in on the key issue in this question using as little information as possible.) Lets say I want to derive the likelihood function of $\beta$ given $x$ and $y$ for ...
1
vote
0answers
42 views

Derivation of likelihood function for latent variable model made explicit

I am trying to make the steps deriving the likelihood function for the following latent variable model as explicit as possible: $$Y^0=X\beta + u$$ where $$u \sim NID(0,\sigma^2).$$ The observed data ...
2
votes
0answers
44 views

Approximating the marginal likelihood in Bayesian Model Comparison

Given some data $y$, my interest centers around a collection of models $\{\mathcal{M}_1,\mathcal{M}_2,\cdots,\mathcal{M}_L\}$ representing competing hypotheses about $y$. Each model $\mathcal{M}_l$ ...
1
vote
0answers
45 views

Profile likelihood

I am considering a normal distribution with mean $\beta_1 + \beta_2\exp(-\phi x)$ and variance $\sigma^2$, i.e. $y \sim N(\beta_1 + \beta_2\exp(-\phi x), \sigma^2) $. My aim is to calculate the ...
0
votes
0answers
31 views

Correction for Log normal distribution

Context: I have observed continuous data $\boldsymbol{O}$, for each observation $i$ I have an assumed known $\sigma_i$ for each observation I have an expected model value $E_i$. $E_i$ was produced ...
1
vote
0answers
41 views

In MCMC simulation, how to deal with very small likelihood values that couldn't be represented by computer? [duplicate]

I am working on a Bayesian project based on Stagnant data from a OpenBugs example, which is a changepoint problem. Basically we assume a model with two straight lines that meet at a certain ...
0
votes
1answer
27 views

Log Likelihoods of Exponential Families

How can one derive the log-likelihood of the saturated model of an exponential family in general? Differentiating the log likelihood w.r.t $\theta$ gives $y_i=\hat{\mu_i}$ but I don't think replacing ...
0
votes
1answer
25 views

Generalised Likelihood Models

Why would you use a re-scaled binomial distribution rather than just the standard binomial distribution as the distributional assumption in a GLM?
0
votes
0answers
21 views

likelihood ratio testing

Let $Y_{1}, Y_{2}, \ldots, Y_{n}$ denote a random sample from a $N(\theta,\ \sigma^{2})$ population. Consider testing $H_{o}: \theta\geq\theta_{o}$ versus $H_{a}: \theta<\theta_{o}.$ If ...
0
votes
0answers
28 views

log-likelihood of multivariate regression

I am struggling with R to calculate log-likelihood of multivariate regression (i.e. more than one response, one or more predictors). I would like to have a function taking object of class "mlm" as an ...
1
vote
1answer
91 views

Interpreting prior and posterior

I am bit puzzled on how we can interpret the posterior. Assume a coin which is 0.1 probable to be unfair. So our prior probability on the coin being unfair is 0.1, and being fair is 0.9. Also by ...
1
vote
1answer
43 views

Can we write the likelihood of a GLM in generality?

So I know we can explicitly write down the likelihood of any specified GLM model, for example the likelihood for the logistic regression model would be ...
3
votes
0answers
43 views

Cauchy MLE Reliability of Asymptotic Results

I have the following regression model $$ p_i = x'_i\beta +s \varepsilon_i $$ with sample size $n \approx 150$ and 4 independent variables. I have reason to believe and $\varepsilon_i$ is distributed ...
3
votes
1answer
92 views

Construct the likelihood with asymmetric uncertainties

I want to study the correlation between 2 parameters, this is done by fitting a straight line. I have uncertainties on both parameters. I want to solve my problem using the Bayesian approach, i.e. I ...
1
vote
0answers
15 views

Strange likelihood construction for information borrowing

I recently encountered a strange model while consulting in industry. The goal was to collect information about the parameter of event $A$, say $\theta_A$, but we only observed events $B$ and $C$, ...
1
vote
0answers
26 views

The most general definition of the Likelihood function for continuous data (including truncation and censoring)

How would you rigorously define the likelihood function for censored/truncated observations? Even in most lifetime/reliability literature (where these types of observations are frequently encountered) ...
4
votes
1answer
125 views

Cauchy distribution (likelihood and Fisher information)

I have a three part question: 1) If I have a $Cauchy(\theta, 1)$ with density: $p(x-\theta) = \frac{1}{\pi\{1+{(x-\theta)^2}\}}$ and $x_1, ..., x_n$ forms i.i.d sample. I see on Wiki that this will ...
1
vote
0answers
9 views

I have problem of my sampling based on secondary data [closed]

I have question and i hope i get the answer I am doing my thesis about the impact of liquidity risk in Islamic banks in GCC countries and I have to choose just two countries one which it ...
1
vote
1answer
60 views

How to calculate likelihood of linear regression

This is a pretty basic question, but one I am having a hard time finding an answer to. How do you calculate the likelihood of a simple linear model? Like, say, $$y=\beta_0+\beta_1x+e$$ I am working on ...
2
votes
1answer
79 views

Construct the likelihood if measurement uncertainties have a Gamma distribution

I want to construct my likelihood. General case: If my data do come from a line of the form $y = mx + b$ and the uncertainties are normally distributed with mean zero and known variance ...
1
vote
1answer
33 views

likelihood of string of events given a string of probabilities

There are two classes of strings of events. E.g. A: 0,0,1,2,2,3,4,0,3,0,0,0 B: 0,0,0,0,3,3,2,1,5,6,7,0 Both class A and B strings exhibit variability. Many (e.g. ...
0
votes
2answers
37 views

Finding the posterior distribution

Suppose that an observation $X$ is drawn from the following distribution $f(x|\theta) = \begin{cases} \frac{1}{\theta} & \text{ if } 0 < x < \theta \\ 0 & \text{ if } otherwise ...
1
vote
0answers
62 views

how to construct the likelihood if my errors are not Gaussian

My aim is to study the correlation between 2 parameters knowing that I have measurement errors in both parameters, i.e. I have uncertainties on the independent and dependent parameters. I want to ...