Given a random variable $X$ which arise from a parameterized distribution $F(X;θ)$, the likelihood is defined as the probability of observed data as a function of $θ: \text{L}(θ)=\text{P}(θ;X=x)$

learn more… | top users | synonyms (1)

0
votes
0answers
6 views

Prior distributions letting a small sample “speak”

I’ve got a general question. Let k be a parameter which must be estimated. It lies within the interval [a;b], a and b being finite real numbers. Let us further assume we dispose of a series of ...
0
votes
0answers
56 views

Log-likelihood proof and AIC hypothesis

First of all, statistics is just not my thing ... yet (I hope!) I'm having a hard time finding out the log-likelihood equation: Given $Y \rightarrow \mathcal{N}(\mu_1,\sigma_1)$ (observation) and ...
0
votes
0answers
24 views

Change detection in hidden markov models

I have many questions about hidden Markov models. Let $Z_1$, $Z_2$, ..., $Z_n$ be the latent variables, and $X_1$, $X_2$, ... $X_n$ be the observed ones. Let's assume that the parameters of the ...
0
votes
0answers
13 views

Forming a likelihood from an implicit function

My problem involves two signals on a common value where the signals are tied together by an implicit function. I want to figure out a generic approach for how to find the likelihood of the common ...
1
vote
1answer
13 views

Long term probability of a game with multiple outcomes?

I'm not a stats expert so please excuse me if this is a really basic question and I've simply missed the reference... So let's say this person is playing a game with a small chance of winning, say ...
2
votes
0answers
64 views

Computing likelihood of a mixed-effect model manually

My question has to do with how to manually compute the likelihood of a mixed-effect model. I understand how to determine the likelihood of a fixed-effect model manually. For example, if I make up ...
0
votes
1answer
20 views

How does normalizing the response affect likelihood?

I have a vector of experiment outcomes, $Q$, and I assume that $Q_i$ are generated by a Gaussian distribution, i.i.d., such that the likelihood is the standard $$\mathcal{L}(q_1, ..., q_n) = ...
0
votes
0answers
27 views

delta function as the likelihood in Bayes theorem

I am reading a paper which is doing a MAP estimation on the following model: $$ \phi_t = \max_k p(\phi_w|\phi_t) p(\phi_t) $$ So we are seeking $\phi_t$ which maximizes that joint distribution. ...
0
votes
0answers
15 views

Comparing likelihoods of different experiments

Call $X_{i,j} \sim B(p_{i,j})$ a set of independent coin tosses with parameters $\{p_{i,j}\}_{i,j}$ where $j\in[1,p]$ and $i\in[1,n]$. In plain text, all the coins are biased in a different way. ...
2
votes
1answer
89 views

What exactly does it mean to and why must one update prior?

I'm still trying to understand prior and posterior distributions in Bayesian inference. In this question, one flips a coin. Priors: unfair is 0.1, and being fair is 0.9 Coin is flipped 10x and ...
0
votes
0answers
56 views

MLE of a multivariate Hawkes process

I'm struggling with implementing the maximum likelihood estimator for a multivariate Hawkes process (HP). Specifically, while the analytical expression for a log-likelihood function of a univariate HP ...
0
votes
1answer
47 views

Comparing two Gaussians with likelihood

Given a univariate Gaussian with mean $\mu_1$ and variance $\sigma_1$ and a second univariate Gaussian with $\mu_2, \sigma_2$. Compare the two using the likelihood in order to find out how similar ...
3
votes
3answers
104 views

How well does a single data point fit a distribution?

I have to come up with a way to measure the 'quality' of a distribution for a research project. We collect data over a a period of time $t_0$ through $t_1$ and then estimate the distribution that ...
1
vote
0answers
13 views

Reference prior for a three-parameter model and likelihood factorization

Let a (regular) statistical model with three parameters $\phi_1$, $\lambda_2$, $\mu$, and three observations $x_1$, $x_2$, $y$. Assume the likelihood has form $$ L(\mu,\phi_1,\lambda_2 \mid y, x_1, ...
0
votes
1answer
18 views

Rationale for MCEM

From what I've read, the main advantage of the EM algorithm is that the expectation step can be expressed in closed form giving a deterministic answer and thus 0 variance. What's the rationale then ...
2
votes
0answers
42 views

Comparing likelihoods from non-nested models

Short: I have a series of joint probabilities (likelihoods) for how likely sample $Q$ belongs to group $K$. I need to compute a p-value describing how "significant" the "top" group is compared to ...
0
votes
0answers
8 views

Likelihood for image measurement

I'm developing a multi-target track-before-detect application in a Bayesian context (i.e. using prediction and update). The likelihood function (for entire image, as I treat the entire image as a ...
0
votes
0answers
24 views

How to compare orders of magnitude?

In Fan and Li's paper "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties", they provided a proof to Theorem 1. The very last part of the proof is as follows. Some ...
3
votes
1answer
40 views

How to prove the properties of penalized likelihood estimator in Fan and Li (2001) paper

I'm reading through Fan and Li's paper "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties". In Page 2 near bottom right corner, they proposed three properties that a ...
0
votes
0answers
30 views

Calculating the Log Likelihood of models in glmnet?

glmnet() returns a lambda sequence fitobj$lambda and I would like to calculate the log likelihood of the models (LL_model) defined by the lambda sequence. The obvious solution is to just take the ...
0
votes
1answer
22 views

How to estimate which distribution a new observation belongs to, when the distributions are given by a set of obervations

The question is that I have several probability distributions. For each distribution $P_i$, I don't know what it is. Instead, I have a set of observations $\{x_i^j\}_{j=1}^{n_i}$ drawn from each of ...
0
votes
0answers
11 views

Likelihood estimator for JMP decision trees

In the documentation for JMP decision trees- it details that the calculation for the "Entropy RSquared" (Actually McFadden's $R^2$) is calculated thus: $R^2 = -2\log(L_m/L_0)$ Where $L_m$ and $L_0$ ...
6
votes
2answers
870 views

Likelihood of 10000:1 probability happening exactly once in 10,000 tries

I am interested in understanding the difference between "likelihood" of a random event with a particular probability actually occurring the exact probability it is said to be likely. i.e. if an event ...
1
vote
0answers
44 views

Question about the calculation of likelihood function

I am looking at the answer on this thread: Why likelihood is not always a density function? Here as I understand that the likelihood function is given by: $$ L(\theta) = \frac{1}{\theta} \quad ...
1
vote
1answer
54 views

Understanding Likelihood Function

Just learning Bayesian techniques through Insua et.al.'s "Bayesian Analysis of Stochastic Process Models." On page 18 they give an example of a gambler estimating the parameter $p$ in a binomial ...
0
votes
0answers
12 views

Find a likelihood function from varying data for Bayes Theorem

I'm not sure how to model a likelihood function for the following problem: Assume I've got a sensor producing the following raw values (normalized to an interval $\pm$1): In reality there are more ...
1
vote
0answers
25 views

How to use a normal log-likelihood function to estimate the variance?

I have an array of data that is normally distributed, i.e. we're dealing with a multivariate Gaussian. We write the data as $X = \{x_1, x_2, \ldots , x_N\}$ So, there are unknown parameters $\mu$ and ...
0
votes
1answer
28 views

What is the dimension of the Gaussian log-likelihood function?

I am having trouble comprehending the log-likelihood of a multivariate normal distribution. For an n-dimensional vector $\mathbf{r}$ of N i.i.d. data points $\mathbf{r}=(r_1,...,r_N)$, the ...
0
votes
0answers
9 views

How to understand a Gaussian likelihood function vs. the variance of data points? [duplicate]

This is an elementary question, but I find myself very confused visualizing this (if there are errors in anything below, please correct me): The likelihood function describes the probability density ...
0
votes
1answer
31 views

How to form likelihood function of a variable that follows a quadratic function

I am having difficulty understanding likelihood functions. If we have a probability density function of a random variable $X$ like this: $f_{X} (x)=ax^2 + bx + c$ (i.e a simple quadratic polynomial), ...
3
votes
1answer
88 views

Bayes theorem: normalisation denominator and likelihood

I have been racking my brains trying to understand Bayes theorem. So, the way I have understood is that the likelihood is the probability of observing the particular outcome given a set of parameter ...
1
vote
0answers
24 views

What does this terminology mean in this introduction to likelihood ratios?

I am currently reading In All Likelihood by Yudi Pawitan (2013 edition) and I am making my way through the second chapter on the likelihood function. In part 2.4 which is where likelihood functions ...
0
votes
0answers
22 views

log-likelihood of regression model with exponential distribution as response

I would like to calculate the log-likelihood of a simple regression model, where the response variable $y$ is exponentially distributed. I thought I could just use least-squares (LS) to find the best ...
3
votes
1answer
95 views

Likelihood-based hypothesis testing

$N_A$ and $N_B$ are variables of the counts of the number of events 'A' and events 'B' respectively. Those variables follow Poisson distributions with parameters $\lambda_A$ and $\lambda_B$. In ...
1
vote
0answers
23 views

Combined likelihood of multiple counting experiments

I have two counting experiments, each experiment is an independent detector (A and B) which detects some events (photons). Detector A has frequency resolution - it measures that a count occurred at a ...
0
votes
0answers
17 views

Using Likelihood Ratio Test to deal with heteroscedastic data results in unreliable results

Suppose $Y$ and $x$ are not related. Therefore the linear regression analysis should not reject the null hypothesis ($H_0: b=0$) in $E(Y) = a+bx$. Suppose the variance in $Y$ increase with $x$ (i.e., ...
0
votes
0answers
22 views

Marginal Likelihood of a Non-Linear Mixed Effects Model

The marginal likelihood of a non-linear mixed effects model does not admit a closed-form expression, unless data is normally distributed... or so I was told. Does anyone know of any literature, ...
1
vote
0answers
40 views

Imposing a model on a pdf

(This question is an attempt to zoom in on the key issue in this question using as little information as possible.) Lets say I want to derive the likelihood function of $\beta$ given $x$ and $y$ for ...
1
vote
0answers
44 views

Derivation of likelihood function for latent variable model made explicit

I am trying to make the steps deriving the likelihood function for the following latent variable model as explicit as possible: $$Y^0=X\beta + u$$ where $$u \sim NID(0,\sigma^2).$$ The observed data ...
3
votes
0answers
49 views

Approximating the marginal likelihood in Bayesian Model Comparison

Given some data $y$, my interest centers around a collection of models $\{\mathcal{M}_1,\mathcal{M}_2,\cdots,\mathcal{M}_L\}$ representing competing hypotheses about $y$. Each model $\mathcal{M}_l$ ...
1
vote
0answers
46 views

Profile likelihood

I am considering a normal distribution with mean $\beta_1 + \beta_2\exp(-\phi x)$ and variance $\sigma^2$, i.e. $y \sim N(\beta_1 + \beta_2\exp(-\phi x), \sigma^2) $. My aim is to calculate the ...
0
votes
0answers
33 views

Correction for Log normal distribution

Context: I have observed continuous data $\boldsymbol{O}$, for each observation $i$ I have an assumed known $\sigma_i$ for each observation I have an expected model value $E_i$. $E_i$ was produced ...
1
vote
0answers
42 views

In MCMC simulation, how to deal with very small likelihood values that couldn't be represented by computer? [duplicate]

I am working on a Bayesian project based on Stagnant data from a OpenBugs example, which is a changepoint problem. Basically we assume a model with two straight lines that meet at a certain ...
0
votes
1answer
27 views

Log Likelihoods of Exponential Families

How can one derive the log-likelihood of the saturated model of an exponential family in general? Differentiating the log likelihood w.r.t $\theta$ gives $y_i=\hat{\mu_i}$ but I don't think replacing ...
0
votes
1answer
25 views

Generalised Likelihood Models

Why would you use a re-scaled binomial distribution rather than just the standard binomial distribution as the distributional assumption in a GLM?
0
votes
0answers
21 views

likelihood ratio testing

Let $Y_{1}, Y_{2}, \ldots, Y_{n}$ denote a random sample from a $N(\theta,\ \sigma^{2})$ population. Consider testing $H_{o}: \theta\geq\theta_{o}$ versus $H_{a}: \theta<\theta_{o}.$ If ...
0
votes
0answers
30 views

log-likelihood of multivariate regression

I am struggling with R to calculate log-likelihood of multivariate regression (i.e. more than one response, one or more predictors). I would like to have a function taking object of class "mlm" as an ...
2
votes
1answer
125 views

Interpreting prior and posterior

I am bit puzzled on how we can interpret the posterior. Assume a coin which is 0.1 probable to be unfair. So our prior probability on the coin being unfair is 0.1, and being fair is 0.9. Also by ...
1
vote
1answer
45 views

Can we write the likelihood of a GLM in generality?

So I know we can explicitly write down the likelihood of any specified GLM model, for example the likelihood for the logistic regression model would be ...
3
votes
0answers
47 views

Cauchy MLE Reliability of Asymptotic Results

I have the following regression model $$ p_i = x'_i\beta +s \varepsilon_i $$ with sample size $n \approx 150$ and 4 independent variables. I have reason to believe and $\varepsilon_i$ is distributed ...