Given a random variable $X$ which arise from a parameterized distribution $F(X;θ)$, the likelihood is defined as the probability of observed data as a function of $θ: \text{L}(θ)=\text{P}(θ;X=x)$

learn more… | top users | synonyms

1
vote
0answers
13 views

Simulated MLE does not exist, when trying to Bootstrap likelihood combinant

Consider this simple logistic model: We have ten $0/1$ observations $y_1,...,y_{10}.$ We model with an intercept and a predictor variable.The ten first observations have predictor value $X_i=0$, ...
2
votes
2answers
28 views

ABC: Why not use the distance measure as a pseudo-likelihood instead?

I've read about the ABC rejection algorithm when not being able to calculate the likelihood directly, and my question is: if we have to introduce a distance measure $\rho(D,D')$ anyways, why not use ...
0
votes
0answers
15 views

Pseudo-likelihood and RBMs

I need to train a restricted Boltzmann machine to model the joint probability of categorical variables. For this I adapted a Bernoulli RBM to have groups of softmax units in the visible layer. The ...
1
vote
1answer
103 views

Why likelihood is not always a density function? [duplicate]

I try to self-learn Bayesian machine learning (mostly by studying Bishop and Kevin Murphy's books). While working with formulas I was puzzled by the quote that "Note that the likelihood function is ...
2
votes
1answer
42 views

fitdistr loglik in R

I am fitting a few time series using fitdistr in R. To see how different distributions fit the data, I compare the log likelihood from the ...
2
votes
1answer
54 views

How to construct a reasonable prior and likelihood for Bayes modelling?

To apply Bayes inference for data analysis or machine learning, we have to construct prior and likelihood, right? But if I fail to come up with a reasonable prior and likelihood, then the Bayes model ...
3
votes
0answers
33 views

Likelihood Function for Complicated Transformations

Suppose that data X have a Normal distribution with some mean $\mu$ and some variance $\sigma^2$. However, you don't get to see X. Instead, you see $Y = g(X)$ where $g$ is a known function. Assume ...
0
votes
0answers
16 views

Marginal Likelihood & Truncated Posterior Mixing

It has now become common to use Bayesian inference to find the best solution for exoplanet orbits fitting. In order to find the best solution one has to explore a very large parameter space and some ...
0
votes
1answer
24 views

Local log-likelihood for multiclass linear regression model

In page 206 of the book 'Elements of statistical learning', the author wrote: The local log-likelihood for this $J$ class model can be written $\sum_{i=1}^NK_\lambda (x_0, ...
0
votes
1answer
32 views

Cannot replicate the AIC in a GARCH model

First I am confused what the ugarchfit in the rugarch package means by likelihood versus loglikelihood. In the complete ...
1
vote
0answers
96 views

assumptions of REML

In Patterson and Thompson 1971, $L$ is the log-likelihood of $y$ $$ L = const - \frac{1}{2}log |H| - \frac{1}{2}n log (\sigma^2) - \frac{1}{2\sigma^2} (y - X\alpha)'H^{-1}(y-Xa) $$ They consider the ...
8
votes
5answers
492 views

What does “likelihood is only defined up to a multiplicative constant of proportionality” mean in practice?

I'm reading a paper where the authors are leading from a discussion of maximum likelihood estimation to Bayes' Theorem, ostensibly as an introduction for beginners. As a likelihood example, they ...
2
votes
1answer
90 views

Relation of Mahalanobis Distance to Log Likelihood

The Wikipedia entry on Mahalanobis Distance contains this note: Another intuitive description of Mahalanobis distance is that it is square root of the negative log likelihood. That is, the ...
0
votes
0answers
18 views

Square distance and likelihood in k-means

In k-means algorithm, the distance minimization step is equivalent to maximize likelihood: $P(X|\theta)$ or to maximize posterior distribution $P(\theta|X)$? I think it's more logical to maximize ...
0
votes
0answers
22 views

Multivariate normal distribution log-likelihood

Let's say we have two 2x5 matrices A and B, and a 4x5 matrix C which is generated by concatenating A and B (C=[A;B] in MATLAB representation). Would the sum of the log-likelihoods of A and B under a ...
0
votes
1answer
35 views

How to find log-likelihood of multiple sequences for hmm using Kevin Murphy toolkit for MATLAB

I have an observation sequence of TPM, EPM and prior. I want to find the log-likelihood of around 100 sequences of length 10 at a time. How can I do this using a forward algorithm?
0
votes
0answers
17 views

Favorable properties for penalized likelihood estimators

I'm reading through Fan and Li's SCAD paper on Nonconcave Penalized Likelihood. The idea is to penalize the loss function by $p_\lambda(\theta)$ where $\lambda$ can be some tuning parameter. Fan ...
-1
votes
2answers
52 views

How do we analyse likelihood in a dataset? [closed]

I am working to analyze poverty rate using census data. I have a huge dataset. I want to extract the likelihood from this dataset in order to create patterns for energy consumption. What is the ...
3
votes
1answer
72 views

Likelihood vs. Probability

I have difficulties with Likelihoods. I do understand Bayes' Theorem $$p(A|B, \mathcal{H}) = \frac{p(B|A, \mathcal{H}) p(A|\mathcal{H})}{p(B|\mathcal{H})}$$ which can be directly deduced from ...
1
vote
0answers
15 views

Empty asymptotic confidence interval?

I have a sample $x=(4, 3, 1, 2, 2, 2, 2, 5, 7, 3, 1, 2, 3, 4, 3, 2, 3, 3, 3, 4)$ of size $n=20$. from a binomial distribution with 10 trials and probability of success $p$. I am asked to construct the ...
0
votes
1answer
40 views

How do I think about conditional probability in this situation?

I am trying to understand some data relating the likelihood of a positive stock return following a certain signal. The frequency of positive returns differ across datasets (over a particular time ...
4
votes
0answers
71 views

Can likelihood be changed when the prior changes?

I have a data which follows gamma distribution and want to know the uncertainty of the parameters of this data. $\text{Data} \sim \text{Gamma} (\alpha, \beta)$ Parameters $\alpha \sim \text{Gamma} ...
0
votes
0answers
21 views

Marginal Likelihood Prior

I have a model with probability matrix for a distribution of $x$, $y=\{0,1\}$, $p(x,y|w)$ where $w=[w_1,w_2,w_3,w_4]$ $p(x=0,y=0)=w_1$, $p(x=0,y=1)=w_2$ $p(x=1,y=0)=w_3$, ...
2
votes
0answers
39 views

The probability that one bernoulli process has a higher p than another?

I have two data generating processes that are independent Bernoulli processes with probabilities of success $p_A$ and $p_B$. I am taking repeated samples from these two data generating processes, so ...
3
votes
2answers
155 views

How to decide on the MLE when pmf is 0?

Suppose you have $\theta=\{1,2\}$ and the sample of (0,1,2) with the task of finding MLE: \begin{array} {|c|c|c|} \hline x & p(x|\theta=1) & p(x|\theta=2) \\ \hline 0 & 1/2 & 1/4 \\ ...
0
votes
0answers
26 views

likelihood of a model

the likelihood of a model is defined as the probability of data given model: Likelihood(Model) = p(DATApoints | Model) which is equivalent to the product of all p(datapoint | Model) for each ...
0
votes
0answers
61 views

how to choose from Poisson and Poisson random effect model?

i have fitted both a Poisson random effect and Poisson model to my data,however some of the estimates are of different signs and SE are quite similar for both of the models. The log likelihood for the ...
3
votes
1answer
74 views

When is Likelihood Function Positive Semidefinite

This may be a very misinformed question, but I cant figure out why its not true. Here goes: According to Wikipedia and this post, the hessian of a likelihood function equals the information matrix, ...
2
votes
1answer
85 views

relation between confidence interval and likelihood function

I once meet the following question,which is also listed by book written by Cosma Rohilla Shalizi ...
0
votes
1answer
69 views

Likelihood score function 101

I have some trouble with score functions in likelihood calculation. I'm not good at statistics or probability, so I'm still confused on formalism and mathematical-probabilistic language. Some ...
0
votes
0answers
28 views

Can I split this likelihood term

I have a likelihood that is modelled using the IID distributed noise assumption. Now the likelihood at a 3D location $i$ is normally distributed with 0 mean and some precision $\sigma$. So, I can ...
7
votes
3answers
199 views

Posterior very different to prior and likelihood

If the prior and the likelihood are very different from each other, then sometimes a situation occurs where the posterior is similar to neither of them. See for example this picture, which uses normal ...
2
votes
0answers
61 views

Expectation of density ratio of two iid variables

Let $X \sim N(0,1)$ and $Y \sim N(0,1)$ be independent RVs and let $f$ be their density function. I'd like to compute the expectation of the density ratio \begin{align} ...
1
vote
0answers
25 views

Fitting of bivariate data to a self-defined probability density function

I have a bivariate set of data points which I want to fit to a self-defined distribution (i.e. not standard normal or chi-square or like that, a different, let's say "new" density function). I would ...
0
votes
1answer
56 views

Posterior and Likelihood probabilities meaning [duplicate]

I am a computer scientist, so I have a background at maths (however limited). I am reading about posterior distribution from here http://en.wikipedia.org/wiki/Posterior_distribution . It says there: ...
2
votes
0answers
64 views

Fisher information of a statistic

I have a random sample $(X_1, X_2,...,X_n)$ and I have an estimator $\bar{X_n}=\sum_{i=1}^{n} X_i$ I need to compute the Fisher information of $\bar{X_n}$. The Fisher information is defined as ...
2
votes
0answers
73 views

Likelihood principle: difference between weak and strong version

Does anyone understand the difference between weak likelihood principle and strong likelihood principle?
1
vote
1answer
35 views

Log-likelihood via posterior sampling

I wish to evaluate the following quantity $\log\int_\mathcal{R}p(y|\beta)p(\beta)d\beta$ where $p(y|\beta)=exp(-|y-\beta|)$ and $p(\beta)=\mathcal{N}(0,1000)$. The $y$'s are known. You could probably ...
1
vote
1answer
81 views

Likelihood of censored data

Let $X_1,X_2,\ldots, X_{n_1}$ be IID with PDF $f(x-\theta) $, for $-\infty<x<\infty$ and $-\infty<\theta<\infty$. Denote the CDF of $X_i$ by $F(x-\theta)$. Let $Z_1,Z_2, \ldots, Z_{n_2}$ ...
3
votes
1answer
132 views

Proof that the log-likelihood is asymptotically quadratic

I was reading this article, where the author says that Maximum Likelihood (ML) estimates are asymptotically normal if the log-likelihood is asymptotically quadratic. I have heard or read other ...
0
votes
0answers
53 views

Multiparameter log-likelihood function in R and power analysis (by using simulations)

I would like to estimate power of the following problem. I am interested in comparing two groups that both follow e.g. Weibull distribution. So, group A has two parameters (...
0
votes
2answers
192 views

Most suitable algorithm for optimizing Maximum likelihood function

What is the most suitable optimization algorithm for optimizing maximum likelihood estimator? In excel I used GRG non linear optimization algorithm, is that good enough? I want to write my own code ...
2
votes
0answers
64 views

Lizards probability

I am trying to learn basic probability and I would like to get the answer of the questions below. I honestly have no idea on how to even start solving it. Could someone show me or give me some advice ...
1
vote
0answers
30 views
1
vote
1answer
256 views

Minimizing relative error (or mean square error) and maximizing likelihood

I'm not a statistician, so I would appreciate an answer in the simplest possible words. I've read that, in some sense, when we minimize the mean square error, we are maximizing the likelihood. This ...
0
votes
0answers
36 views

Testing time homogeneous Markov chains

I am working with different transition diagrams and want to calculate the likelihood ratio statistic for testing time-homogeneous. I saw that there are already some comparable questions, but I still ...
1
vote
0answers
40 views

Formal statistical test for comparing likelihood distributions obtained via MCMC

I am trying to formally compare the distribution of the likelihood values generated using two different models with marginal posterior values of the parameters obtained using MCMC in order to assess ...
0
votes
1answer
106 views

Doubt in derivative of logarithm

This seem to silly but I wanted to confirm if the derivative of the log-likelihood $\hskip 2 pt l(x_i)$. The derivative of $$\frac{d (\sum_{i=1}^{M} log(x_i))}{dx} = \frac{1}{x_i} \sum_{i=1}^{M} ...
1
vote
1answer
69 views

How can I use likelihoods to compare these three groups? Should I want to do this?

EDIT: Perhaps I should also note that my initial attempt at analyzing this data used a hierarchical model in rJAGS sampling the means from uniform distributions and variances from gamma distributions ...
1
vote
0answers
72 views

How to use profile likelihood?

I am new to profile likelihood and do not really understand what advantages it may have. Lets say I have the following results estimating the means of three groups. What can I say about them? R ...