Given a random variable $X$ which arise from a parameterized distribution $F(X;θ)$, the likelihood is defined as the probability of observed data as a function of $θ: \text{L}(θ)=\text{P}(θ;X=x)$

learn more… | top users | synonyms

0
votes
0answers
10 views

Maximizing Log-Likelihood Estimation for Changepoint Detection

I'm trying to code the changepoint detection algo described here: ...
0
votes
0answers
24 views

Recursive Bayesian Estimation, $p(C_k|\mathbf{x})$ as (discrete) likelihood

I''ve been struggeling with this problem for the last couple of days. The main goal is to use the probabilistic classification output $p(C_k|\mathbf{x})$, from for example a logistic regression, to ...
1
vote
1answer
47 views

Probability to Likelihood

I have a problem on calculating the likelihood of observing a data point x given the predicted lable. My application is on text classification where I have to detect Spam and No Spam documents. I ...
2
votes
1answer
63 views

What would be the likelihood function of a pdf, $p(n)=1-|n|$ for $|n|<1$?

This might seem like a basic question to some but I am utterly confused by the fact that the given pdfs are not Gaussian or any other distribution commonly seen in examples. I have two hypotheses ...
0
votes
0answers
58 views

Gaussian Process Regression/ Classification

How do we estimate parameters of the model while performing Gaussian Process Regression or Classification? While performing regression, we estimate parameters such that the model is the best fit to ...
1
vote
0answers
24 views

Cross validation with unequal sample size for the left out sets

I am trying to do cross validation on several (20) subsets of samples, which all have unequal sample size. I cannot subsample so that sizes are equal. Example: batch 1: 500 samples batch 2: 400 ...
1
vote
1answer
40 views

R - MLE of modified Champernowne density

I've come across an article (http://papers.ssrn.com/sol3/papers.cfm?abstract_id=704903), in which author wrote about maximum likelihood estimates of parameters in the so called modified Champernowne ...
8
votes
1answer
101 views

Observed Fisher information under a transformation

From "In All Likelihood: Statistical Modeling and Inference Using Likelihood" by Y. Pawitan, the likelihood of a re-parameterization $\theta\mapsto g(\theta)=\psi$ is defined as $$ ...
2
votes
1answer
25 views

Copula estimation

I want to fit a copula distribution. My question is: Is it equivalent to estimate the marginal distributions using marginal samples and later estimate the parameters of a copula to estimating all the ...
1
vote
0answers
18 views

Marginal pseudo-likelihood and consistency?

For a given set of random variables, $X_1,...,X_n$ we know that in many cases finding the maximum of the pseudo likelihood: $$PL(x_1,\ldots,x_n) = \prod_{i=1}^n p(x_i | ...
3
votes
1answer
42 views

How is it meaningful to give fitted values for random effects if they don't appear in the likelihood function?

Software for fitting random effects models tends to output values for the random effects. For example, suppose the model is $$y_i = \alpha_i + \varepsilon_i, \qquad 1 \le i \le n$$ where $$\alpha_i ...
0
votes
0answers
6 views

Segregation Analysis for predicting age-specific cancer risk

I am relatively new to the worlds of bioinformatics and genetics research. I have been tasked with presenting to my lab the potential value of a paper that uses Complex Segregation Analysis for a risk ...
0
votes
0answers
16 views

How adapt MCMC when (constant) weights for oberservations are introduced?

I have the following problem: I have already set up a model and MCMC sampler for a mixed model without weights, i.e., every observation contributes the same amount of information. Now I would like to ...
1
vote
1answer
25 views

Distribution of log-likelihood gradient

My question is simple: Is there any results regarding the distribution of log-likelihood function gradient? It may be asymptotic results as well.
0
votes
0answers
27 views

likelihood problem with exponential distribution using R

How can I solve this problem using likelihood in R? The data below are the survival time (in years) for 10 patients over 65 years after the diagnosis of a particular type of cancer: 10.5 0.2 7.4 0.8 ...
0
votes
0answers
10 views

In-sample likelihood ratio test for losses

Probably a very simple question for practitioners but I am doing this the very first time: I have written a programm for the estimation of a conditional variance model (HEAVY) of return data and an ...
2
votes
0answers
53 views

Problem with Finding Likelihood: Bayesian

I am really unfamiliar with Bayesian methods particularly parameter estimation. Suppose I have a test to find a parameter, theta which is the number of packaged bag for retail sale that could contain ...
0
votes
0answers
9 views

Trying to reproduce predictive likelihood

I'm trying to reproduce the average predictive likelihood from page 13 (Eq 4.1) from the following paper and I'm struggling to find the function f_M,k(...). It details more about it on page 14 but I ...
0
votes
0answers
35 views

Distribution of a MA(1) process

Suppose I have this MA(1) model: $y_t = \mu + \epsilon_t + \theta \epsilon_{t-1}$ with $\epsilon_t \sim \mathcal{N}(0,\sigma^2)$ The marginal distribution of $y_t$ for all $t$ is ...
0
votes
0answers
18 views

Log likelihood of a Markov network

This is the plot of the log likelihood function from Daphne Koller's course. Two axes are $\theta_{a_1, b_1}$ and $\theta_{b_0, c_1}$. Can someone please explain for me the shape of this plot? From ...
1
vote
0answers
39 views

log-likelihood for a beta-binomial in R

Although I may ask a question that is already solved (but I found none that would explicitly refer to this), I would like to know, if (and I am no statistician) I am right with programming the ...
1
vote
0answers
14 views

Marginal Likelihood Latent Variable Model

I am trying to apply the method proposed by Chib in Marginal Likelihood from the Metropolis Hastings output to calculate the marginal likelihood of a logit model the includes latent variables. ...
0
votes
0answers
19 views

Can we use log-likelihood to cluster classes?

I have an SVM classifier for m classes and n data points (somewhat evenly distributed across each class). Could I use the resulting MxN log likelihood matrix to merge classes that are similar?
1
vote
1answer
57 views

Loglikelihood and unusual transformations with binary response models

I was just doing some thought on the transformation for likelihoods to log likelihoods for binary response models and I realized what I am sure many people have realized before that the transformation ...
0
votes
1answer
42 views

Log-likelihood (and AIC) of robust nlrob model differs from standard nls model

Comparing models generated by nlrob to ones generated by nls, I've noticed that even though the models might be nearly identical, the log-likelihood of the models is sometimes significantly different, ...
1
vote
0answers
24 views

Estimation of log-likelihood via importance sampling

I am looking at a model trained with stochastic gradient variational Bayes. In this paper an importance sampler is proposed to estimate the likelihood: $$p(x) \approx {1 \over S} \sum_{s=1}^S ...
1
vote
0answers
15 views

Simulated MLE does not exist, when trying to Bootstrap likelihood combinant

Consider this simple logistic model: We have ten $0/1$ observations $y_1,...,y_{10}.$ We model with an intercept and a predictor variable.The ten first observations have predictor value $X_i=0$, ...
3
votes
2answers
48 views

ABC: Why not use the distance measure as a pseudo-likelihood instead?

I've read about the ABC rejection algorithm when not being able to calculate the likelihood directly, and my question is: if we have to introduce a distance measure $\rho(D,D')$ anyways, why not use ...
0
votes
0answers
45 views

Pseudo-likelihood and RBMs

I need to train a restricted Boltzmann machine to model the joint probability of categorical variables. For this I adapted a Bernoulli RBM to have groups of softmax units in the visible layer. The ...
1
vote
1answer
114 views

Why likelihood is not always a density function? [duplicate]

I try to self-learn Bayesian machine learning (mostly by studying Bishop and Kevin Murphy's books). While working with formulas I was puzzled by the quote that "Note that the likelihood function is ...
2
votes
1answer
96 views

fitdistr loglik in R

I am fitting a few time series using fitdistr in R. To see how different distributions fit the data, I compare the log likelihood from the ...
2
votes
1answer
67 views

How to construct a reasonable prior and likelihood for Bayes modelling?

To apply Bayes inference for data analysis or machine learning, we have to construct prior and likelihood, right? But if I fail to come up with a reasonable prior and likelihood, then the Bayes model ...
3
votes
0answers
40 views

Likelihood Function for Complicated Transformations

Suppose that data X have a Normal distribution with some mean $\mu$ and some variance $\sigma^2$. However, you don't get to see X. Instead, you see $Y = g(X)$ where $g$ is a known function. Assume ...
0
votes
0answers
18 views

Marginal Likelihood & Truncated Posterior Mixing

It has now become common to use Bayesian inference to find the best solution for exoplanet orbits fitting. In order to find the best solution one has to explore a very large parameter space and some ...
1
vote
1answer
34 views

Local log-likelihood for multiclass linear regression model

In page 206 of the book 'Elements of statistical learning', the author wrote: The local log-likelihood for this $J$ class model can be written $\sum_{i=1}^NK_\lambda (x_0, ...
0
votes
1answer
54 views

Cannot replicate the AIC in a GARCH model

First I am confused what the ugarchfit in the rugarch package means by likelihood versus loglikelihood. In the complete ...
2
votes
0answers
104 views

assumptions of REML

In Patterson and Thompson 1971, $L$ is the log-likelihood of $y$ $$ L = const - \frac{1}{2}log |H| - \frac{1}{2}n log (\sigma^2) - \frac{1}{2\sigma^2} (y - X\alpha)'H^{-1}(y-Xa) $$ They consider the ...
8
votes
5answers
583 views

What does “likelihood is only defined up to a multiplicative constant of proportionality” mean in practice?

I'm reading a paper where the authors are leading from a discussion of maximum likelihood estimation to Bayes' Theorem, ostensibly as an introduction for beginners. As a likelihood example, they ...
2
votes
1answer
114 views

Relation of Mahalanobis Distance to Log Likelihood

The Wikipedia entry on Mahalanobis Distance contains this note: Another intuitive description of Mahalanobis distance is that it is square root of the negative log likelihood. That is, the ...
0
votes
0answers
33 views

Square distance and likelihood in k-means

In k-means algorithm, the distance minimization step is equivalent to maximize likelihood: $P(X|\theta)$ or to maximize posterior distribution $P(\theta|X)$? I think it's more logical to maximize ...
0
votes
0answers
24 views

Multivariate normal distribution log-likelihood

Let's say we have two 2x5 matrices A and B, and a 4x5 matrix C which is generated by concatenating A and B (C=[A;B] in MATLAB representation). Would the sum of the log-likelihoods of A and B under a ...
0
votes
1answer
74 views

How to find log-likelihood of multiple sequences for hmm using Kevin Murphy toolkit for MATLAB

I have an observation sequence of TPM, EPM and prior. I want to find the log-likelihood of around 100 sequences of length 10 at a time. How can I do this using a forward algorithm?
0
votes
0answers
21 views

Favorable properties for penalized likelihood estimators

I'm reading through Fan and Li's SCAD paper on Nonconcave Penalized Likelihood. The idea is to penalize the loss function by $p_\lambda(\theta)$ where $\lambda$ can be some tuning parameter. Fan ...
-1
votes
2answers
56 views

How do we analyse likelihood in a dataset? [closed]

I am working to analyze poverty rate using census data. I have a huge dataset. I want to extract the likelihood from this dataset in order to create patterns for energy consumption. What is the ...
3
votes
1answer
92 views

Likelihood vs. Probability

I have difficulties with Likelihoods. I do understand Bayes' Theorem $$p(A|B, \mathcal{H}) = \frac{p(B|A, \mathcal{H}) p(A|\mathcal{H})}{p(B|\mathcal{H})}$$ which can be directly deduced from ...
1
vote
0answers
18 views

Empty asymptotic confidence interval?

I have a sample $x=(4, 3, 1, 2, 2, 2, 2, 5, 7, 3, 1, 2, 3, 4, 3, 2, 3, 3, 3, 4)$ of size $n=20$. from a binomial distribution with 10 trials and probability of success $p$. I am asked to construct the ...
0
votes
1answer
45 views

How do I think about conditional probability in this situation?

I am trying to understand some data relating the likelihood of a positive stock return following a certain signal. The frequency of positive returns differ across datasets (over a particular time ...
4
votes
0answers
88 views

Can likelihood be changed when the prior changes?

I have a data which follows gamma distribution and want to know the uncertainty of the parameters of this data. $\text{Data} \sim \text{Gamma} (\alpha, \beta)$ Parameters $\alpha \sim \text{Gamma} ...
0
votes
0answers
27 views

Marginal Likelihood Prior

I have a model with probability matrix for a distribution of $x$, $y=\{0,1\}$, $p(x,y|w)$ where $w=[w_1,w_2,w_3,w_4]$ $p(x=0,y=0)=w_1$, $p(x=0,y=1)=w_2$ $p(x=1,y=0)=w_3$, ...
2
votes
0answers
41 views

The probability that one bernoulli process has a higher p than another?

I have two data generating processes that are independent Bernoulli processes with probabilities of success $p_A$ and $p_B$. I am taking repeated samples from these two data generating processes, so ...