Given a random variable $X$ which arise from a parameterized distribution $F(X;θ)$, the likelihood is defined as the probability of observed data as a function of $θ: \text{L}(θ)=\text{P}(θ;X=x)$

learn more… | top users | synonyms (1)

0
votes
0answers
15 views

Expected ratio given a list of observed ratios

This might be a strange question (or not). Given a case where: I have a box of objects. Within the box are 3 types of objects (A, B, C) I have several observations of the contents of the box made ...
1
vote
1answer
29 views

Learning just a decoder (autoencoder without encoder)

I am trying to do something quite unusual: learning a latent representation of some data just by optimizing a decoder. Basically, a probabilistic model of a neural network autoencoder without the ...
2
votes
1answer
25 views

Need help understand likelihood in survival analysis

I really appreciate it if you could help me understand the likelihood in a survival model with a time varying covariate. To be more clear, let's first start with a survival analysis with fixed ...
0
votes
0answers
21 views

How to measure clustering algorithm performance? [duplicate]

For supervised learning, both regression and classification have ground truth. The model performance can be measured against ground truth. For example, $R^2$ in regression or accuracy (0-1) in ...
3
votes
0answers
50 views

Transforming observed Fisher information

Let $x_1$,$x_2$,....$x_n$ be a random sample from the exponential distribution with p.d.f.: $$f(x;\lambda)=\lambda e^{-\lambda x}\;\;\;x\geq0\;\;\;\lambda>0$$ The log-likelihood is: $$l(\lambda)=...
1
vote
0answers
22 views

Appropriate logistic regression model and log- likelihood

Are two simple independent causal samples from two distributions of bernoulli medium respectively: $i=1,...,10$ $ π_1 = e^{B_1} / (1 + e^{B_1})$ $i=1,...,10$ $ π_2 = e^{B_1 + B_2} / (1 + e^{B_1 +...
7
votes
1answer
123 views

Strange likelihood trace from MCMC chain

I've got a model that goes: Single parameter -> Complex likelihood function -> Log-likelihood. I executed an MCMC chain (using pymc) and plotted the trace of ...
0
votes
0answers
10 views

Proportional Hazards Model With Multiple Treatments?

Is there some way to do a proportional hazards model where the "treatment" occurs multiple times? Here's an example: Let's say you are running a Ford dealership where you sell only one car... The ...
0
votes
0answers
51 views

Recalculate log-likelihood from a R lmer model with one random effect

I'm trying to recalculate the REML log-likelihood given by the logLik function from a linear mixed model with one random effect. I use the ...
2
votes
2answers
178 views

How to evaluate how well some samples come from some given distributions?

I have a complicated procedure that, given an input $x$, outputs several random samples $S_i(x)$ for $i\in\{1,\dots,k\}$ (each sample consists of $n_i$ points). Each sample $S_i(x)$ follows an unknown ...
1
vote
1answer
75 views

Likelihood of observations for HMM with continuous state and observation distributions

I have a "real" and estimated HMM model given as $(\pi,\mu, \nu)$ and $(\pi^{\text{est}},\mu^{\text{est}}, \nu^{\text{est}})$, where $\pi$ is initial state distribution of Markov chain, $\mu$ is state ...
0
votes
1answer
14 views

inferential structure determination

I'm trying to get my head around Bayesian inference and the difference between the posterior and likelihood. Going off the back of these answers, I'm under the impression that the posterior is $P(data|...
2
votes
1answer
19 views

How to interpret marginal likelihood definition?

Say we have a Beta-Bernoulli model where $X_i$ are i.i.d. Bernoulli variables, $p(X_i=1)=\theta$, and $\theta\sim\operatorname{Beta}(\alpha,\beta)$. The marginal likelihood is defined as $$ p(X)=\...
0
votes
0answers
36 views

assessing confidence of simple likelihood ratio test

I have a Gaussian mixture model of two normal distributions, which are scaled relatively to each other by a factor $\alpha$. I observe the data vector $x$ with values from $x_1$ to $x_n$, which is ...
1
vote
0answers
18 views

Composite-likelihood ratio test

I am interested in a number of articles (such as Kim and Stephan 2002 and follow-up articles) that use composite likelihood ratio tests to infer selection pressure on linked (phased) genetic data. I ...
0
votes
1answer
33 views

How do u find the log likelihood function of Y^1/2 = XB + u?

Let $Y_i>0$ for all $Y_i$ and $u_i \sim N(0,\sigma^2)$. Where the $u_i$ are iid. How do you find the log-likelihood function of $Y_i ^{1/2} = B X_i + u_i$? I am confused because the dependent ...
0
votes
0answers
4 views

What is the effect of using the partial or maximum likelihood upon estimation sample size?

I'm running a series of different models on some longitudinal data with repeated waves. I've noticed that a Cox proportional hazards with time-varying covariates drops entire participants even where ...
0
votes
1answer
84 views

Distribution of posterior mean from different datasets

This question has originated from this question. Suppose we have the following simple setup, for $i = 1, \dots, n$ $$y_i \mid \mu \sim N( \mu, 1) \text{ and } \mu \sim N(0,1). $$ Then due to the ...
2
votes
1answer
35 views

The likelihood function of related samples

I know how to get the likelihood function for two independent samples from normal distribution. What will happen to the likelihood function if the samples are related? What will be its form?
0
votes
1answer
13 views

Bayesian estimation for the distribution of the results of an experiment when the cardinality of the result set is unknown

Suppose I have an experiment X with mutually exclusive outcomes from a set S. My goal is to determine the probability distribution for S. The problem is that I do not know how many elements are in S ...
0
votes
0answers
25 views

Log-likelihood function for Multinomial Logistic Regression

Can you help me to calculate log-likelihood function in R code for multinomial logistic regression, if I have X as design matrix(N x K+1), Y is dependent variable matrix(N x J-1), and B as coefficient ...
3
votes
0answers
42 views

Maximum Likelihood of n

Let $X=\{x_1,x_2,...x_m\}$ is a rv with density $f(x)=cx^{2n}$ if $-1<x<1$, $0$ therwise and $n \in \{ 1,2,3,4\}$. Find fe mle of n. I tried to find the Likelihood function : $(n+1/2)^m\...
0
votes
0answers
31 views

First Derivative of Log-Likelihood of Multinomial Logistic Regression

I want to create my own Multinomial Logistic Regression function in R and get stuck on the first derivative of Log likelihood matrix. The matrix have (J-1)(K+1) rows and (J-1)(K+1) columns, where K is ...
1
vote
0answers
38 views

Is there any point to Reverse Engineering the Fisher Information Matrix from an Inverse Covariance Matrix?

Would there be any advantage in deriving a Fisher Information Matrix backwards from an inverse covariance matrix? I've discovered that this is much easier to do on the SQL Server platform I use than ...
0
votes
0answers
18 views

Using likelihood for event correlation in Bayesian network

Assume we have a simple Bayesian network, each event (alarm propagation) is a binary vector. Each node can trigger an alarm independently, but alarm propagates in the network with the probabilities of ...
11
votes
1answer
267 views

Computation of the marginal likelihood from MCMC samples

This is a recurring question (see this post, this post and this post), but I have a different spin. Suppose I have a bunch of samples from a generic MCMC sampler. For each sample $\theta$, I know the ...
0
votes
0answers
26 views

Using likelihood to compare model accuracy on two different data sets

I have a model that produces conditional probabilities, $p(Y|X)$, where $Y$ is either 0 or 1, and $X$ is just some random variable. I have two different data sets $Z_1, Z_2$ consisting of pairs of ...
2
votes
2answers
39 views

What is the difference and relationship between posterior distribution function and likelihood function in MCMC?

I am learning MCMC in class, and I encounter one question about the relationship between posterior probability and likelihood function. In our lecture, the professor asked us to take samples from ...
2
votes
1answer
24 views

How to estimate the likelihood that a distribution is not random

Not sure I phrase the title right, but here's by question in a nutshell. Say you have a box of balls with different colors. Let's say there are $N$ balls of $k$ different colors, and that the ...
0
votes
0answers
32 views

Calculating likelihood in R

I want to "by hand" calculate a Bayes factor in a simple case. I'm sure I'm off somewhere in my calculation of likelihood of data under H1. I think I don't "scale" my likelihood under H1 correctly. ...
0
votes
0answers
13 views

likelihood equation for a simple model

Consider a simple model $y_{i}=\alpha_{i}+e_{i}$, for $i=1,...,T$ with $\alpha_{i} \sim \mathcal{N}(0,\tau^2)$ and $e_{i} \sim \mathcal{N}(0,\sigma^2)$. Could anyone please help in writing a ...
0
votes
0answers
22 views

compute likelihood

How can I compute the likelihood or posterior probability on data when the distribution function is continuous. For example the dist. function is a Gaussian and I have one hundred data points. Now how ...
0
votes
0answers
28 views

Likelihood ratio test of two normals

If $(X_1, . . .,X_m)$ is a sample of $\mathcal{N}(\mu_1,\sigma_1^2)$ random variables and $(Y_1,. . .Y_n)$ is a sample of $\mathcal{N}(\mu_2,\sigma_2^2)$ random variables. We have no known parameters, ...
0
votes
1answer
65 views

Log likelihood in EM Algorithm

I try understand the log likelihood in weka. I read about that is a probabilistic metric, but i cant understand, if is better when have low value or high value? How i can get the likelihood value, ...
1
vote
0answers
33 views

Maximum likelihood of coin toss of different type?

I was self-studying EM (Expectation Maximization) algorithm, where I came across this example given by the paper. In this paper, there are two types of coins A, B with unknown parameters $θ_A$ and $...
1
vote
1answer
26 views

Jeffreys' invariance principle - A doubt on the one-dim case

In this wiki link concerning a change of parameterization of a likelihood $L$ from $\theta$ to $\varphi$, why can the $d\theta/d \varphi$ go inside the expectation? $$\eqalign{ p(\varphi) &= p(\...
0
votes
1answer
72 views

Is likelihood ratio test a nonparametric test?

I have a few doubts about the Likelihood ratio test. I understand that we compute a p-value based on the ratio of likelihoods between two models. I am wondering: Is the likelihood ratio test, a ...
0
votes
0answers
23 views

Likelihood based on sample mean from Gaussian

If we know that mean of sample is 4, and that sample size is 3, and that samples are taken from Gaussian distribution with variance 1, i.e. $N(\mu, 1)$, what can we tell about distribution's mean?
1
vote
0answers
19 views

Is it safe to assume that $\ln{L} = -\frac{\chi^2}{2}$?

I have written a code that fits a 2D model image to some observed 2D image. One of the optimization libraries I use (MultiNest) in order to do that requires to be ...
0
votes
0answers
29 views

Which one is more robust ? CI with maximum likelihood method or by bootstrap?

I have a question about confidence interval calculation. I am wondering is calculatin the CI with the function confint is more robust with the likelihood method or the boostrap method ? thanks !
0
votes
0answers
18 views

Which likelihood function for ML estimate of continuous variable?

Let's say my data consists of the counts of the following words in a text. I can then estimate the probabilities of occurrences of these words using MLE: ...
0
votes
0answers
9 views

Profile Likelihood Algorithm Assumptions

Given a statistical model $$\{p_\theta(y) : \theta \in \Theta \}$$ and same data $y$. The log likelihood function is then $l(\theta)=\log(p_\theta(y))$. For a one parameter hypothesis $\theta_j=\beta$ ...
1
vote
1answer
41 views

insurance exponential

i would like to ask for help with following example, i know how to derive next steps that I am not even showing here, but cant derive the loglike function, thank you.
3
votes
1answer
55 views

Help explaining (offering intuition/examples of) the score function and fisher's information to students

Next week I will teach my students the score function and its variance (i.e.: fisher information). I am looking for way(s) to illustrate these concepts so to help my students understand them (and not ...
3
votes
0answers
19 views

Properties of Average Multinomial Likelihood

I am trying to understand the Kullback-Leibler Information: I read in http://arxiv.org/pdf/1404.2000v1.pdf the following: Ideally, we want the probability to be invariant to the number of ...
4
votes
1answer
72 views

Posterior distribution of parameter

I am looking at pg. 257 of the textbook "Statistical computing with R" by Maria L. Rizzo, where an example is given of a simple model of stock returns: We have 5 stocks, and at the end of a year of ...
3
votes
1answer
81 views

maximum likelihood two unknown parameters

I have a problem. I have to find the mle's (maximum likelihood estimators) for $a$ and $b$ whose density is $f(x|a,b)$, when I derive the $l(b|x)$ (The log-likelihood function) to find the maximum, $b$...
0
votes
1answer
57 views

Likelihood function and MLE

Please let me know how to find the likelihood function and MLE when the data have density $f(x; \theta ) = ( \theta +1) x^\theta$. I have tried using the general formula however not sure how to ...
0
votes
1answer
45 views

EM algorithm with dependent observations

I am trying to implement an EM algorithm for dependent observations. Specifically, I am dealing with families where the hidden variables $Z$ of the children are dependent on the hidden variables of ...
2
votes
1answer
28 views

Generalized likelihood test question

This example appears in Rice's stats book. In the first rectangle, why do we have to maximize the denominator? and why do we use MLE as the denominator to maximize it? I know that MLE best reduces ...