A likelihood function gives the probability of observing the given data as a function of a parameter $\theta$.

learn more… | top users | synonyms

1
vote
3answers
35 views

why minimize loss function instead of maximizing reward function?

Why is the "de-facto" in statistics to minimize the sum of squared errors cost function instead of maximizing some reward function like the likelihood function?
1
vote
0answers
17 views

Logistic mixed model

In the logistic mixed model ${\rm logit}(P(Y_i=1))= α + βX_i + u_i + ε_i , i=1,...,m$, when we know $u_i\sim \mathcal N(0,σu^2)$, and $ε_i\sim\mathcal N(0,σi^2)$, and if we know $σi^2$ in each area ...
0
votes
0answers
26 views

Unique or multiple maxima of log-likelihood function?

How can I find out if the log-likelihood function has only one global maximum or if it has multiple local maxima?
0
votes
2answers
37 views

how to find out the likelihood of a model given data

If i have a non-stochastic model that predicts the following dataset: [.2, .2] and the actual dataset found empirically (averaged over participants) is [.3, .3] How would I determine the ...
1
vote
0answers
24 views

Model selection for nonlinear regression of a Gaussian CDF mixture distribution

I have a number of distributions which I want to fit to a CDF that is comprised of one or more Gaussian CDFs. I was able to use weighted least squares regression to find the best fit parameters for ...
0
votes
0answers
25 views

Solving a difficult equation for a variable?

I'm trying to obtain the maximum likelihood estimate of the parameters for a model I'm building. I have constants $\sigma$, $\mu$, and $q_0$; a boolean matrix $\alpha$; and vectors $A, \beta, r, d,$ ...
3
votes
1answer
55 views

When is Likelihood Function Positive Semidefinite

This may be a very misinformed question, but I cant figure out why its not true. Here goes: According to Wikipedia and this post, the hessian of a likelihood function equals the information matrix, ...
0
votes
1answer
53 views

Likelihood score function 101

I have some trouble with score functions in likelihood calculation. I'm not good at statistics or probability, so I'm still confused on formalism and mathematical-probabilistic language. Some ...
4
votes
1answer
79 views

LogLikelihood Parameter Estimation for Linear Gaussian Kalman Filter

I have written some code that can do Kalman filtering (using a number of different Kalman-type filters [Information Filter et al.]) for Linear Gaussian State Space Analysis for an n-dimensional state ...
1
vote
0answers
36 views

Normalization of circularly-symmetric complex Gaussian distribution

I have a hard time describing my problem, but I'll try my best. It's all about the well-known zero-mean, circularly-symmetric, multivariate complex Gaussian distribution ...
2
votes
0answers
62 views

Likelihood principle: difference between weak and strong version

Does anyone understand the difference between weak likelihood principle and strong likelihood principle?
0
votes
0answers
33 views

Likelihood of a set of ranks

I'm sorry if I cannot formulate the problem precisely. Let us consider an ordered set $Q$ with $n_q$ elements, and a number of its (possibly overlapping) subsets $B_1, \dots, B_k$. For a given $i$, ...
0
votes
0answers
67 views

Log Likelihood and Score Vector of Conditional Grouped Continuous Model (CGCM) when the data is given

I am interested in the relationship between weight Y_i at mating and the number Z_i of lambs born in a flock of n=25 female sheep.Assume Z_i can be 0(no lambs born) 1 and 2, then I know that Y_i* ...
0
votes
1answer
242 views

How to interpret Hidden Markov Model parameters (transition matrix, emission matrix, and pi values)?

I am working on channel modeling for cognitive radio using HMM. I've written a MATLAB program for forward, backward and Baum-Welch algorithm for multiple sequences. After given some random input and ...
1
vote
0answers
76 views

Difference between Maximum a posteriori and penalized likelihood

Can anyone explain to me the difference between penalised likelihood and maximum a posteriori? I read a paper where the likelihood function is $$L(\theta_1, \theta_2,\theta_3 ; x)=f(x|\theta_1, ...
8
votes
2answers
265 views

AR(1) process with heteroscedastic measurement errors

1. The problem I have some measurements of a variable $y_t$, where $t=1,2,..,n$, for which I have a distribution $f_{y_t}(y_t)$ obtained via MCMC, which for simplicity I'll assume is a gaussian of ...
0
votes
0answers
53 views

Writing out the log likelihood for a mixed effects model with multiple variance components

For a given model: $y_{ik} = \mu_k + \beta X + u_k X + b_i + \epsilon_{ik}$ where $u \sim N(0, \sigma_{u}^{2} I)$, $b ...
3
votes
1answer
133 views

AIC and likelihood function applied to a dataset

Update: First part answered. Directly jump to the second part for those wanting to answer! First Part I am trying to have a better understanding of what is a likelihood function and what is AIC. ...
0
votes
0answers
105 views

Output of Baum-Welch algorithm and clustering of HMM

I have trouble understanding the output of Baum-Welch algorithm in the context of clustering of time series of unequal length using HMM. Suppose I have N sequences with length L_i. An article that ...
3
votes
1answer
114 views

Questions about Likelihood Principle

I currently try to understand Likelihood Principle and I frankly don't get it at all. So, I will write all my question as a list, even if those might be pretty basic questions. What exactly does ...
1
vote
1answer
100 views

Why is posterior density proportional to prior density times likelihood function?

According to Bayes' theorem, $P(y|\theta)P(\theta) = P(\theta|y)P(y)$. But according to my econometric text, it says that $P(\theta|y) \propto P(y|\theta)P(\theta)$. Why is it like this? I don't get ...
10
votes
1answer
312 views

Comparing AIC of a model and its log-transformed version

The essence of my question is this: Let $Y \in \mathbb{R}^n$ be a multivariate normal random variable with mean $\mu$ and covariance matrix $\Sigma$. Let $Z := \log(Y)$, i.e. $Z_i = \log(Y_i), i ...
6
votes
3answers
231 views

Robust MCMC estimator of marginal likelihood?

I'm trying to compute the marginal likelihood for a statistical model by Monte Carlo methods: $$f(x) = \int f(x\mid\theta) \pi(\theta)\, d\theta$$ The likelihood is well behaved - smooth, ...
1
vote
1answer
300 views

Problem with the formulation of a gaussian copula likelihood function

I recently got to hear about copulas which to me sounded like a nice tool to model relationships between variables. I decided to try to implement the likelihood function for a bivariate Gaussian ...
6
votes
2answers
186 views

How to incorporate costs (into logit model) of false positive, false negative, true positive, true negative if they are different costs?

How to incorporate costs (into logit model) of false positive, false negative, true positive, true negative responses, if they are different costs ? Is it possible to do that on the level of ...
2
votes
0answers
49 views

Manipulating Binomial Distribution

Recently, I've been reading Yudi Pawitan's book, In All Likelihood. In the book, there's a section on profile likelihood; the methods explored in this section are subsequently applied to some data on ...
3
votes
0answers
127 views

Simple question about notation

In the context of likelihood-based inference, I've seen some notation concerning the parameter(s) of interest which I've found a little confusing. For example, notation such as $p_{\theta}(x)$ and ...
0
votes
1answer
147 views

Optim result highly dependent on starting value

I want to fit a standardized Student's-t distribution. The log-likelihood is given by: \begin{align*} log \mathcal{L}(\nu | l_1,...,l_n)=\sum_{i=1}^n \left( log \left( (\pi ...
0
votes
1answer
74 views

Multi-parameter log likelihood of Normal distribution for two separate samples

I have been given two sets of independent random variables distributed by two different normal distributions $X_1,...,X_n \sim N(\theta_1,1)$ and $Y_1,...,Y_m \sim N(\theta_2,1)$. And have been asked ...
7
votes
1answer
258 views

MLE/Likelihood of lognormally distributed interval

I have a variable set of responses that are expressed as an interval such as the sample below. ...
0
votes
1answer
106 views

Maximum likelihood solution in classification problem

I have a couple of questions regarding the maximum likelihood solution in a classification problem (with only two classes $C_{1}$ and $C_{2}$) Basically, I have the following likelihood function: ...
0
votes
0answers
149 views

Validity of maximising log-likelihood for maximum likelihood estimation

For reasons owing to mathematical convenience, when finding MLEs (maximum likelihood estimates), it is often the log-likelihood function---as opposed to the standard likelihood function---which is ...
3
votes
1answer
143 views

What is the name of the estimator that takes the mean of likelihood?

Let $X,Y$ be input and output (observed) continuous variables in $\mathbb{R}$. Let $\{y_1,...,y_n\}$ be the set of $n$ observations. Is there a name for the estimator $\hat x = \int_{x \in X} x ...
3
votes
1answer
190 views

Calculating the likelihood of time series data when there are missing data

I am trying to calculate the log-likelihood of some time series data given parameter sets estimated in BUGS. I can not figure out how to handle some missing values at random points in time. For the ...
2
votes
0answers
132 views

Not quite sure where the log-likelihood function comes from

A Poisson variable $Y_i$ is believed to depend on a covariate $x_i$ and it is propsed that log-linear model with a systematic component $a + bx_i$ is appropriate. In an experiment the following ...
0
votes
1answer
246 views

Log - likelihood function, why does the summation sign vanish?

I have the log-likelihood function: $$l(p_i,y_i) = \sum_{i = 1}^n \left( \ln(p_i) + y_i \ln(1 - p_i) \right) $$ And I need to calculate the maximum likelihood estimator of $p_i$. When I do this, ...
0
votes
1answer
198 views

Kolmogorov-Smirnov test in likelihood function

I want to test how well my data fits a uniform distribution and use this as one factor in a likelihood function I am constructing. Unfortunately, I have no solid basis in statistics. So far, I ...
5
votes
2answers
693 views

Why does log likelihood function for a model use SSE/n and not SSE/df?

I'm trying to find out how log-likelihood function works for linear regression. I found the formula here and here. Making some experiments with it (see code below), I was quite surprised that the ...
3
votes
1answer
128 views

KS, AD and loglike results

I'm using R to test some distribution families to my data. I've done KS, AD tests and determined the loglike. For one of the data the indications given by KS and AD do not agree with the ones given ...
4
votes
2answers
321 views

Models for calculating consumer behavior at coffee shop

I have the occasion to sit in a Starbuck's almost every day. I have noticed there are rush hours sometimes. It's like hundred of people decided to buy something at Starbucks at the very same time. ...
5
votes
3answers
289 views

MCMC to handle flat likelihood issues

I have a quite flat likelihood leading Metropolis-Hastings sampler to move through the parameter space very irregularly, i.e. no convergence can be achieved no matter what the parameters of proposal ...
2
votes
0answers
104 views

Likelihood for Poisson data

In my book, it says: Independent random variables $X_1, X_2, \dots, X_n$ are modeled by a Poisson distribution with mean $\lambda > 0$. The likelihood for $\lambda$ based on data ...
3
votes
1answer
752 views

Is it okay to compare fitted distributions with the AIC?

suppose I have a data set $x_1, \ldots, x_n$ and I would fit a normal, an exponential and a uniform distribution to them. The fitting function spits out a bunch of goodness-of-fit statistics, e.g. the ...
11
votes
2answers
2k views

What is the reason that a likelihood function is not a pdf?

What is the reason that a likelihood function is not a pdf (probability density function)?
17
votes
3answers
732 views

What are some illustrative applications of empirical likelihood?

I have heard of Owen's empirical likelihood, but until recently paid it no heed until I came across it in a paper of interest (Mengersen et al. 2012). In my efforts to understand it, I have gleaned ...
5
votes
2answers
204 views

Gaussian Process goodness of fit

Let's say I got a Gaussian Process model $M$ based on some training data. Now I get a stream of sample data of a certain batch size coming in. The GP does not model a time series, but it's trying to ...
3
votes
1answer
196 views

Likelihood based model selection

Let's say I got a set of models $M = \{M_1, M_2, \dots M_n\}$. Now say I got some data $x$ and I would like to know, which model represents the data best. I know how to calculate the likelihood ...
16
votes
4answers
848 views

How to rigorously define the likelihood?

The likelihood could be defined by several ways, for instance : the function $L$ from $\Theta\times{\cal X}$ which maps $(\theta,x)$ to $L(\theta \mid x)$ the random function $L(\cdot \mid X)$ we ...
0
votes
1answer
214 views

Maximizing: likelihood vs likelihood ratio

Say I have an observed data set ($n_i$) and I want to obtain the best fit out of 10 data sets produced by a model dependent on a single parameter $a$ ($m_i(a)\;a=1..10$). Suppose I use a Poisson ...
23
votes
6answers
1k views

Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?

If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to ...