A likelihood function gives the probability of observing the given data as a function of a parameter $\theta$.

learn more… | top users | synonyms

1
vote
1answer
43 views

Learner in R to predict data from a complex function: if a>b then a*b else a+b

What sort of general purpose learner could learn the data generated by the following function: if a>b then a*b else a+b or something of that sort of complexity. Ideally something general enough to be ...
0
votes
0answers
8 views

Clear explanation of pseudo likehood

In generalized linear mixed model (glimmix) parameters are estimated using pseudo likelihood. I was trying to understand how this type of likelihood calculated. Thanks !!!
0
votes
1answer
24 views

Help in finding the pdf for Gaussian distribtuion of time series model

PROBLEM STATEMENT: The original data $y_t$ is a noisy version of a time series obtained from an autoregressive process excited by a deterministic non-linear signal $x_t$. The error terms $u_t$ is : ...
6
votes
1answer
148 views

Beginner learning resources : Pdf and likelihood function for non-Gaussian time series model

I am struggling with exercise problems related to blind system identification where the knowledge about the source input is assumed to be known using maximum likelihood estimation of univariate time ...
0
votes
0answers
29 views

jump in the sign of loglikelihood

I am trying to find the maximum of a loglikelihood function in a two dimensional parameter space__ e.g. X,Y positions are the free parameters__ by making grids in the parameter space and compute the ...
0
votes
1answer
31 views

Compare Two Logistic Regression Models

I have worked out two models to fit the data (blue) - the first (in green) is the baseline model with the intercept only. The second (red) is the model with the intercept and 2 parameters. Obviously, ...
2
votes
1answer
68 views

What would be the likelihood function of a pdf, $p(n)=1-|n|$ for $|n|<1$?

This might seem like a basic question to some but I am utterly confused by the fact that the given pdfs are not Gaussian or any other distribution commonly seen in examples. I have two hypotheses ...
3
votes
1answer
61 views

Hessian of Laplace distribution

The density of the Laplace distribution is given by: $$f(x;\mu,\sigma)=\frac{1}{2\sigma}\exp\left(-\frac{\vert x- \mu\vert}{\sigma}\right).$$ It is easy to see that this function is not ...
3
votes
3answers
124 views

Taylor's expansion on log likelihood

As far as I know, Taylors expansion works for fixed functions. I was wondering why it is justified to use it on the log likelihood. Even if we consider it as a function of only $\theta$, doesn't it ...
2
votes
1answer
71 views

Comparing OLS and ML through log likelihood value

The log-like likelihood values that are computed when I do a regression (by for instance eviews), are they comparable for different estimation techniques, specifically OLS and Maximum Likelihood? My ...
0
votes
0answers
59 views

Pseudo-likelihood and RBMs

I need to train a restricted Boltzmann machine to model the joint probability of categorical variables. For this I adapted a Bernoulli RBM to have groups of softmax units in the visible layer. The ...
1
vote
0answers
65 views

GARCH(1,1) Implementation question

Could someone shed some light into the implementation of the GARCH(1,1) model contained in page6 of the following document? ...
2
votes
1answer
246 views

Why is this likelihood function equal to the noise PDF?

My professor has this slide up here: Here, $y$ is an observed signal. $H$ is a deterministic transformation, which is assumed known. $f$ is the original signal (which we dont know), and $w$ is ...
1
vote
0answers
50 views

Can you calculate a AIC value using the non-linear maximization (nlm) minimum value in R?

So the formula for AIC is: AIC = 2k - 2ln(L) L is the maximized value of the likelihood function. I'm modeling oxygen data in R using Non-Linear Minimization (nlm) of a maximum likelihood estimation ...
1
vote
1answer
38 views

What is the likelihood of an observed standard deviation given a known normal distribution?

Given a sample X of size n from a normal distribution $N(\mu,\sigma)$ one can estimate $\sigma$ by $\hat{\sigma}$ from X. Then we know that: ...
3
votes
0answers
43 views

Likelihood Function for Complicated Transformations

Suppose that data X have a Normal distribution with some mean $\mu$ and some variance $\sigma^2$. However, you don't get to see X. Instead, you see $Y = g(X)$ where $g$ is a known function. Assume ...
1
vote
3answers
131 views

why minimize loss function instead of maximizing reward function?

Why is the "de-facto" in statistics to minimize the sum of squared errors cost function instead of maximizing some reward function like the likelihood function?
1
vote
0answers
28 views

Logistic mixed model

In the logistic mixed model ${\rm logit}(P(Y_i=1))= α + βX_i + u_i + ε_i , i=1,...,m$, when we know $u_i\sim \mathcal N(0,σu^2)$, and $ε_i\sim\mathcal N(0,σi^2)$, and if we know $σi^2$ in each area ...
0
votes
0answers
34 views

Unique or multiple maxima of log-likelihood function?

How can I find out if the log-likelihood function has only one global maximum or if it has multiple local maxima?
0
votes
2answers
50 views

how to find out the likelihood of a model given data

If i have a non-stochastic model that predicts the following dataset: [.2, .2] and the actual dataset found empirically (averaged over participants) is [.3, .3] How would I determine the ...
2
votes
1answer
122 views

Model selection for nonlinear regression of a Gaussian CDF mixture distribution

I have a number of distributions which I want to fit to a CDF that is comprised of one or more Gaussian CDFs. I was able to use weighted least squares regression to find the best fit parameters for ...
0
votes
0answers
39 views

Solving a difficult equation for a variable?

I'm trying to obtain the maximum likelihood estimate of the parameters for a model I'm building. I have constants $\sigma$, $\mu$, and $q_0$; a boolean matrix $\alpha$; and vectors $A, \beta, r, d,$ ...
3
votes
1answer
123 views

When is Likelihood Function Positive Semidefinite

This may be a very misinformed question, but I cant figure out why its not true. Here goes: According to Wikipedia and this post, the hessian of a likelihood function equals the information matrix, ...
0
votes
1answer
81 views

Likelihood score function 101

I have some trouble with score functions in likelihood calculation. I'm not good at statistics or probability, so I'm still confused on formalism and mathematical-probabilistic language. Some ...
4
votes
1answer
466 views

LogLikelihood Parameter Estimation for Linear Gaussian Kalman Filter

I have written some code that can do Kalman filtering (using a number of different Kalman-type filters [Information Filter et al.]) for Linear Gaussian State Space Analysis for an n-dimensional state ...
1
vote
0answers
127 views

Normalization of circularly-symmetric complex Gaussian distribution

I have a hard time describing my problem, but I'll try my best. It's all about the well-known zero-mean, circularly-symmetric, multivariate complex Gaussian distribution ...
2
votes
1answer
102 views

Likelihood principle: difference between weak and strong version

Does anyone understand the difference between weak likelihood principle and strong likelihood principle?
0
votes
0answers
39 views

Likelihood of a set of ranks

I'm sorry if I cannot formulate the problem precisely. Let us consider an ordered set $Q$ with $n_q$ elements, and a number of its (possibly overlapping) subsets $B_1, \dots, B_k$. For a given $i$, ...
0
votes
1answer
454 views

How to interpret Hidden Markov Model parameters (transition matrix, emission matrix, and pi values)?

I am working on channel modeling for cognitive radio using HMM. I've written a MATLAB program for forward, backward and Baum-Welch algorithm for multiple sequences. After given some random input and ...
1
vote
0answers
93 views

Difference between Maximum a posteriori and penalized likelihood

Can anyone explain to me the difference between penalised likelihood and maximum a posteriori? I read a paper where the likelihood function is $$L(\theta_1, \theta_2,\theta_3 ; x)=f(x|\theta_1, ...
9
votes
2answers
372 views

AR(1) process with heteroscedastic measurement errors

1. The problem I have some measurements of a variable $y_t$, where $t=1,2,..,n$, for which I have a distribution $f_{y_t}(y_t)$ obtained via MCMC, which for simplicity I'll assume is a gaussian of ...
1
vote
0answers
65 views

Writing out the log likelihood for a mixed effects model with multiple variance components

For a given model: $y_{ik} = \mu_k + \beta X + u_k X + b_i + \epsilon_{ik}$ where $u \sim N(0, \sigma_{u}^{2} I)$, $b ...
3
votes
1answer
190 views

AIC and likelihood function applied to a dataset

Update: First part answered. Directly jump to the second part for those wanting to answer! First Part I am trying to have a better understanding of what is a likelihood function and what is AIC. ...
3
votes
1answer
172 views

Questions about Likelihood Principle

I currently try to understand Likelihood Principle and I frankly don't get it at all. So, I will write all my question as a list, even if those might be pretty basic questions. What exactly does ...
2
votes
1answer
214 views

Why is posterior density proportional to prior density times likelihood function?

According to Bayes' theorem, $P(y|\theta)P(\theta) = P(\theta|y)P(y)$. But according to my econometric text, it says that $P(\theta|y) \propto P(y|\theta)P(\theta)$. Why is it like this? I don't get ...
10
votes
1answer
532 views

Comparing AIC of a model and its log-transformed version

The essence of my question is this: Let $Y \in \mathbb{R}^n$ be a multivariate normal random variable with mean $\mu$ and covariance matrix $\Sigma$. Let $Z := \log(Y)$, i.e. $Z_i = \log(Y_i), i ...
6
votes
3answers
411 views

Robust MCMC estimator of marginal likelihood?

I'm trying to compute the marginal likelihood for a statistical model by Monte Carlo methods: $$f(x) = \int f(x\mid\theta) \pi(\theta)\, d\theta$$ The likelihood is well behaved - smooth, ...
1
vote
1answer
478 views

Problem with the formulation of a gaussian copula likelihood function

I recently got to hear about copulas which to me sounded like a nice tool to model relationships between variables. I decided to try to implement the likelihood function for a bivariate Gaussian ...
6
votes
2answers
319 views

How to incorporate costs (into logit model) of false positive, false negative, true positive, true negative if they are different costs?

How to incorporate costs (into logit model) of false positive, false negative, true positive, true negative responses, if they are different costs ? Is it possible to do that on the level of ...
2
votes
0answers
57 views

Manipulating Binomial Distribution

Recently, I've been reading Yudi Pawitan's book, In All Likelihood. In the book, there's a section on profile likelihood; the methods explored in this section are subsequently applied to some data on ...
3
votes
0answers
273 views

Simple question about notation

In the context of likelihood-based inference, I've seen some notation concerning the parameter(s) of interest which I've found a little confusing. For example, notation such as $p_{\theta}(x)$ and ...
0
votes
1answer
338 views

Optim result highly dependent on starting value

I want to fit a standardized Student's-t distribution. The log-likelihood is given by: \begin{align*} log \mathcal{L}(\nu | l_1,...,l_n)=\sum_{i=1}^n \left( log \left( (\pi ...
0
votes
1answer
86 views

Multi-parameter log likelihood of Normal distribution for two separate samples

I have been given two sets of independent random variables distributed by two different normal distributions $X_1,...,X_n \sim N(\theta_1,1)$ and $Y_1,...,Y_m \sim N(\theta_2,1)$. And have been asked ...
7
votes
1answer
464 views

MLE/Likelihood of lognormally distributed interval

I have a variable set of responses that are expressed as an interval such as the sample below. ...
0
votes
1answer
138 views

Maximum likelihood solution in classification problem

I have a couple of questions regarding the maximum likelihood solution in a classification problem (with only two classes $C_{1}$ and $C_{2}$) Basically, I have the following likelihood function: ...
0
votes
0answers
268 views

Validity of maximising log-likelihood for maximum likelihood estimation

For reasons owing to mathematical convenience, when finding MLEs (maximum likelihood estimates), it is often the log-likelihood function---as opposed to the standard likelihood function---which is ...
3
votes
1answer
152 views

What is the name of the estimator that takes the mean of likelihood?

Let $X,Y$ be input and output (observed) continuous variables in $\mathbb{R}$. Let $\{y_1,...,y_n\}$ be the set of $n$ observations. Is there a name for the estimator $\hat x = \int_{x \in X} x ...
3
votes
1answer
263 views

Calculating the likelihood of time series data when there are missing data

I am trying to calculate the log-likelihood of some time series data given parameter sets estimated in BUGS. I can not figure out how to handle some missing values at random points in time. For the ...
2
votes
0answers
139 views

Not quite sure where the log-likelihood function comes from

A Poisson variable $Y_i$ is believed to depend on a covariate $x_i$ and it is propsed that log-linear model with a systematic component $a + bx_i$ is appropriate. In an experiment the following ...
0
votes
1answer
340 views

Log - likelihood function, why does the summation sign vanish?

I have the log-likelihood function: $$l(p_i,y_i) = \sum_{i = 1}^n \left( \ln(p_i) + y_i \ln(1 - p_i) \right) $$ And I need to calculate the maximum likelihood estimator of $p_i$. When I do this, ...