Tagged Questions

A likelihood function gives the probability of observing the given data as a function of a parameter $\theta$.

learn more… | top users | synonyms

0
votes
0answers
28 views

jump in the sign of loglikelihood

I am trying to find the maximum of a loglikelihood function in a two dimensional parameter space__ e.g. X,Y positions are the free parameters__ by making grids in the parameter space and compute the ...
0
votes
1answer
28 views

Compare Two Logistic Regression Models

I have worked out two models to fit the data (blue) - the first (in green) is the baseline model with the intercept only. The second (red) is the model with the intercept and 2 parameters. Obviously, ...
2
votes
1answer
67 views

What would be the likelihood function of a pdf, $p(n)=1-|n|$ for $|n|<1$?

This might seem like a basic question to some but I am utterly confused by the fact that the given pdfs are not Gaussian or any other distribution commonly seen in examples. I have two hypotheses ...
3
votes
1answer
58 views

Hessian of Laplace distribution

The density of the Laplace distribution is given by: $$f(x;\mu,\sigma)=\frac{1}{2\sigma}\exp\left(-\frac{\vert x- \mu\vert}{\sigma}\right).$$ It is easy to see that this function is not ...
3
votes
3answers
120 views

Taylor's expansion on log likelihood

As far as I know, Taylors expansion works for fixed functions. I was wondering why it is justified to use it on the log likelihood. Even if we consider it as a function of only $\theta$, doesn't it ...
2
votes
1answer
58 views

Comparing OLS and ML through log likelihood value

The log-like likelihood values that are computed when I do a regression (by for instance eviews), are they comparable for different estimation techniques, specifically OLS and Maximum Likelihood? My ...
0
votes
0answers
54 views

Pseudo-likelihood and RBMs

I need to train a restricted Boltzmann machine to model the joint probability of categorical variables. For this I adapted a Bernoulli RBM to have groups of softmax units in the visible layer. The ...
1
vote
0answers
62 views

GARCH(1,1) Implementation question

Could someone shed some light into the implementation of the GARCH(1,1) model contained in page6 of the following document? ...
2
votes
1answer
245 views

Why is this likelihood function equal to the noise PDF?

My professor has this slide up here: Here, $y$ is an observed signal. $H$ is a deterministic transformation, which is assumed known. $f$ is the original signal (which we dont know), and $w$ is ...
1
vote
0answers
49 views

Can you calculate a AIC value using the non-linear maximization (nlm) minimum value in R?

So the formula for AIC is: AIC = 2k - 2ln(L) L is the maximized value of the likelihood function. I'm modeling oxygen data in R using Non-Linear Minimization (nlm) of a maximum likelihood estimation ...
1
vote
1answer
38 views

What is the likelihood of an observed standard deviation given a known normal distribution?

Given a sample X of size n from a normal distribution $N(\mu,\sigma)$ one can estimate $\sigma$ by $\hat{\sigma}$ from X. Then we know that: ...
3
votes
0answers
42 views

Likelihood Function for Complicated Transformations

Suppose that data X have a Normal distribution with some mean $\mu$ and some variance $\sigma^2$. However, you don't get to see X. Instead, you see $Y = g(X)$ where $g$ is a known function. Assume ...
1
vote
3answers
117 views

why minimize loss function instead of maximizing reward function?

Why is the "de-facto" in statistics to minimize the sum of squared errors cost function instead of maximizing some reward function like the likelihood function?
1
vote
0answers
28 views

Logistic mixed model

In the logistic mixed model ${\rm logit}(P(Y_i=1))= α + βX_i + u_i + ε_i , i=1,...,m$, when we know $u_i\sim \mathcal N(0,σu^2)$, and $ε_i\sim\mathcal N(0,σi^2)$, and if we know $σi^2$ in each area ...
0
votes
0answers
34 views

Unique or multiple maxima of log-likelihood function?

How can I find out if the log-likelihood function has only one global maximum or if it has multiple local maxima?
0
votes
2answers
49 views

how to find out the likelihood of a model given data

If i have a non-stochastic model that predicts the following dataset: [.2, .2] and the actual dataset found empirically (averaged over participants) is [.3, .3] How would I determine the ...
2
votes
1answer
103 views

Model selection for nonlinear regression of a Gaussian CDF mixture distribution

I have a number of distributions which I want to fit to a CDF that is comprised of one or more Gaussian CDFs. I was able to use weighted least squares regression to find the best fit parameters for ...
0
votes
0answers
39 views

Solving a difficult equation for a variable?

I'm trying to obtain the maximum likelihood estimate of the parameters for a model I'm building. I have constants $\sigma$, $\mu$, and $q_0$; a boolean matrix $\alpha$; and vectors $A, \beta, r, d,$ ...
3
votes
1answer
113 views

When is Likelihood Function Positive Semidefinite

This may be a very misinformed question, but I cant figure out why its not true. Here goes: According to Wikipedia and this post, the hessian of a likelihood function equals the information matrix, ...
0
votes
1answer
80 views

Likelihood score function 101

I have some trouble with score functions in likelihood calculation. I'm not good at statistics or probability, so I'm still confused on formalism and mathematical-probabilistic language. Some ...
4
votes
1answer
379 views

LogLikelihood Parameter Estimation for Linear Gaussian Kalman Filter

I have written some code that can do Kalman filtering (using a number of different Kalman-type filters [Information Filter et al.]) for Linear Gaussian State Space Analysis for an n-dimensional state ...
1
vote
0answers
116 views

Normalization of circularly-symmetric complex Gaussian distribution

I have a hard time describing my problem, but I'll try my best. It's all about the well-known zero-mean, circularly-symmetric, multivariate complex Gaussian distribution ...
2
votes
1answer
92 views

Likelihood principle: difference between weak and strong version

Does anyone understand the difference between weak likelihood principle and strong likelihood principle?
0
votes
0answers
37 views

Likelihood of a set of ranks

I'm sorry if I cannot formulate the problem precisely. Let us consider an ordered set $Q$ with $n_q$ elements, and a number of its (possibly overlapping) subsets $B_1, \dots, B_k$. For a given $i$, ...
0
votes
1answer
415 views

How to interpret Hidden Markov Model parameters (transition matrix, emission matrix, and pi values)?

I am working on channel modeling for cognitive radio using HMM. I've written a MATLAB program for forward, backward and Baum-Welch algorithm for multiple sequences. After given some random input and ...
1
vote
0answers
93 views

Difference between Maximum a posteriori and penalized likelihood

Can anyone explain to me the difference between penalised likelihood and maximum a posteriori? I read a paper where the likelihood function is $$L(\theta_1, \theta_2,\theta_3 ; x)=f(x|\theta_1, ...
9
votes
2answers
361 views

AR(1) process with heteroscedastic measurement errors

1. The problem I have some measurements of a variable $y_t$, where $t=1,2,..,n$, for which I have a distribution $f_{y_t}(y_t)$ obtained via MCMC, which for simplicity I'll assume is a gaussian of ...
1
vote
0answers
65 views

Writing out the log likelihood for a mixed effects model with multiple variance components

For a given model: $y_{ik} = \mu_k + \beta X + u_k X + b_i + \epsilon_{ik}$ where $u \sim N(0, \sigma_{u}^{2} I)$, $b ...
3
votes
1answer
182 views

AIC and likelihood function applied to a dataset

Update: First part answered. Directly jump to the second part for those wanting to answer! First Part I am trying to have a better understanding of what is a likelihood function and what is AIC. ...
3
votes
1answer
154 views

Questions about Likelihood Principle

I currently try to understand Likelihood Principle and I frankly don't get it at all. So, I will write all my question as a list, even if those might be pretty basic questions. What exactly does ...
2
votes
1answer
197 views

Why is posterior density proportional to prior density times likelihood function?

According to Bayes' theorem, $P(y|\theta)P(\theta) = P(\theta|y)P(y)$. But according to my econometric text, it says that $P(\theta|y) \propto P(y|\theta)P(\theta)$. Why is it like this? I don't get ...
10
votes
1answer
488 views

Comparing AIC of a model and its log-transformed version

The essence of my question is this: Let $Y \in \mathbb{R}^n$ be a multivariate normal random variable with mean $\mu$ and covariance matrix $\Sigma$. Let $Z := \log(Y)$, i.e. $Z_i = \log(Y_i), i ...
6
votes
3answers
392 views

Robust MCMC estimator of marginal likelihood?

I'm trying to compute the marginal likelihood for a statistical model by Monte Carlo methods: $$f(x) = \int f(x\mid\theta) \pi(\theta)\, d\theta$$ The likelihood is well behaved - smooth, ...
1
vote
1answer
461 views

Problem with the formulation of a gaussian copula likelihood function

I recently got to hear about copulas which to me sounded like a nice tool to model relationships between variables. I decided to try to implement the likelihood function for a bivariate Gaussian ...
6
votes
2answers
304 views

How to incorporate costs (into logit model) of false positive, false negative, true positive, true negative if they are different costs?

How to incorporate costs (into logit model) of false positive, false negative, true positive, true negative responses, if they are different costs ? Is it possible to do that on the level of ...
2
votes
0answers
56 views

Manipulating Binomial Distribution

Recently, I've been reading Yudi Pawitan's book, In All Likelihood. In the book, there's a section on profile likelihood; the methods explored in this section are subsequently applied to some data on ...
3
votes
0answers
262 views

Simple question about notation

In the context of likelihood-based inference, I've seen some notation concerning the parameter(s) of interest which I've found a little confusing. For example, notation such as $p_{\theta}(x)$ and ...
0
votes
1answer
311 views

Optim result highly dependent on starting value

I want to fit a standardized Student's-t distribution. The log-likelihood is given by: \begin{align*} log \mathcal{L}(\nu | l_1,...,l_n)=\sum_{i=1}^n \left( log \left( (\pi ...
0
votes
1answer
83 views

Multi-parameter log likelihood of Normal distribution for two separate samples

I have been given two sets of independent random variables distributed by two different normal distributions $X_1,...,X_n \sim N(\theta_1,1)$ and $Y_1,...,Y_m \sim N(\theta_2,1)$. And have been asked ...
7
votes
1answer
427 views

MLE/Likelihood of lognormally distributed interval

I have a variable set of responses that are expressed as an interval such as the sample below. ...
0
votes
1answer
132 views

Maximum likelihood solution in classification problem

I have a couple of questions regarding the maximum likelihood solution in a classification problem (with only two classes $C_{1}$ and $C_{2}$) Basically, I have the following likelihood function: ...
0
votes
0answers
255 views

Validity of maximising log-likelihood for maximum likelihood estimation

For reasons owing to mathematical convenience, when finding MLEs (maximum likelihood estimates), it is often the log-likelihood function---as opposed to the standard likelihood function---which is ...
3
votes
1answer
151 views

What is the name of the estimator that takes the mean of likelihood?

Let $X,Y$ be input and output (observed) continuous variables in $\mathbb{R}$. Let $\{y_1,...,y_n\}$ be the set of $n$ observations. Is there a name for the estimator $\hat x = \int_{x \in X} x ...
3
votes
1answer
254 views

Calculating the likelihood of time series data when there are missing data

I am trying to calculate the log-likelihood of some time series data given parameter sets estimated in BUGS. I can not figure out how to handle some missing values at random points in time. For the ...
2
votes
0answers
139 views

Not quite sure where the log-likelihood function comes from

A Poisson variable $Y_i$ is believed to depend on a covariate $x_i$ and it is propsed that log-linear model with a systematic component $a + bx_i$ is appropriate. In an experiment the following ...
0
votes
1answer
328 views

Log - likelihood function, why does the summation sign vanish?

I have the log-likelihood function: $$l(p_i,y_i) = \sum_{i = 1}^n \left( \ln(p_i) + y_i \ln(1 - p_i) \right) $$ And I need to calculate the maximum likelihood estimator of $p_i$. When I do this, ...
0
votes
1answer
250 views

Kolmogorov-Smirnov test in likelihood function

I want to test how well my data fits a uniform distribution and use this as one factor in a likelihood function I am constructing. Unfortunately, I have no solid basis in statistics. So far, I ...
5
votes
2answers
1k views

Why does log likelihood function for a model use SSE/n and not SSE/df?

I'm trying to find out how log-likelihood function works for linear regression. I found the formula here and here. Making some experiments with it (see code below), I was quite surprised that the ...
3
votes
1answer
147 views

KS, AD and loglike results

I'm using R to test some distribution families to my data. I've done KS, AD tests and determined the loglike. For one of the data the indications given by KS and AD do not agree with the ones given ...
4
votes
2answers
377 views

Models for calculating consumer behavior at coffee shop

I have the occasion to sit in a Starbuck's almost every day. I have noticed there are rush hours sometimes. It's like hundred of people decided to buy something at Starbucks at the very same time. ...