a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

2
votes
2answers
23 views

Generalized log likelihood ratio test for non-nested models

I understand that if I have two models A and B and A is nested in B then, given some data, I can fit the parameters of A and B using MLE and apply the generalized log likelihood ratio test. In ...
1
vote
0answers
22 views

When does l1 regularisation give a sparse solution?

I was maximising a likelihood function, which is convex. I know that the system has a K-sparse solution. I wanted to know the conditions (or some sufficient conditions) on the likelihood function ...
0
votes
1answer
24 views

Why does maximum likelihood estimation not work in estimating signal in deterministic chaotic noise

I have few conceptual questions related to application of chaos in communications. In few application such as radar Chaotic signal reconstruction with application to noise radar system, cryptography, ...
1
vote
0answers
14 views

Simulated MLE does not exist, when trying to Bootstrap likelihood combinant

Consider this simple logistic model: We have ten $0/1$ observations $y_1,...,y_{10}.$ We model with an intercept and a predictor variable.The ten first observations have predictor value $X_i=0$, ...
0
votes
0answers
23 views

Expected value of likelihood function: Coin flips, biased and unbiased estimators

I've been reading the following (great!) book http://www.inference.phy.cam.ac.uk/itila/, which has sparked some questions about MLE. I'm comfortable with the notion that ML estimators are often ...
0
votes
0answers
21 views

Random effects models / Integrate over the random effect

I am trying to do maximum likelihood estimation and trying to see if the problem can be formulated using a random effect model. Here is the problem description: There are $100$ pairs $(N_i, D_i)$ ...
-1
votes
0answers
7 views

Maximum likelihood estimator for Weibull Parameters calculation [duplicate]

Could you please provide me an example showing the calculation of weibull parameters using MLE method?
2
votes
1answer
30 views

Comparing OLS and ML through log likelihood value

The log-like likelihood values that are computed when I do a regression (by for instance eviews), are they comparable for different estimation techniques, specifically OLS and Maximum Likelihood? My ...
0
votes
0answers
40 views

Estimation parameters for latent (unobserved) variable

Here is my problem: I have 3 variables $X,Y,Z$ : $X$ is the number of clicks we observed on an web advertisement; $Y$ is the number of time a customer do a sign-up on the website after clicking ...
2
votes
1answer
108 views

Question with MLE

I'm having some problems with this question, and was hoping someone here could help. Let $X_1,\ldots,X_2$ be $n$ determinations of a physical constant $\theta$. Consider the model $X_i = ...
0
votes
1answer
20 views

Marquardt Loglikelihood Calculation in Eviews

I paper I am trying to replicate used Eviews to estimate their state space model (by maximizing the associated maximum likelihood). They used the BHHH and Marquardt algorithms. My question is given ...
1
vote
0answers
20 views

Maximum Likelihood through a noisy channel

I have a random variable $X$, which can take $n$ values and is distributed according to multinomial $\Theta=(\theta_1, \theta_2, \cdots, \theta_n)$. I observe a random variable $Y$, where I have that ...
1
vote
1answer
32 views

Heavy-tailed distribution with closed-form ML fit from data

Which (if any) heavy-tailed distributions can we compute the maximum likelihood parameters of, given some data to fit the distribution to?
1
vote
0answers
14 views

Quasi maximum likelihood estimation versus pseudo MLE

If I'm not wrong both "quasi" and "pseudo" denote the same thing, namely the optimization under wrong distributional assumptions. Moreover I think that the terms are not restricted to the assumption ...
4
votes
1answer
85 views

Tricky question about MLE

$X_1, \ldots, X_n$ iid ~ Pois($\lambda$). Suppose, you don't know the value of each $X_i$, but you know if $X_i = 0$ or not for every i. Find MLE for $\lambda$. Does MLE always exist? I ...
0
votes
0answers
31 views

Maximum likelihood estimator for variance in two linear models

I am learning MLE's at my inference class and this is a problem I came accross. Consider two simple linear models. $y_{1j}=\alpha _1+\beta_{1}x_{1j}+\epsilon_{1j}$ and $y_{2j}=\alpha ...
2
votes
1answer
95 views

Asymptotically unbiased estimator using MLE

I am learning Maximum likelihood estimators for a inference class. And this is a problem I came across. Let $X_1,X_2,X_3,\ldots, X_n$ be a random sample with p.m.f $$p(X)=\theta(1-\theta)^x; ...
0
votes
0answers
16 views

Maximum Likelihood estimators in reation to linear models

Consider two simple linear models. $y_{1j}=\alpha _1+\beta_{1}x_{1j}+\epsilon_{1j}$ and $y_{2j}=\alpha _2+\beta_{2}x_{2j}+\epsilon_{2j}$ , $ j=1,2,...,n>2$ where $ ...
0
votes
1answer
18 views

Partial Parameter Estimation (MLE/LMMSE)

I have a basic question. MLE/LMSSE is introduced as follows: $$Y = H\theta + W$$ where $H$ is the linear model matrix, $W$ is measurement noise (let's assume it is normal so MLE = LMSSE). $\theta$ is ...
0
votes
0answers
25 views

Standard Errors of Transformed Variables

I am carrying out an MLE where some I use a log transformation on the variance parameters which are being optimized. When I calculate the standard errors (se) the se of the transformed variables is ...
0
votes
1answer
88 views

Advantage of Maximum Likelihood estimation [closed]

We can estimate unknown parameters by Least square, Least Mean Square, Blue estimators and Maximum Likelihood estimation. Q1: What is the advantage of MLE over others and when should MLE be chosen? ...
0
votes
0answers
14 views

Fisher Scoring v/s Coordinate Descent for MLE in R

R base function glm() uses Fishers Scoring for MLE, while the glmnet uses the coordinate descent method to solve the same equation ? Coordinate descent is more time efficient than Fisher Scoring as ...
0
votes
0answers
26 views

Log likelihood - understand depper

I want to use log likelihood formula to relate between two items. The formula is: LLR = 2 sum(k) (H(k) - H(rowSums(k)) - H(colSums(k))) When this is the table: ...
4
votes
1answer
165 views

Parameter estimation based on MLE estimate of another parameter

There is a situation explained below where I intend to apply MLE. The problem statement is that I am estimating a measure $X$. This measure is obtained my Maximum Likelihood estimation technique. ...
0
votes
0answers
17 views

Combining BHHH and Levenberg Marquardt

I already asked a question related to this here: When is Maximum Likelihood the same as Least Squares I know understand how Levenberg Marquardt (LM) can be applied to the objective function. In ...
4
votes
1answer
60 views

When is Maximum Likelihood the same as Least Squares

In this paper on p315: http://www.ssc.upenn.edu/~fdiebold/papers/paper55/DRAfinal.pdf They explain that they use Levenberg Marquardt (LM) (along with BHHH) to maximize the likelihood. However as I ...
0
votes
0answers
26 views

co-twin study regression modelling

I want to conduct a regression analysis on a dataset of identical twins. There is a paper by Carlin et. al. 2005 which reviews the models that are designed for twin studies and includes the Stata ...
2
votes
1answer
24 views

Scale parameter MLE scheme known but how to find according distribution PDF?

For known location, we can find the scale parameter of a normal distribution by calculating the sum of squared differences to the location, then dividing by n-1 and taking the square root. This is the ...
1
vote
0answers
49 views

GARCH(1,1) Implementation question

Could someone shed some light into the implementation of the GARCH(1,1) model contained in page6 of the following document? ...
2
votes
1answer
109 views

Why is this likelihood function equal to the noise PDF?

My professor has this slide up here: Here, $y$ is an observed signal. $H$ is a deterministic transformation, which is assumed known. $f$ is the original signal (which we dont know), and $w$ is ...
3
votes
1answer
71 views

Maximum likelihood estimation for mixed Poisson and Gaussian data

Background I've been doing a little bit of work lately on maximum likelihood estimation (MLE), for cases where the data is normally-distributed and also for cases where the data is Poisson ...
0
votes
0answers
8 views

Accounting for minimum dependent measure in data when fitting a distribution

I have what is possible a naive question. I am current comparing various models (i.e. distributions). And the comparisons do not involve different distributions but rather how the model is fed the ...
3
votes
1answer
44 views

MLE2 (in R): I can't find the right starting values

I am trying to fit several MLEs on a small dataset. I have managed to get it done for all my potential models except for this last one. This last one is actually the model that I expected to be the ...
1
vote
0answers
21 views

Power of maximum likelihood parameter estimates for a linear model

The maximum likelihood parameter estimates for the linear model where $\Pr(Y|X\beta) \sim \mathcal{N}(0,\sigma^2)$ are: $$\hat{\beta} = (X'X)^{-1}X'Y$$ How do you compute the statistical power of ...
2
votes
0answers
13 views

Estimate power for linear model with Bernoulli-distributed error

I want to estimate a linear model for a phenotype $y_i$ defined by a liability threshold model, using observable data $x_i, c_i$. $$y_i = \mu + \beta x_i + \epsilon_i$$ $$\text{Pr}(\epsilon_i | x_i, ...
1
vote
1answer
42 views

Relation between Gaussian mixture models and maximum likelihood?

I need some help understanding the relation between the maximum likelihood and Gaussian mixture models. I have seen that there is a relationship between the expectation maximization algorithms and ...
2
votes
0answers
128 views

Any alternative way to compute standard errors for maximum likelihood estimates?

I am dealing with an example stated in here. Given the same data in the above link and following a parametric bootstrap method suggested in here, I computed the standard errors for maximum ...
12
votes
1answer
218 views

Why doesn't Wilks' 1938 proof work for misspecified models?

In the famous 1938 paper ("The large-sample distribution of the likelihood ratio for testing composite hypotheses", Annals of Mathematical Statistics, 9:60-62), Samuel Wilks derived the asymptotic ...
0
votes
0answers
10 views

Properties of MLE and least squares methods for estimating parameters of ar(ma) models

I have annual data that seem to have a bimodal density function. My explaination is that there is a distinction between wet and dry years. For my work I would like to use an ar(1)-model for this. ...
0
votes
0answers
13 views

How to incorporate a noise model into a probabilistic model?

Let's say have a probabilistic model $m$ which I fit to data using maximum likelihood. Now, I would like to add a noise model $n$ which I can also fit independently using maximum likelihood. So I am ...
0
votes
0answers
26 views

Kalman Filter with Linear Constraints

A question on this topic has been asked before: Combining a linear Kalman Filter with additional linear constraints? and I checked out some of the references given: ...
1
vote
1answer
26 views

Probability density and distribution function

I am new to probability course and find concepts very confusing. I am learning maximum likelihood estimation and as a starting point for that do we find the density or the distribution? In general, ...
1
vote
0answers
30 views

Can you calculate a AIC value using the non-linear maximization (nlm) minimum value in R?

So the formula for AIC is: AIC = 2k - 2ln(L) L is the maximized value of the likelihood function. I'm modeling oxygen data in R using Non-Linear Minimization (nlm) of a maximum likelihood estimation ...
1
vote
0answers
34 views

Intuitive explanation of “integrate out random effect”

We are trying to figure out an intuitive reasoning behind integrate out the unobserved random effect. The specific formula is: $f\big(y_i|x_i;\beta, ...
3
votes
0answers
33 views

Likelihood Function for Complicated Transformations

Suppose that data X have a Normal distribution with some mean $\mu$ and some variance $\sigma^2$. However, you don't get to see X. Instead, you see $Y = g(X)$ where $g$ is a known function. Assume ...
1
vote
0answers
47 views

Help in formulation of Parameter estimation in presence of determinisitc noise by MLE

I am trying to study how to estimate parameters when the signal of interest is embedded in deterministic chaotic noise and measurement noise. A deterministic signal when operating in chaotic regime ...
1
vote
0answers
27 views

Help in Maximum likelihood etimation in presence of colored noise

I am trying to test system identification in presence of measurement noise (1) A white Gaussian noise (2) Colored noise - pink, violet. When we are estimating parameters we do so in presence of iid, ...
2
votes
1answer
83 views

What are the regularity conditions for Likelihood Ratio test

Could anyone please tell me what the regularity conditions are for the asymptotic distribution of Likelihood Ratio test? Everywhere I look, it is written 'Under the regularity conditions' or 'under ...
0
votes
1answer
30 views

Averaging Variables to create general equation

My data best fit is the 3 parameter inverse gamma distribution(thereby giving a, b, y variables with each event having a specific value for each variable) but I am not sure how to create a general ...
3
votes
1answer
103 views

Maximum likelihood estimator, exact distribution

$$\frac{e^{-(y-θx)^2/2x^2) -x/λ}}{(λ\sqrt{2πx^2})}$$ This is the joint distribution function. a)I have to find the marginal function of $X$ and $Y|X$. Now the $X\sim \text{Exp}(1/λ)$ and $Y|X\sim ...