a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

0
votes
0answers
17 views

Which groups are significantly different from within a random effect (using lmer in R)

I am using a mixed effects model as created here (using dummy data for now) in this R script ...
2
votes
1answer
35 views

MLE vs MAP vs conditional MLE with regards to logistic regression

We have some set of iid RV's: $(X_i, Y_i), \; i=1,\ldots n$. We believe each to be distributed as $P(X_i, Y_i | \theta)$. So that $$ P(X,Y | \theta) = \prod_i P_i(X_i, Y_i | \theta) $$ Now using ...
4
votes
2answers
42 views

ML estimation of parameters that do not completely specify the model

I was wondering how ML is defined when the parameter does not completely specify the model. More concretely, suppose $X_1, X_2, \cdots, X_n$ are drawn iid such that $P(X_1=i)=\theta_i$, $ 1 \leq i ...
0
votes
0answers
2 views

Segregation Analysis for predicting age-specific cancer risk

I am relatively new to the worlds of bioinformatics and genetics research. I have been tasked with presenting to my lab the potential value of a paper that uses Complex Segregation Analysis for a risk ...
0
votes
1answer
21 views

Standard errors of the MLEs

Can anybody tell me how to find numerical values for standard errors of the MLEs of Weibull distribution using the uncensored real data set on the breaking stress of carbon fibres (in Gba) reported by ...
1
vote
0answers
12 views

What properties of a likelihood function are required for quasi-likelihood estimation?

Quasi-likelihood seems like a great way to use Iteratively Weighted Least Squares to fit linear linear models with a very general class of likelihoods. But what is that class? Obviously the ...
5
votes
1answer
40 views

Does the log likelihood become unimodal when the sample size goes to infinity?

I know that, under the usual regularity conditions, the MLE converges to the true parameter values as the sample size gets large. And the scaled MLE tends to being normally distributed. However, in a ...
0
votes
0answers
26 views

Question on MLE

I apologize for the basic question. If $\{p_\theta(x): \theta\in K\subseteq\mathbb{R}\}$ is a smooth family of distributions, then the MLE $\hat{\theta}_n,$ under suitable regularity conditions ...
1
vote
2answers
58 views

Does least squares regression imply normality of errors?

For the linear model $$y_i=\beta_0 +\sum_{k=1}^{n}\beta_k x_{ik} + \epsilon_i$$ the parameter estimates are the same for the maximum likelihood method and the least square method (minimizing ...
5
votes
2answers
88 views

Maximum likelihood estimator for $\theta$ and $E[X]$

Let $X_1,..., X_n $ be a random sample of a variable with PDF: $$f(x|\theta)=\frac{\theta}{x^2} I_{(\theta, \infty)}(x), \theta >0$$ Find the maximum likelihood estimator for $\theta$ and $ E[X]$ ...
0
votes
0answers
12 views

Parameter Estimation for Naive Bayes - Maximum a posteriori and Maximum Likelihood

I am wondering if I understand those terms correctly. To summarize my thoughts: In naive Bayes, our decision rule is basically the Maximum a posteriori (MAP) estimate of our hypothesis. We assign an ...
3
votes
2answers
122 views

Under what circumstances is the log likelihood function of a point process concave?

I am trying to understand under what circumstances the log likelihood function of a point process concave. Assume that the process can be defined by a conditional intensity function and that the log ...
3
votes
2answers
73 views

How to compute (or numerically estimate) the standard error of the MLE

I have a model for which I know the log likelihood function, the gradient of the log likelihood and the Hessian of the log likelihood. For given data I can compute the MLE using a generic optimizer ...
5
votes
1answer
89 views

Estimating $n$ and $p$ for Binomial distribution, repeated counting of partly hidden population

A brief motivation: $n$ critters live in an aquarium, where sadly they often hide in, under or behind things. When the aquarium is observed, each critter is only seen with probability $p$ ...
0
votes
0answers
25 views

How to justify a distribution fit is good

I've got 5 datasets to which I would like to fit a model distribution. I'd like to use the same distribution for each dataset but with different parameters. So I use MLE to compute the best parameters ...
1
vote
1answer
19 views

Supervised Pattern Recognition with Probabilistic Labels

I am interested in supervised pattern recognition problems where the the label associated with each pattern gives the probability of membership for each of the $c$ classes, rather than assigning each ...
0
votes
0answers
29 views

Numerical estimation of MLE in Python — normal distribution and gradient is close to zero away from the mean

I am exploring how to model a data set using normal distributions with both mean and variance defined as linear functions of independent variables. Something like $\mathcal{N} \sim \left (f(x), ...
4
votes
1answer
65 views

It it possible in R to specify a regression formula for the hazard rate for a survival analysis model?

I am currently trying to fit a survival analysis model which has the following survival function: $S(t) = \lambda_i e^{-\lambda_i t}$ but with $\lambda_i = e^{\beta_0 +\beta_1 log(1+X_i)}$ where ...
1
vote
0answers
19 views

Nagelkerke pseudo-R2 with positive log likelihoods

I'm trying to calculate a pseudo-R2 for linear mixed models using Nagelkerke's method . My understanding is that Nagelkerke's pseudo-R2=1-EXP[(-2/n)(l(B)-l(0))], where l(B) and l(0) are the ...
0
votes
1answer
26 views

Mixing probabilities in mixture models using EM

Assume we want to estimate the mixing probabilities ($\pi_{k}$) for each member distribution in the mixture model. We know that $\sum_{m}^{K}\pi_{m}=1$, so we can formulate the optimization problem ...
1
vote
0answers
53 views

Why are these MLE estimates biased?

I estimate the parameters of survival data with censoring which is simulated from Weibull distribution. The mean time to event was set to 10 by choosing the combinations of shape and scale ...
12
votes
7answers
1k views

MLE in layman terms

Could anyone explain to me in detail about maximum likelihood estimation (MLE) in layman's terms? I would like to know the underlying concept before going into mathematical derivation or equation.
-1
votes
0answers
33 views

In the glm function for logistic regression, where is the likelihood function stored? Is it in family? [migrated]

I am currently trying to run a logistic regression on my own, using the functions optim, nlm, etc. However, I am somehow getting ...
4
votes
1answer
230 views

Likelihood and estimates for mixed effects Logistic regression

First let's simulate some data for a logistic regression with fixed and random parts: ...
1
vote
1answer
44 views

Help with MLE regression

I have a data set containing two variables x and y. I want to estimate the parameters for a regression model. The regression ...
8
votes
2answers
255 views

What is the maximum likelihood estimate of the covariance of bivariate normal data when mean and variance are known?

Suppose we have a random sample from a bivariate normal distribution which has zeroes as means and ones as variances, so the only unknown parameter is the covariance. What is the MLE of the ...
0
votes
0answers
32 views

Problem with maximum likelihood estimation in R: NaNs produced

I'm trying to estimate 4 maximum likelihood estimators on a data set containing two variables: x and y. When using the following ...
5
votes
1answer
81 views

Exponential Family: Observed vs. Expected Sufficient Statistics

My question arises from reading reading Minka's "Estimating a Dirichlet Distribution", which states the following without proof in the context of deriving a maximum-likelihood estimator for a ...
1
vote
0answers
25 views

Document classification problem

Assume we have $L$ labelled documents, and $U$ unlabeled ones, where all the documents from class $k$ were generated from a multinomial or Naive Bayes distribution with parameter $\theta_k$, and ...
1
vote
1answer
47 views

Consistency of the extremum estimators

I am trying to understand intuitively the assumptions mentioned in the following theorem: Theorem. If there is a function $Q_0(\theta)$ s.t. (i) $Q_0(\theta)$ is uniquely maximized at $\theta_0$; ...
3
votes
1answer
48 views

Hessian of Laplace distribution

The density of the Laplace distribution is given by: $$f(x;\mu,\sigma)=\frac{1}{2\sigma}\exp\left(-\frac{\vert x- \mu\vert}{\sigma}\right).$$ It is easy to see that this function is not ...
0
votes
0answers
40 views

Confusion between empirical Bayes and Hierarchical Bayes Model for beta-binomial model

I am confuse with difference between empirical Bayes method and hierarchical Bayes method. Take an example (ref : http://www.cs.cmu.edu/~xuerui/papers/ctr.pdf). Suppose we have : $C$ is the ...
2
votes
1answer
44 views

Estimating standard error in a probit: econometrics or programming problem?

This question has two parts, as I do not understand whether my problem is theoretical (identification of the parameters) or practical (insufficient R skills). Econometrics Most "probit" style ...
0
votes
2answers
44 views

Likelihood ratio test to determine if average number of accidents has dropped?

I can't find any worked (non-trivial) practical example for a likelihood ratio test, believe me I have spent hours looking. Here is a question I've been trying to complete but I can't get any further. ...
0
votes
0answers
23 views

Gaussian Process Regression with additional Basis Functions

I'm working through Rasmussen's Gaussian Processes book, and I have a question about the possibility of optimizing additional basis function hyperparameters (in section 2.7 ...
1
vote
1answer
51 views

A modelling question about point processes with heavy tails

I am trying to model a number of point processes for which I have data. If I choose to model each one using a (different) homogeneous Poisson process and estimate the rate using MLE then for some of ...
0
votes
1answer
43 views

2x2 contingency table - maximum likelihood estimate of the odds ratio and exact confidence intervals

I have two questions regarding the following 2x2 contingency table: 1, How can I derive the maximum likelihood estimate of the odds ratio ($OR = (a*d)/(c*b)$) 2, How can I derive exact confidence ...
3
votes
2answers
46 views

Generalized log likelihood ratio test for non-nested models

I understand that if I have two models A and B and A is nested in B then, given some data, I can fit the parameters of A and B using MLE and apply the generalized log likelihood ratio test. In ...
1
vote
0answers
33 views

When does l1 regularisation give a sparse solution?

I was maximising a likelihood function, which is convex. I know that the system has a K-sparse solution. I wanted to know the conditions (or some sufficient conditions) on the likelihood function ...
0
votes
1answer
34 views

Why does maximum likelihood estimation not work in estimating signal in deterministic chaotic noise

I have few conceptual questions related to application of chaos in communications. In few application such as radar Chaotic signal reconstruction with application to noise radar system, cryptography, ...
1
vote
0answers
14 views

Simulated MLE does not exist, when trying to Bootstrap likelihood combinant

Consider this simple logistic model: We have ten $0/1$ observations $y_1,...,y_{10}.$ We model with an intercept and a predictor variable.The ten first observations have predictor value $X_i=0$, ...
0
votes
0answers
36 views

Expected value of likelihood function: Coin flips, biased and unbiased estimators

I've been reading the following (great!) book http://www.inference.phy.cam.ac.uk/itila/, which has sparked some questions about MLE. I'm comfortable with the notion that ML estimators are often ...
0
votes
0answers
32 views

Random effects models / Integrate over the random effect

I am trying to do maximum likelihood estimation and trying to see if the problem can be formulated using a random effect model. Here is the problem description: There are $100$ pairs $(N_i, D_i)$ ...
2
votes
1answer
36 views

Comparing OLS and ML through log likelihood value

The log-like likelihood values that are computed when I do a regression (by for instance eviews), are they comparable for different estimation techniques, specifically OLS and Maximum Likelihood? My ...
0
votes
0answers
51 views

Estimation parameters for latent (unobserved) variable

Here is my problem: I have 3 variables $X,Y,Z$ : $X$ is the number of clicks we observed on an web advertisement; $Y$ is the number of time a customer do a sign-up on the website after clicking ...
2
votes
1answer
118 views

Question with MLE

I'm having some problems with this question, and was hoping someone here could help. Let $X_1,\ldots,X_2$ be $n$ determinations of a physical constant $\theta$. Consider the model $X_i = ...
0
votes
1answer
27 views

Marquardt Loglikelihood Calculation in Eviews

I paper I am trying to replicate used Eviews to estimate their state space model (by maximizing the associated maximum likelihood). They used the BHHH and Marquardt algorithms. My question is given ...
1
vote
0answers
24 views

Maximum Likelihood through a noisy channel

I have a random variable $X$, which can take $n$ values and is distributed according to multinomial $\Theta=(\theta_1, \theta_2, \cdots, \theta_n)$. I observe a random variable $Y$, where I have that ...
1
vote
1answer
33 views

Heavy-tailed distribution with closed-form ML fit from data

Which (if any) heavy-tailed distributions can we compute the maximum likelihood parameters of, given some data to fit the distribution to?
1
vote
0answers
19 views

Quasi maximum likelihood estimation versus pseudo MLE

If I'm not wrong both "quasi" and "pseudo" denote the same thing, namely the optimization under wrong distributional assumptions. Moreover I think that the terms are not restricted to the assumption ...