Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

719 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
16
votes
0answers
537 views

Fitting custom distributions by MLE

My question relates to fitting custom distributions in R but I feel it has enough of a probability element to remain on CV. I have an interesting set of data which has the following characteristics: ...
11
votes
0answers
963 views

Testing for a significant difference between ML estimates: Likelihood ratio or Wald test?

I am trying to test whether or not there is a significant difference between maximum likelihood estimates of two genetic parameters (selection and dominance) across two environments with genotype data ...
8
votes
1answer
889 views

Unbiased estimator for AR($p$) model

Consider an AR($p$) model (assuming zero mean for simplicity): $$ x_t = \varphi_1 x_{t-1} + \dotsc + \varphi_p x_{t-p} + \varepsilon_t $$ The OLS estimator (equivalent to the conditional maximum ...
7
votes
0answers
136 views

MLE, regularity conditions, finite and infinite parameter spaces

The problem I have is in figuring out why the MLE is no longer consistent in countable parameter spaces under conditions specified below. The set up is as follows: we are consider a parameters space ...
7
votes
0answers
461 views

Does using bootstrapped samples improve parameter estimates for a fitted distribution?

The R package retimes has a function for fitting an ex-Gaussian distribution to a set of observations. The method involves taking multiple bootstrapped samples of the observations, and fitting the ex-...
7
votes
0answers
215 views

asymptotic unbiasedness of weibull mle

It's known that the MLEs of the two-parameter Weibull distribution scale and shape parameters are not available in a closed form. It is, however, known that they do exist, are unique, and moreover, ...
6
votes
1answer
91 views

Maximum Likelihood Estimator of $P(Y_1=1)$ where $Y_i=1$ if $X_i>0$ and $0$ otherwise, given $X_1,\dots,X_n\sim N(\theta,1)$

This is part(a) of exercise 6 of Chapter 9 from Wasserman's All of Statistics. Let $X_1,\dots,X_n\sim N(\theta,1)$. Define $Y_i=\begin{cases} 1 &\text{ if }X_i>0 \\ 0 &\text{ if }X_i\le 0....
6
votes
0answers
123 views

Finding MLE and MSE of $\theta$ where $f_X(x\mid\theta)=\theta x^{−2} I_{x\geq\theta}(x)$

Consider i.i.d random variables $X_1$, $X_2$, . . . , $X_n$ having pdf $$f_X(x\mid\theta) = \begin{cases} \theta x^{−2} & x\geq\theta \\ 0 & x\lt\theta \end{cases}$$ where $\theta \...
6
votes
0answers
130 views

Bayesian inference via approximate data likelihood

Suppose that we have a very large i.i.d. sample $x_1,...,x_n$ and a data likelihood defined by $$p(x | \theta,\beta) = \prod_ip(x_i | \theta,\beta)$$. Further suppose that $\theta$ is the parameter ...
6
votes
0answers
628 views

Likelihood maximization: MCEM algorithm versus MCMC algorithm

Hello Everyone this is my first question. I am a particle physicist and I am doing some empirical studiues on parameters estimation using different methods (this might give me some handle to study on ...
6
votes
0answers
1k views

“weight” input in glm.nb function in R. How exactly does the weight affect the likelihood?

I would like to understand how the weight argument of glm.nb is affecting the likelihood function. I understand that glm.nb find the MLE in an alternating iteration process where for a given theta the ...
6
votes
0answers
140 views

Is this problem Bayesian? And can I use variational approximation?

Suppose there are $N$ samples of observations $\mathbf X(n)$ ($n=1,\cdots,N$), which are given by probability distribution $p(\mathbf X(n)|\mathbf Z(n))$ with their conditions are given by hidden ...
6
votes
0answers
594 views

Computing Standard Errors in EM algorithm

I'm applying the EM to a hidden markov chain (the $\mathbf{Z}=\{Z_1,...,Z_n\}$ variable), with observations(the $\mathbf{Y}=\{Y_0,...,Y_n\}$ variable) dependent not only on the hidden markov chain, ...
5
votes
0answers
117 views

Multiple maximum likelihood estimates for discrete parameter

Suppose I have a bivariate likelihood function, $L(\theta ,\lambda |\mathbf{x})$, where $\theta$ can take on continuous values, but $\lambda$ can only take 'count' values $(0,1,2,...)$, and $\mathbf{x}...
5
votes
0answers
92 views

Maximum likelihood and degenerate Fisher information

I am wondering if there are some standard results to find rates of convergence of the MLE for different sub-parameters, when the Fisher information is degenerate. More precisely, suppose that I ...
5
votes
0answers
319 views

Relationship of Poisson regression to determining proportions in a mixture distribution

I am a particle physicist, and a very frequent task is: given data sampled from a distribution mixture $$ Z \sim \sum_i \phi_i F_i, $$ where $\phi_i$ are the mixture proportion / prior probabilities ...
5
votes
1answer
355 views

What is the objective of maximum likelihood estimation?

I was reading in "Using Maximum Likelihood Estimation" from "Econometrics for Dummies", and here's what the author had to say: "The objective of maximum likelihood (ML) estimation is to choose values ...
5
votes
0answers
874 views

Why is Fisher Scoring easier to compute?

In practice, the observed information matrix (Newton-Raphson) is usually replaced by its expectation, known as Fisher scoring. Link: https://en.wikipedia.org/wiki/Scoring_algorithm#Fisher_scoring ...
5
votes
0answers
243 views

Bayesian estimates for Deming regression coinciding with least-squares estimates

Consider the following Deming model with independent replicates : $$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, \...
5
votes
0answers
2k views

Frequency weights, rare events and logistic regression

I'm working on a model that requires me to look for predictors for a rare event (less than 0.5% of the total of my observations). My total sample is a significant part of the total population (50,000 ...
4
votes
0answers
41 views

What is the likelihood for this process?

A patient is admitted to the hospital. Their length of stay depends on 2 things: The severity of their injury, and how much their insurance is willing to pay to keep them in the hospital. Some ...
4
votes
0answers
381 views

Fisher information matrix in logistic regression

I am self-studying the basics of logistic regression. I came across this sentence: In logistic regression expected and observed information matrixes are equal I am aware that the information ...
4
votes
1answer
135 views

What is the likelihood function of this random variable (beta distribution parameterizing a Bernoulli distribution)?

This is related to an earlier self-study question of mine. The setup is that there are $N$ individuals, indexed by $i$, and two time periods. Individuals choose whether to "invent" something in the ...
4
votes
0answers
198 views

Likelihood function of a hierarchical model

I have the following model: $$ y\sim\textrm{MvNormal}\left(\mu,\Sigma\right)\\ p=\textrm{logistic}\left(y\right)\\ k\sim\textrm{Binomial}\left(p,n\right) $$ Where $\mu$ and $\Sigma$ are free ...
4
votes
0answers
45 views

MLE in a stable Gaussian VAR$(p)$ process. Are the Lutkepohl formulas correct?

In «New Introduction to Multiple Time Series», page 90, we have the following formulas for the ML estimators of a stable Gaussian VAR$(p)$ process: where $\tilde \alpha = vec(\tilde A_1,...,\tilde ...
4
votes
1answer
46 views

Need help to understand the log-likelihood annotation?

Trying to know the steps to find the maximum likelihood estimate for the covariance matrix, assuming normal probability distribution, I want to differentiate log-likelihood function but what confuses ...
4
votes
0answers
223 views

Connecting Poisson and multinomial models

Let's say we have multinomial counts $y_{jp}$ (corresponding to observations $j$ over categories $p=1,...P$) that are arranged in a table of $n$ rows and $P$ columns. Then say we have the log-linear ...
4
votes
1answer
78 views

Computing the marginal MAP estimate

Suppose I have some IID Gaussian data with priors for mean and standard deviation: \begin{align} P(x|\mu,\sigma)&=\prod_{i=1}^n\frac{1}{\sqrt{2\pi\sigma^2}}e^\frac{-(x_i-\mu)^2}{2\sigma^2}\\ p(\...
4
votes
1answer
505 views

Likelihood of LDA compared to logistic regression

I've come across an interesting exercise. We are given four classification models for binary response and a $d$-dimensional independent variable: A Linear Discriminant Analysis model where the ...
4
votes
0answers
2k views

What is the exact log-likelihood of an AR(2) model?

Let's say we have the following AR(2) model: $y_t=\phi_0+\phi_1y_{t-1}+\phi_2y_{t-2}+e_t, \; e_t\sim N(0,\sigma^2_e)$ with T observations in total. Working out the conditional log-likelihood is ...
4
votes
0answers
1k views

Expected and observed Fisher information?

Studying asymptotics, I bumped into the concept of Observed Fisher Information, as a way to compute Fisher Information when the parameter $\theta$ is unknown. I am also aware that it is related in ...
4
votes
0answers
204 views

Unbiased Estimator of the truncation points in a truncated normal distribution?

Consider the variables $x_i \text{~} \mathcal{N}(\mu, \sigma^2,a,b)$ iid with truncation points $a$ and $b$, i.e. $a < x_i < b$. Suppose all 4 parameters, namely $\mu, \sigma, a, b$ are all ...
4
votes
0answers
88 views

Show that MLE of $\lambda = \frac{n-T_n}{S_n+cT_n}$

$X_i$ are i.i.d exponential, mean $\lambda^{-1}$ for $1 \leq i \leq n$ and, the values are measured such that $X_i = c$ if $X_i \geq c$ and $X_i$ otherwise. Show that MLE of $\lambda = \frac{n-T_n}{...
4
votes
0answers
272 views

Maximum likelihood estimation involving both probabilities and probability densities

Note: based on suggestions in the comments, I have rewritten this question. Please refer to the history for the original version. In general my question regards how to compute likelihoods in mixed ...
4
votes
0answers
2k views

Deriving the maximum likelihood for a generative classification model for K classes

In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for $K$ classes defined by the prior class ...
4
votes
0answers
667 views

Fitting Multivariate Bernoulli distribution

I want to fit a model to a number of observations, each of them being a k-dimensional binary vector $(x_1, x_2, ..., x_k)$ where $x_i \in \{0,1\}$. Naturally I would like to fit a multivariate ...
4
votes
0answers
267 views

What properties of a likelihood function are required for quasi-likelihood estimation?

Quasi-likelihood seems like a great way to use Iteratively Weighted Least Squares to fit linear models with a very general class of likelihoods. But what is that class? Obviously the distribution ...
4
votes
0answers
1k views

Parameter Estimation for Naive Bayes - Maximum a posteriori and Maximum Likelihood

I am wondering if I understand those terms correctly. To summarize my thoughts: In naive Bayes, our decision rule is basically the Maximum a posteriori (MAP) estimate of our hypothesis. We assign an ...
4
votes
0answers
586 views

2x2 contingency table - maximum likelihood estimate of the odds ratio and exact confidence intervals

I have two questions regarding the following 2x2 contingency table: 1, How can I derive the maximum likelihood estimate of the odds ratio ($OR = (a*d)/(c*b)$) 2, How can I derive exact confidence ...
4
votes
0answers
439 views

Intuitive explanation of “integrate out random effect”

We are trying to figure out an intuitive reasoning behind integrate out the unobserved random effect. The specific formula is: $f\big(y_i|x_i;\beta, \sigma_c\big)=\int_{-\infty}^{+\infty}\Big(\prod_{...
4
votes
0answers
83 views

Likelihood Function for Complicated Transformations

Suppose that data X have a Normal distribution with some mean $\mu$ and some variance $\sigma^2$. However, you don't get to see X. Instead, you see $Y = g(X)$ where $g$ is a known function. Assume ...
4
votes
0answers
157 views

How to obtain simultaneous confidence intervals for correlated frequencies?

I observed 400 episodes of nursing care in a hospital. I tracked the movement of the nurses between 5 rooms $A,B,C,D$ and $E$. The maximum likelihood estimates of the probabilities of moving from ...
4
votes
0answers
309 views

Estimating right-censored data

I am VERY new to stats. I have a large amount of life-time data (delay in arrival since start of experiment) from repeat experiments. Some data is missing, but essentially represents a delay longer ...
4
votes
0answers
899 views

Relationship between regularized least squares and MLE

We know that the least square method is equivalent to the MLE for Gaussian distributed errors. What is the relationship (if any) between regularized (Tichonov regularization) least squares and MLE?
4
votes
0answers
3k views

Gamma distribution and Cramér-Rao bound

There are two definitions of the GAMMA distribution: I did the ML estimation, generated the Fisher Information, compared it to the Variance and the Cramer Lower Bound was reached, so the estimator ...
4
votes
0answers
273 views

Methodology for validation of stochastic simulations with Kolmogorov-Smirnov test

I'm a phd student in Geography, i need some help (or good ressources) to understand why and when i need to use PIT (Probability integral transform) in my validation program for simulation. I explain ...
4
votes
0answers
122 views

Factor models with small noises

The standard factor model formulation is $y=W x+\epsilon$ where $x \sim \mathcal{N}(0, I)$, $\epsilon \sim\mathcal{N}(0, \Sigma)$. $W$ and $\Sigma$ are typically estimated from MLE. The solution can ...
4
votes
1answer
51 views

Help on statistical modeling of pedestrian flow in subways

I'm a New Yorker and take the subways every day. I have a growing interest in understanding the distribution of paths people take on the subways to work every day. I.e. if there are $n$ subway ...
3
votes
0answers
45 views

Does any `R` package offer `gnorm`, `hnorm`, and similar? What about other languages?

R typically offers functions prefixed by d, p, q, and r ...
3
votes
0answers
57 views

Maximum likelihood: Why is the number of non-zero eigenvalues equal to $x^T \hat{\Sigma}^{-1} x$

I've been reading this code (based on this R package) and I found that the number of non-zero eigenvalues of the estimated covariance is roughly equal to $x_i^T \hat{\Sigma}^{-1} x_i$. I want to know ...