Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
2,151
questions
1
vote
0answers
13 views
Parameter estimation by averaging over all high-likelihood possibilities?
I am refereeing a chemistry paper. The authors are trying to interpret some experimental data by comparison with numerical simulations. They have run many simulations using different combinations of ...
1
vote
0answers
14 views
How to simultaneously maximize multiple likelihoods?
This is a more generic question. Assuming I have closed-form likelihood functions of 2 related observations|distributions (for e.g. likelihood of observations from 2 MVN dists). However, because the ...
3
votes
0answers
22 views
MLE when variance of residuals is null (y is a linear combination of x)
Suppose I have the following model to be estimated via MLE assuming normal errors $y_{t}=x_{t} \beta +e_{t}$ with $e=N(0,\sigma^{2})$, where $y, x$ are matrixes and $\beta$ is a vector, so $\sigma^{2}$...
1
vote
3answers
31 views
Mean of bootstrap maximum likelihood samples?
I am trying to estimate best-fit parameters from some data for a model.
For that I use the maximum likelihood estimation (MLE), where eventually I end up finding an estimate of the parameters. In ...
7
votes
1answer
102 views
Does a logistic regression maximizing likelihood necessarily also maximize AUC over linear models?
Given a data set with binary outcomes $y\in\{0,1\}^n$ and some predictor matrix $X\in\mathbb{R}^{n\times p}$, the standard logistic regression model estimates coefficients $\beta_{MLE}$ which maximize ...
0
votes
0answers
13 views
Discriminative Models with Class Priors
In discriminative models, we model $p(Y|X)$ directly while in generative models we model $p(X|Y)p(Y)$ where $X$ is the input and $Y$ is the output variable. I am confused when the parameters and ...
0
votes
0answers
18 views
Derive the expression for the standard errors of the intercept in the simple regression via MLE
I am trying to derive the expression for the variance/standard errors of the alpha parameter in the simple regression framework.
The model is:
$$y_i = \alpha + \beta x_i + \epsilon_i$$ for $i=1,...,...
0
votes
0answers
11 views
Mix pdf and cdf in binary response model [duplicate]
Let's suppose that I have a model that tells me how likely is for an event to have come after a certain time lapsed, given by some kind an exponential distribution, i.e.
$$ \mathbb{P}(T_E < t) = \...
0
votes
0answers
14 views
MLE vs Expectation Maximization to estimate time-changing parameters in state space model
Suppose I have a generic model in state-space form described as
$$x_{t+1}=\phi_{t} x_{t}+w_{t+1}\epsilon_{t+1}$$
$$y_{t}=H_{t}x_{t}+v_{t}e_{t+1}$$
where both $e_{t+1}$ and $\epsilon_{t+1}$ are iid ...
1
vote
0answers
14 views
Clarification on Akaike's IC (AIC) and BIC for Expectation Maximization with time-changing parameters
I apologize in advance for the trivial question, but I need a clarification on the following issue. Suppose I have a generic model in state-space form described as
$$x_{t+1}=\phi_{t} x_{t}+w_{t+1}$$
$...
5
votes
3answers
176 views
Which likelihood function is used in linear regression?
When trying to derive the maximum likelihood estimation for a linear regression, We start by a likelihood function. Does it matter if we use either of these 2 forms?
$P(y|x,w)$
$P(y,x|w)$
All pages ...
0
votes
0answers
8 views
What is behind “forecast” in Eviews?
I have been trying to use state space models in order to represent some gestural data. Until now I have been using Eviews to to do all the dynamic forecasting part, so I was curious what is behind ...
2
votes
0answers
49 views
James Stein Estimator for more than one Sample
I have a hard time understanding the James-Stein Estimator. I show you how I tried to comprehend it by using a python example. I take a normally distributed random vector with mean $(0.1, 0.2, 0.3, 0....
0
votes
2answers
32 views
Kalman Filter with heteroscedastic Q (covariance of the transition noise)
I am looking at a generic derivation of the Kalman Filter (like this but you can take any). And I was wondering, checking all the derivation, why are we forced to assume that the covariance matrix Q ...
0
votes
1answer
34 views
estimate the parameters of t-distribution and fitting with MLE
Here is Fitting the t-Distribution by Maximum Likelihood t-method in book Statistics and Data Analysis for Financial Engineering with R examples page 113 and 168.
...
0
votes
2answers
33 views
Writing the likelihood and conditional variance in a ARMAX model or regression with GARCH (R rugarch with external covariates)
I was looking at the r package called rugarch (docs) also mentioned in this question and in the Matlab Guide but I cannot see any example of how the likelihood or log-likelihood is computed when ...
0
votes
0answers
23 views
Regression with GARCH error term
Let's consider the following multivariate regression ($y_{t}$ and $x_{t}$ below are matrixes of appropriate size) where the error term is assumed to follow a GARCH process:
$$y_{t}=\beta x_{t} + e_{t}$...
0
votes
1answer
40 views
Non-parametric MLE?
I have a question regarding non-parametric MLE as follows, and I am not sure whether my reasoning makes sense.
What I know about MLE usually concerns maximizing a formula of the form $\theta = \...
2
votes
1answer
37 views
Degenerate multivariate normal in Maximum Likelihood Estimator (Akaike's Info Criterion, BIC, LR Test usage)
Let's suppose that the considered set of random variable has a covariance matrix which is psd. Therefore the Gaussian pdf must be written in its degenerate form, where the determninat of the ...
1
vote
1answer
56 views
Proportions: why changing to population mean estimator doesn't change the distribution
I saw in a few places (e.g. here) when you compare proportions of 2 samples, under a null hypothesis that they are equal, you eventually get to this:
$$
\frac{\bar X - \bar Y}{\sqrt{P(1-P)(\frac{1}{n}...
0
votes
0answers
24 views
Maximum Likelihood of “Mean Parameters”
I'm dealing exclusively with the exponential family of distributions.
The mean parameters of the distribution are defined as the expected value of the sufficient statistics in terms of the ...
0
votes
1answer
32 views
Maximization of a likelihood function for a GARCH(1,1) process [closed]
I'm currently studying Pseudo Maximum Likelihood estimation. I'm trying to fit a GARCH model with Gaussian Pseudo Maximum Likelihood (and then non Gaussian), but before doing it on actual data I ...
0
votes
0answers
23 views
Maximum likelihood estimation for trajectories estimation
I am currently working on a project where I have to estimate the parameters of an equation in order to estimate next states.
More specifically, I have the state space:
$RHXt= a10+a11* RHZ(t-1)+a12*...
0
votes
0answers
15 views
Estimating parameters with GLM in R
There is a paper, published in the journal of Public Economics called: The marginal utility of income, by Layard, Nickell & Mayraz. They use a maximum likelihood estimation for ...
1
vote
1answer
83 views
How to write the log likelihood for the sum of independent poisson events
Say that the per each event of TYPE 1, the average number of occurrences is $\lambda_1$. Then the likelihood for the number of occurrences in a single event, $k_1$, is $\lambda_1^k / k_1! * e^{-\...
1
vote
0answers
27 views
Estimate population variance / mean from truncated distribution with known cutoff, but without parametric assumptions
Suppose you have a sample of $N$ iid random variables $X_i$ drawn from an unknown (but finite variance) distribution but with a known upper-cutoff $K$ and therefore support $[0,1,2,...,K]$ but un-...
0
votes
0answers
15 views
Estimating the noise to estimate profile likelihood
I am trying to obtain profile likelihoods around parameters obtained by fitting an ODE model to some data. I am using the method discussed in the study
Data is assumed to have normal errors ~$N(0,\...
1
vote
1answer
36 views
Where does Jensen's Inequality come into the EM derivation?
I am working my way through the original EM paper Maximum Likelihood from Incomplete Data by Dempster, et al.
I have run into a problem with a statement made in section 3. "General Properties". ...
0
votes
2answers
100 views
Maximum Likelihood Estimation of a dataset
I am coding a Maximum Likelihood Estimation of a given dataset (Data.csv). The goal is to estimate the mean and sigma. Note that the log of the dataset is well approximated by a normal distribution.
...
0
votes
1answer
11 views
How to “fit” the size of a binomial distribution
I'm solving a problem where I have 15 samples of a unknown distribution. They ask me to fit somehow the parameters of the Binomial, Poisson and Normal distributions. I could use the sample mean and ...
0
votes
0answers
19 views
Asymptotic Mean Squared Error of Maximum Likelihood estimator
I want to show that $n$ times mean squared error for the maximum likelihood estimator converges to the inverse of Fisher information, where $n$ is the number of samples.
But The standard proofs of ...
1
vote
1answer
14 views
Statistical comparison of (covariance) matrices
I am trying to test whether the covariance matrix for the maximum likelihood estimates for a gaussian general lienear model approaches the inverse Fisher information matrix (times 1/n , n being the ...
0
votes
0answers
26 views
Maximum likelihood estimation of simple multilevel regression model
I have a two-level regression model:
$$ Y_{ij} = \beta_{0j} + \beta_{1j} X_{ij} + \epsilon_{ij},$$ where
$$ \beta_{0j} = \gamma_{00} + \gamma_{01} Z_{j} + \mu_{0j},$$ and
$$ \beta_{1j} = \gamma_{...
0
votes
1answer
26 views
Maximum Likelyhood Estimator (MLE) vs. Bias [duplicate]
If we use the MLE methhod to find the estimator of the variance we get:
$\hat\sigma ^2 = \frac{\sum(x_i - \mu)^2}{n}$
Where we can plug the MLE estimator for $\mu$ and get:
$\hat\sigma ^2 = \frac{\...
1
vote
2answers
66 views
Why estimate using GEEs inspite of their disadvantages over using ML?
I know sometimes I want to know the population-level estimates, but the problem with GEEs is I can't calculate the likelihood, and therefore all models I make with it aren't comparable, and I don't ...
0
votes
0answers
8 views
Estimating false accept rates from imposter scores below a threshold
I have a system that compares two items and produces a match score. Scores below a threshold are manually inspected to determine if they match or don't(imposter). Scores above the threshold are ...
0
votes
0answers
26 views
What statistic to use in testing the variance of maximum likelihood estimators
(A physicist self-studying statistics here)
I was previously confused about the meaning of the standard error in a maximum likelihood estimate. Certain stack exchange posts (linked below) have gone ...
4
votes
0answers
42 views
What is the likelihood for this process?
A patient is admitted to the hospital. Their length of stay depends on 2 things: The severity of their injury, and how much their insurance is willing to pay to keep them in the hospital. Some ...
0
votes
1answer
48 views
Parameter Estimation in Generalized Linear Mixed Models
Let us assume a generalized linear mixed model with a binary dependent variable $y_{i, t } $ that is explained by a fixed effect matrix X and a simple random intercept for each individual $i$
$y_{i,t}...
0
votes
1answer
27 views
QML vs MLE for GJR-GARCH models
I am writing my master's thesis and using a AR(1) GJR-GARCH(1,1)-EVT-Copula model on my data. One of the main papers I use is McNeil & Frey (2000), in which they only do AR-GARCH-EVT. In this ...
0
votes
0answers
16 views
How do we combine probability distributions component-wise to make a more accurate probability distribution? [duplicate]
This subject intrigues me. My application is in the field of sports prediction. In sports prediction the experts compete who is the best - or we make it look as if they compete. So we try to find from ...
0
votes
1answer
34 views
Neural language model: Derivation of MLE
Recently, I studied NNLM and I saw the derivation of softmax using MLE:
\begin{align}
& \frac{\partial\log P(w_t\mid h)}{\partial\theta} \\[8pt]
= {} & \frac{\partial \log
\exp(s_\theta(w_t,...
2
votes
0answers
29 views
MLE for an ellipse
I'm not at all a statistician, but in the course of my work I've come across a non-trivial maximum likelihood estimation problem, and I'm looking for ideas and/or references on how to approach it. ...
0
votes
0answers
13 views
Binned maximum likelihood estimation: 1-sigma area around measurements
I hope my question isn't too stupid or obvious.
So I am supposed to analyze some data doing lifetime estimation radioactive elements, each data-point is poisson distributed. Using a binned log-...
1
vote
1answer
61 views
Unsure how this MLE was derived from given normal distribution
I came across this mle formula in some (undocumented) code performing linear regression with input matrices $A$ and $B,$ and was wondering how it was derived. It might also be some level of ...
0
votes
1answer
20 views
Maximum likelihood estimate of 1 Bernoulli trial?
I was recently asked to calculate the maximum likelihood estimate of a single Bernoulli trial. Since the MLE of a binomial distribution is just the mean of the observed number of successes, I reasoned ...
2
votes
2answers
90 views
Are “improper uniform priors” in Bayesian analysis equivalent to maximum likelihood estimations?
The improper uniform distribution for parameter $\theta$ is :
$p(\theta)=1,\ for -\infty<\theta<\infty$.
It is called "improper" since it does not integrate to 1. Because Bayesian theorem is ...
0
votes
0answers
9 views
MAP estimate and Maximization step
I have a very simple question. My reference textbook is the Murphy, "machine learning, a probabilistic perspective". Let's imagine we are trying to fit a GMM $\gamma$ with MAP.
We know that the ...
1
vote
2answers
133 views
Relationship between distribution fitting and simple regression?
This is a bit of a conceptual question that has been nagging me for a long time.
Based on a set of data, $(X_1, X_2, X_3, \ldots, X_k)$, with sample size $i = 1 \ldots n$ ,
is there an explicit ...
1
vote
0answers
30 views
What is the likelihood function of the starting time of diffusion?
I need to find the likelihood that a set of molecules was instantaneously released at time $t_0$, say $t_0=0$.
Toy System Example:
Let $N$ be the set of molecules released from a specific point in a ...
