Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
3,012
questions
0
votes
0
answers
3
views
Find asymptotic variance of the moment estimator
I have that
$$f(x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}x^2}$$
I have the conditional distribution:
$$f_{\beta}(y|x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}(y-\beta_0-\beta_1x-\beta_2x^2)^2}$$
and we ...
- 253
0
votes
0
answers
35
views
How can I get the Binary Cross Entropy from the Cross Entropy function for GANs
I got the definition of log-likelihood by Goodfellow's Deep Learning book:
\begin{equation}
\label{eq:loglikelihood}
\theta_{ML} = {argmax}\sum_{i=1}^{m} \log p_{model}(x_i; \theta).
\end{...
0
votes
0
answers
16
views
Diffusion race model with censoring. Help to verify overall logic
I conducted a Go-noGO experiment in which the subject had to press a button if the stimulus on the screen was an orange ($O$) and had to refrain from pressing it if he saw an apple ($A$). The ...
- 663
2
votes
1
answer
72
views
The probability/cumulative density function for inequality of two random variables
I have two random variables X and Y which came from different inverse gaussian (IG) ...
- 663
0
votes
0
answers
10
views
How can I estimate the variance of the error terms in a conditional/multinomial logit model?
Conditional/multinomial logit models(CML) can be esimated by the Maximum Likelihood Estimation(MLE). The likelihood would consists of choice probabilities:
\begin{equation}
P_{ij}=\frac{e^{V_{ij}}}...
0
votes
1
answer
34
views
Find the MLE density function of uniform [-\theta,\theta] [duplicate]
For $X_1,\dots,X_n$, i.i.d $X_n \sim \mathrm{unif}[-\theta,\theta]$, the ML: $\hat\theta_{MLE}=\mathrm{max}\{-X_{(1)},X_{(n)}\}$. Find the density function. Hint: For $x_1,\dots,x_n$ : $\textrm{max}\{-...
1
vote
1
answer
28
views
MLE of the Uniform Distribution
In a uniform distribution where $0\leq X \leq \theta$, the pdf is represented as $f(X|\theta) = \frac{1}{\theta}I(0\leq X \leq \theta)$, and the likelihood is $L(\theta) = \prod\frac{1}{\theta}I(0\leq ...
- 61
0
votes
0
answers
35
views
MLE for the Uniform distribution [duplicate]
I understand how a random sample $x_1, ..., x_n$ following the Uniform Distribution with $0 \leq x \leq \theta$ has a log-likelihood proportional to $\frac{1}{\theta^2}$. I am told that the MLE for $\...
- 61
0
votes
0
answers
22
views
weighted maximum likelihood as loss function
I have built a little neural network that I use for regression.
...
- 113
0
votes
0
answers
35
views
Fisher matrix for a discrete distribution
Let $\mathbf{X} = \{X_1, \ldots, X_n\}$ be a sample of i.i.d. variables following a discrete distribution with parameters $\mathbf{p}^T = (p_1, p_2, p_3)$. How can I find the Fisher information matrix ...
- 405
1
vote
0
answers
38
views
Find a maximum likelihood estimator in discrete uniform distribution
X is distributed as discrete uniform distribution on {1,2,3,...C) where C is an integer higher than or equal to 4
Pr(X=x)=p for x ∈{1,2,...,𝐶} which means Cp=1
X is censored and we can only observe Y=...
- 11
0
votes
0
answers
9
views
Nonlinear constrained optimization for a CIR model
I want to calibrate a CIR model which is commonly used to model the evolution of interest rates. Briefly speaking, we know that its dynamics is of the form
\begin{equation}
r_t = \kappa (\theta - r_t) ...
- 133
0
votes
0
answers
12
views
FIML in growth mixture modelling
I have a question about Full Information Maximum Likelihood (FIML). I’m fitting growth mixture models to outcome variables measured at three timepoints. Some individuals are missing outcome data at ...
- 1
2
votes
1
answer
29
views
Estimating Mixture Models with Maximum Likelihood
Suppose you have a Normal Mixture Model with 2 Components - you could write this model as follows:
$\pi_1 N(\mu_1, \sigma_1) + \pi_2 N(\mu_2, \sigma_2)$
In the above model, there are 6 unknowns : $\...
- 5,698
1
vote
1
answer
34
views
Why is it said that maximum likelihood becomes intractable if there are latent variables?
It is said that EM algorithm helps in cases where direct MLE cannot be carried out due to missing/latent variables. However, I could not understand why direct MLE cannot be carried out when there are ...
- 61
0
votes
0
answers
20
views
mgcv GAM models in R package caret - how to interpret output
I am attempting to evaluate two GAM models I developed in mgcv via leave-one-out cross validation in the caret package. I am a newbie to both GAMs and cross-validation. For the purposes of this ...
- 11
0
votes
0
answers
14
views
Maximum Likelihood Estimation Uniform Distribution [duplicate]
A sample is drawn from the uniform distribution:
$X_{1},X_{2},...,X_{n}:U(0,2\theta )$
I am interested in finding the MLE for $\theta$.
My intuition say that
$\hat{\theta }=max(X_{i})/2$
How do I ...
- 418
0
votes
0
answers
8
views
Is there a good package to implement latent profile analysis in R, which allows use of FIML for missing data
We are looking to conduct latent profile analysis in R, using a large number of continuous variables (physical activity level at different times of day). We have objective measurement of physical ...
- 1
1
vote
0
answers
23
views
Using restricted maximum likelihood on marginal residuals
In a mixed-model setting, I want to estimate the variance components and the pertaining random effects of a random-intercept/random-slope model. The coefficients of the fixed effects have already been ...
- 493
1
vote
1
answer
29
views
Maximum Likelihood Estimation for data with non normal distribution [closed]
I have a set of data (results) that does not follow Normal distribution. In this case, wow can I get Maximum Likelihood Estimation?
Thank you very much
- 143
0
votes
0
answers
9
views
GPD and GEV Fitting: Maximum Likelihood vs. Least Squares
I am trying to build a model based on real world data which involves fitting generalized extreme value distributions and generalized Pareto distributions. Most literature immediately turns to the ...
0
votes
0
answers
10
views
Renewal counting process with inter-arrival time gamma distribution: Model estimation
Let's start with the Poisson process:
If $N_t$ is a Poisson process with parameter $\lambda$, then we know that the inter-arrival time distribution is an exponential distribution with parameter $\...
- 133
2
votes
1
answer
34
views
If $X_1 \sim \text{binom}(p_1,n_1)$ and $X_2 \sim \text{binom}(p_2,n_2)$, how to prove that the MLE of $p = p_1 - p_2$ is $\hat{p}_1 - \hat{p}_2$?
Suppose $X_1 \sim \text{binom}(p_1,n_1)$ and $X_2 \sim \text{binom}(p_2,n_2)$, where $X_1$ and $X_2$ are independent, and let $p = p_1 - p_2$. How can I prove that $\hat{p} = \hat{p}_1 - \hat{p}_2$? (...
- 59
-1
votes
0
answers
28
views
Integration in R: "longer object length is not a multiple of shorter object length" Error [migrated]
I am trying to use R to calculate the marginal likelihood of a set of data (likelihood_N) with respect to the parameter $v_0$. To do this, I created the following ...
- 109
0
votes
0
answers
8
views
REML for multilevel modeling comparison
With the very same data,
I want to compare the two multilevel models.
(these two models differs in both fixed and random effects)
so, I am planning simulation study.
I will generate the data set, and ...
- 109
0
votes
0
answers
17
views
Equivalent ways of writing the log-likelihood of a sample of normal RVs
I am going through my econometrics textbook right now and the textbook writes the log-likelihood equation for a sample of normal random variables in a way I have never seen before. Specifically, for a ...
- 135
0
votes
0
answers
12
views
verify minimizing the summation of the Kullback–Leibler is equal to maximum likelihood estimate for theta
I wondered it seems like the maximum likelihood equals minimizing
$\sum_x P(x)\log{P(x)}-log{\hat P(x)}$,as P(X) is the truth but why it equals to $\sum_x P(x)\log{P(x)}-P(x)log{\hat P(x)}$?I cannot ...
- 1
3
votes
0
answers
65
views
$Y_1, Y_2, ... $ are iid Poisson($\lambda$)
We observe $Y_1, Y_2, ..., Y_T$ such that $T$ is the first $t\geq 1$ for which $Y_t>0$. Define $Y=Y_T$
(a) Find MLE $\hat{\lambda}$.
(b) What is the relative bias, $[E(\hat{\lambda})- \lambda]/\...
- 31
0
votes
0
answers
14
views
How to estimate Pearson type-IV-GARCH using MLE in R?
I am just wondering whether there is an R program that can be applied to run GARCH specifications with Pearson types IV distributions. If you are familiar with any or can guide, it is greatly ...
- 1
0
votes
0
answers
15
views
t-Copula MLE on nu (DoF) only - log-likelihood function possibly convex?
I am working with t-Copula's to generate random synthetic data eventually. The paper I use as the foundation is Benali et al., 2021. To determine the best fitting t-Copula, they propose determining ...
0
votes
0
answers
13
views
emmeans, should I use ML or REML fitted models [duplicate]
I see a lot of examples out there which use ML or REML fitted models as an input for emmeans.
If I understand correctly for model comparison of fixed effects for "fixed effects nested models"...
- 33
0
votes
0
answers
19
views
Self-Study: Score Statistic of a Poisson
I have been attempting a question on Generalised Linear Models. I thought I understood it but doing exercises myself now, proves otherwise.
I have attached an image of the question here, struggling ...
0
votes
0
answers
22
views
What is the limiting distribution of $$R_n=2(l(\hat{\theta})-l(\tilde{\theta}))$$ under null model $H_0$ for Poisson?
If $X_1,\dots, X_n\sim Poisson (\theta_1)$ and $Y_1,\dots, Y_n\sim Poisson (\theta_2)$, and $X_i$ is independent of $Y_i$, we want to test
$$
H_0: \theta_1=\theta_2, \, H_a: \theta_1\neq \theta_2
$$
...
- 232
0
votes
0
answers
43
views
Determine the asymptotic distribution $\sqrt{n}(\hat{\theta}_n-\theta) $ for trinomial distribution over the group sizes $(x,y,z)$
A random sample of $n$ individuals are classified into three groups, with probabilities $\theta^2$, $2\theta(1-\theta)$, and $(1-\theta)^2$ respectively, yielding the trinomial distribution over the ...
- 232
0
votes
0
answers
15
views
Is the likelihood of a discrete binomial variable same as it probability? Like in the case of tossing a coin lets say 12 times
I am working on a probability project where we first generate random variables for a given Binomial experiment and then we generate a PMF for 10 coin tosses using the list of random variables we ...
- 1
3
votes
1
answer
91
views
Is the likelihood for Gaussian mixture models still multimodal when Y is partially observed?
In discussing Gaussian mixture models (GMMs), https://normaldeviate.wordpress.com/2012/08/04/mixture-models-the-twilight-zone-of-statistics/ highlights the issue of
Multimodality of the Likelihood. ...
- 3,768
0
votes
0
answers
28
views
MLE for sample mean estimation in practice
I usually use median for a truncated normal distribution mean estimate. In theory the best approach is to do gradient descent optimizing the mean for data generation. Does anyone have a good example ...
- 111
0
votes
0
answers
42
views
Proportion data: Logistic with MLE vs. OLS with logit-transformed response
This is an expansion of @Beethoven_90's comment on this question. Suppose I have proportion data $Y_i$ computed from a binomial; $Y_i = \frac{S_i}{N_i}$ where $S_i \sim Bin(N_i, p_i)$ and $p_i$ is the ...
- 1
0
votes
0
answers
25
views
Why is it said, that GEE is not likelihood based, IF the estimating equations are derived exactly thru d(log(L)/dB?
I found this article: https://sakai.unc.edu/access/content/group/2842013b-58f5-4453-aa8d-3e01bacbfc3d/public/Ecol562_Spring2012/docs/lectures/lecture22.htm#choosing
and most of it is showing the ...
- 41
0
votes
0
answers
12
views
Parameter Estimation on S1 model
I have the following model of creating a random graph on the circle:
First, N nodes are uniformly distributed on the circle of radius $N/(2\pi)$ to give a node density of $1$. We sample the expected ...
1
vote
1
answer
48
views
Why cannot MLE be implemented for Gaussian mixture model directly?
Consider the following density, the mixture of two Gaussian distributions,
\begin{align*}
p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0) ,
\end{align*}
where $p(k=1)+p(k=0)=\pi_1+\...
- 95
0
votes
0
answers
20
views
How to estimate the parameters of a hypergeomtric distribution?
I am looking for some guidance on how to estimate the parameters of a hyper-geometric distribution, based on a random sample.
For example, if I generate a distribution as follows:
...
- 462
1
vote
0
answers
20
views
MLE estimates of regime shifted discrete process
I have a discrete process as defined,
$y_t$ = $a_1$$x_t$ + $b_1$ + $\mathcal{N}(0,\alpha)$ when $t <= \theta$ and $y_t$ = $a_2$$x_t$ + $b_2$ + $\mathcal{N}(0,\alpha)$ when $t > \theta$ for $t=0,...
- 21
-1
votes
1
answer
154
views
How to get the $95\%$ Wald test confidence interval for $\theta$? [closed]
Suppose that iid random samples $X_i$ from a discrete CDF $F(x)$ on $\{x_1,\dots, x_n\}$ with mean $EX=\theta$. We want to estimate $F(x)$. We consider empirical likelihood for $F(x)=\sum_{i=1}^n p_i ...
- 232
0
votes
1
answer
57
views
How to use R to plot $f_n(\theta)$ based on following samples? [closed]
For iid random samples $X_1,\dots, X_n$ with mean value $E X_1=\theta$, take $X_{(1)}=\min \{X_1,\dots, X_n\}$ and $X_{(n)}=\max \{X_1,\dots, X_n\}$. If $\lambda$ solves equation
$$
\sum_{i=1}^n \frac{...
- 232
1
vote
1
answer
40
views
Issue with Casella&Berger derivation of EM likelihood equality
In the explanation of the EM (Expectation maximization) algorithm p.328 in the book "Statistical inference" by G. Casella and R. Berger, 2nd edition, they present the following:
$\mathbf{Y} =...
- 357
0
votes
0
answers
29
views
Rao Cramèr Lower Bound problem
Let $X_1, · · · , X_n$ be a random sample from the uniform distribution on $[0, θ]$. I want to get the variance of the maximum likelihood estimator of $θ$ and check whether the variance decrease at ...
- 21
0
votes
0
answers
33
views
marginal likelihood in relevance vector machines RVM
I need to do a fit of 1d response versus 1d input, both real quantities.
I wanted to implement Linear Regression, but because I wanted uncertainty in the resulting fit-params, I studied about Bayesian ...
- 101
1
vote
2
answers
74
views
Why can you not estimate Beta with least squares in logistic regression? [duplicate]
As the title says, why can you not estimate Beta (coefficients) in logistic regression with least-squares? I read in a book that it is not possible and that we use maximum likelihood instead, but I ...
- 35
0
votes
0
answers
46
views
Likelihood-based CI is better than Wald CI?
The text I'm reading mentions the following:
Let us consider a scalar parameter case. The likelihood-based approximate 95% cI is
$$\left\{\theta,2\log\frac{L(\hat{\theta})}{L(\theta)} \le3.84 \right\}$...
