Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

Filter by
Sorted by
Tagged with
0 votes
0 answers
3 views

Find asymptotic variance of the moment estimator

I have that $$f(x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}x^2}$$ I have the conditional distribution: $$f_{\beta}(y|x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}(y-\beta_0-\beta_1x-\beta_2x^2)^2}$$ and we ...
user avatar
  • 253
0 votes
0 answers
35 views

How can I get the Binary Cross Entropy from the Cross Entropy function for GANs

I got the definition of log-likelihood by Goodfellow's Deep Learning book: \begin{equation} \label{eq:loglikelihood} \theta_{ML} = {argmax}\sum_{i=1}^{m} \log p_{model}(x_i; \theta). \end{...
user avatar
0 votes
0 answers
16 views

Diffusion race model with censoring. Help to verify overall logic

I conducted a Go-noGO experiment in which the subject had to press a button if the stimulus on the screen was an orange ($O$) and had to refrain from pressing it if he saw an apple ($A$). The ...
user avatar
  • 663
2 votes
1 answer
72 views

The probability/cumulative density function for inequality of two random variables

I have two random variables X and Y which came from different inverse gaussian (IG) ...
user avatar
  • 663
0 votes
0 answers
10 views

How can I estimate the variance of the error terms in a conditional/multinomial logit model?

Conditional/multinomial logit models(CML) can be esimated by the Maximum Likelihood Estimation(MLE). The likelihood would consists of choice probabilities: \begin{equation} P_{ij}=\frac{e^{V_{ij}}}...
user avatar
0 votes
1 answer
34 views

Find the MLE density function of uniform [-\theta,\theta] [duplicate]

For $X_1,\dots,X_n$, i.i.d $X_n \sim \mathrm{unif}[-\theta,\theta]$, the ML: $\hat\theta_{MLE}=\mathrm{max}\{-X_{(1)},X_{(n)}\}$. Find the density function. Hint: For $x_1,\dots,x_n$ : $\textrm{max}\{-...
user avatar
1 vote
1 answer
28 views

MLE of the Uniform Distribution

In a uniform distribution where $0\leq X \leq \theta$, the pdf is represented as $f(X|\theta) = \frac{1}{\theta}I(0\leq X \leq \theta)$, and the likelihood is $L(\theta) = \prod\frac{1}{\theta}I(0\leq ...
user avatar
  • 61
0 votes
0 answers
35 views

MLE for the Uniform distribution [duplicate]

I understand how a random sample $x_1, ..., x_n$ following the Uniform Distribution with $0 \leq x \leq \theta$ has a log-likelihood proportional to $\frac{1}{\theta^2}$. I am told that the MLE for $\...
user avatar
  • 61
0 votes
0 answers
22 views

weighted maximum likelihood as loss function

I have built a little neural network that I use for regression. ...
user avatar
  • 113
0 votes
0 answers
35 views

Fisher matrix for a discrete distribution

Let $\mathbf{X} = \{X_1, \ldots, X_n\}$ be a sample of i.i.d. variables following a discrete distribution with parameters $\mathbf{p}^T = (p_1, p_2, p_3)$. How can I find the Fisher information matrix ...
user avatar
1 vote
0 answers
38 views

Find a maximum likelihood estimator in discrete uniform distribution

X is distributed as discrete uniform distribution on {1,2,3,...C) where C is an integer higher than or equal to 4 Pr(X=x)=p for x ∈{1,2,...,𝐶} which means Cp=1 X is censored and we can only observe Y=...
user avatar
  • 11
0 votes
0 answers
9 views

Nonlinear constrained optimization for a CIR model

I want to calibrate a CIR model which is commonly used to model the evolution of interest rates. Briefly speaking, we know that its dynamics is of the form \begin{equation} r_t = \kappa (\theta - r_t) ...
user avatar
0 votes
0 answers
12 views

FIML in growth mixture modelling

I have a question about Full Information Maximum Likelihood (FIML). I’m fitting growth mixture models to outcome variables measured at three timepoints. Some individuals are missing outcome data at ...
user avatar
  • 1
2 votes
1 answer
29 views

Estimating Mixture Models with Maximum Likelihood

Suppose you have a Normal Mixture Model with 2 Components - you could write this model as follows: $\pi_1 N(\mu_1, \sigma_1) + \pi_2 N(\mu_2, \sigma_2)$ In the above model, there are 6 unknowns : $\...
user avatar
  • 5,698
1 vote
1 answer
34 views

Why is it said that maximum likelihood becomes intractable if there are latent variables?

It is said that EM algorithm helps in cases where direct MLE cannot be carried out due to missing/latent variables. However, I could not understand why direct MLE cannot be carried out when there are ...
user avatar
  • 61
0 votes
0 answers
20 views

mgcv GAM models in R package caret - how to interpret output

I am attempting to evaluate two GAM models I developed in mgcv via leave-one-out cross validation in the caret package. I am a newbie to both GAMs and cross-validation. For the purposes of this ...
user avatar
  • 11
0 votes
0 answers
14 views

Maximum Likelihood Estimation Uniform Distribution [duplicate]

A sample is drawn from the uniform distribution: $X_{1},X_{2},...,X_{n}:U(0,2\theta )$ I am interested in finding the MLE for $\theta$. My intuition say that $\hat{\theta }=max(X_{i})/2$ How do I ...
user avatar
  • 418
0 votes
0 answers
8 views

Is there a good package to implement latent profile analysis in R, which allows use of FIML for missing data

We are looking to conduct latent profile analysis in R, using a large number of continuous variables (physical activity level at different times of day). We have objective measurement of physical ...
user avatar
1 vote
0 answers
23 views

Using restricted maximum likelihood on marginal residuals

In a mixed-model setting, I want to estimate the variance components and the pertaining random effects of a random-intercept/random-slope model. The coefficients of the fixed effects have already been ...
user avatar
1 vote
1 answer
29 views

Maximum Likelihood Estimation for data with non normal distribution [closed]

I have a set of data (results) that does not follow Normal distribution. In this case, wow can I get Maximum Likelihood Estimation? Thank you very much
user avatar
0 votes
0 answers
9 views

GPD and GEV Fitting: Maximum Likelihood vs. Least Squares

I am trying to build a model based on real world data which involves fitting generalized extreme value distributions and generalized Pareto distributions. Most literature immediately turns to the ...
user avatar
0 votes
0 answers
10 views

Renewal counting process with inter-arrival time gamma distribution: Model estimation

Let's start with the Poisson process: If $N_t$ is a Poisson process with parameter $\lambda$, then we know that the inter-arrival time distribution is an exponential distribution with parameter $\...
user avatar
2 votes
1 answer
34 views

If $X_1 \sim \text{binom}(p_1,n_1)$ and $X_2 \sim \text{binom}(p_2,n_2)$, how to prove that the MLE of $p = p_1 - p_2$ is $\hat{p}_1 - \hat{p}_2$?

Suppose $X_1 \sim \text{binom}(p_1,n_1)$ and $X_2 \sim \text{binom}(p_2,n_2)$, where $X_1$ and $X_2$ are independent, and let $p = p_1 - p_2$. How can I prove that $\hat{p} = \hat{p}_1 - \hat{p}_2$? (...
user avatar
-1 votes
0 answers
28 views

Integration in R: "longer object length is not a multiple of shorter object length" Error [migrated]

I am trying to use R to calculate the marginal likelihood of a set of data (likelihood_N) with respect to the parameter $v_0$. To do this, I created the following ...
user avatar
  • 109
0 votes
0 answers
8 views

REML for multilevel modeling comparison

With the very same data, I want to compare the two multilevel models. (these two models differs in both fixed and random effects) so, I am planning simulation study. I will generate the data set, and ...
user avatar
  • 109
0 votes
0 answers
17 views

Equivalent ways of writing the log-likelihood of a sample of normal RVs

I am going through my econometrics textbook right now and the textbook writes the log-likelihood equation for a sample of normal random variables in a way I have never seen before. Specifically, for a ...
user avatar
  • 135
0 votes
0 answers
12 views

verify minimizing the summation of the Kullback–Leibler is equal to maximum likelihood estimate for theta

I wondered it seems like the maximum likelihood equals minimizing $\sum_x P(x)\log{P(x)}-log{\hat P(x)}$,as P(X) is the truth but why it equals to $\sum_x P(x)\log{P(x)}-P(x)log{\hat P(x)}$?I cannot ...
user avatar
  • 1
3 votes
0 answers
65 views

$Y_1, Y_2, ... $ are iid Poisson($\lambda$)

We observe $Y_1, Y_2, ..., Y_T$ such that $T$ is the first $t\geq 1$ for which $Y_t>0$. Define $Y=Y_T$ (a) Find MLE $\hat{\lambda}$. (b) What is the relative bias, $[E(\hat{\lambda})- \lambda]/\...
user avatar
  • 31
0 votes
0 answers
14 views

How to estimate Pearson type-IV-GARCH using MLE in R?

I am just wondering whether there is an R program that can be applied to run GARCH specifications with Pearson types IV distributions. If you are familiar with any or can guide, it is greatly ...
user avatar
  • 1
0 votes
0 answers
15 views

t-Copula MLE on nu (DoF) only - log-likelihood function possibly convex?

I am working with t-Copula's to generate random synthetic data eventually. The paper I use as the foundation is Benali et al., 2021. To determine the best fitting t-Copula, they propose determining ...
user avatar
0 votes
0 answers
13 views

emmeans, should I use ML or REML fitted models [duplicate]

I see a lot of examples out there which use ML or REML fitted models as an input for emmeans. If I understand correctly for model comparison of fixed effects for "fixed effects nested models"...
user avatar
0 votes
0 answers
19 views

Self-Study: Score Statistic of a Poisson

I have been attempting a question on Generalised Linear Models. I thought I understood it but doing exercises myself now, proves otherwise. I have attached an image of the question here, struggling ...
user avatar
0 votes
0 answers
22 views

What is the limiting distribution of $$R_n=2(l(\hat{\theta})-l(\tilde{\theta}))$$ under null model $H_0$ for Poisson?

If $X_1,\dots, X_n\sim Poisson (\theta_1)$ and $Y_1,\dots, Y_n\sim Poisson (\theta_2)$, and $X_i$ is independent of $Y_i$, we want to test $$ H_0: \theta_1=\theta_2, \, H_a: \theta_1\neq \theta_2 $$ ...
user avatar
  • 232
0 votes
0 answers
43 views

Determine the asymptotic distribution $\sqrt{n}(\hat{\theta}_n-\theta) $ for trinomial distribution over the group sizes $(x,y,z)$

A random sample of $n$ individuals are classified into three groups, with probabilities $\theta^2$, $2\theta(1-\theta)$, and $(1-\theta)^2$ respectively, yielding the trinomial distribution over the ...
user avatar
  • 232
0 votes
0 answers
15 views

Is the likelihood of a discrete binomial variable same as it probability? Like in the case of tossing a coin lets say 12 times

I am working on a probability project where we first generate random variables for a given Binomial experiment and then we generate a PMF for 10 coin tosses using the list of random variables we ...
user avatar
  • 1
3 votes
1 answer
91 views

Is the likelihood for Gaussian mixture models still multimodal when Y is partially observed?

In discussing Gaussian mixture models (GMMs), https://normaldeviate.wordpress.com/2012/08/04/mixture-models-the-twilight-zone-of-statistics/ highlights the issue of Multimodality of the Likelihood. ...
user avatar
  • 3,768
0 votes
0 answers
28 views

MLE for sample mean estimation in practice

I usually use median for a truncated normal distribution mean estimate. In theory the best approach is to do gradient descent optimizing the mean for data generation. Does anyone have a good example ...
user avatar
  • 111
0 votes
0 answers
42 views

Proportion data: Logistic with MLE vs. OLS with logit-transformed response

This is an expansion of @Beethoven_90's comment on this question. Suppose I have proportion data $Y_i$ computed from a binomial; $Y_i = \frac{S_i}{N_i}$ where $S_i \sim Bin(N_i, p_i)$ and $p_i$ is the ...
user avatar
  • 1
0 votes
0 answers
25 views

Why is it said, that GEE is not likelihood based, IF the estimating equations are derived exactly thru d(log(L)/dB?

I found this article: https://sakai.unc.edu/access/content/group/2842013b-58f5-4453-aa8d-3e01bacbfc3d/public/Ecol562_Spring2012/docs/lectures/lecture22.htm#choosing and most of it is showing the ...
user avatar
  • 41
0 votes
0 answers
12 views

Parameter Estimation on S1 model

I have the following model of creating a random graph on the circle: First, N nodes are uniformly distributed on the circle of radius $N/(2\pi)$ to give a node density of $1$. We sample the expected ...
user avatar
1 vote
1 answer
48 views

Why cannot MLE be implemented for Gaussian mixture model directly?

Consider the following density, the mixture of two Gaussian distributions, \begin{align*} p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0) , \end{align*} where $p(k=1)+p(k=0)=\pi_1+\...
user avatar
  • 95
0 votes
0 answers
20 views

How to estimate the parameters of a hypergeomtric distribution?

I am looking for some guidance on how to estimate the parameters of a hyper-geometric distribution, based on a random sample. For example, if I generate a distribution as follows: ...
user avatar
1 vote
0 answers
20 views

MLE estimates of regime shifted discrete process

I have a discrete process as defined, $y_t$ = $a_1$$x_t$ + $b_1$ + $\mathcal{N}(0,\alpha)$ when $t <= \theta$ and $y_t$ = $a_2$$x_t$ + $b_2$ + $\mathcal{N}(0,\alpha)$ when $t > \theta$ for $t=0,...
user avatar
-1 votes
1 answer
154 views

How to get the $95\%$ Wald test confidence interval for $\theta$? [closed]

Suppose that iid random samples $X_i$ from a discrete CDF $F(x)$ on $\{x_1,\dots, x_n\}$ with mean $EX=\theta$. We want to estimate $F(x)$. We consider empirical likelihood for $F(x)=\sum_{i=1}^n p_i ...
user avatar
  • 232
0 votes
1 answer
57 views

How to use R to plot $f_n(\theta)$ based on following samples? [closed]

For iid random samples $X_1,\dots, X_n$ with mean value $E X_1=\theta$, take $X_{(1)}=\min \{X_1,\dots, X_n\}$ and $X_{(n)}=\max \{X_1,\dots, X_n\}$. If $\lambda$ solves equation $$ \sum_{i=1}^n \frac{...
user avatar
  • 232
1 vote
1 answer
40 views

Issue with Casella&Berger derivation of EM likelihood equality

In the explanation of the EM (Expectation maximization) algorithm p.328 in the book "Statistical inference" by G. Casella and R. Berger, 2nd edition, they present the following: $\mathbf{Y} =...
user avatar
0 votes
0 answers
29 views

Rao Cramèr Lower Bound problem

Let $X_1, · · · , X_n$ be a random sample from the uniform distribution on $[0, θ]$. I want to get the variance of the maximum likelihood estimator of $θ$ and check whether the variance decrease at ...
user avatar
  • 21
0 votes
0 answers
33 views

marginal likelihood in relevance vector machines RVM

I need to do a fit of 1d response versus 1d input, both real quantities. I wanted to implement Linear Regression, but because I wanted uncertainty in the resulting fit-params, I studied about Bayesian ...
user avatar
  • 101
1 vote
2 answers
74 views

Why can you not estimate Beta with least squares in logistic regression? [duplicate]

As the title says, why can you not estimate Beta (coefficients) in logistic regression with least-squares? I read in a book that it is not possible and that we use maximum likelihood instead, but I ...
user avatar
0 votes
0 answers
46 views

Likelihood-based CI is better than Wald CI?

The text I'm reading mentions the following: Let us consider a scalar parameter case. The likelihood-based approximate 95% cI is $$\left\{\theta,2\log\frac{L(\hat{\theta})}{L(\theta)} \le3.84 \right\}$...
user avatar

1
2 3 4 5
61