Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

In theory, does the form of the latent dynamics matter?

I am experimenting with latent variable models of the form $\mathbf{x}_t = \mathbf{W}\mathbf{z}_t + \mathcal{E}_t$ where $\mathcal{E}_t \sim \mathcal{N}(0,\sigma^2\mathbf{I})$ and $\mathbf{z}_t \sim \...
2
votes
0answers
12 views

How do I calculate the log-likelihood for prospect theory function?

I want to estimate parameters of prospect theory using Maximum Likelihood. My data structure and my estimation procedure closely follow this paper. I attached my R code below. ...
0
votes
0answers
20 views

Maximum Likelihood in the Markov Switching GARCH(1,1) Model

In the standard GARCH(1, 1) model with normal innovations: $${\displaystyle ~\epsilon _{t}=\sigma _{t}z_{t}},$$ $$\sigma^2_t=\omega+\alpha\epsilon^2_{t-1}+\beta\sigma^2_{t-1}.$$ The (negative) log-...
3
votes
1answer
56 views

Estimate MLE of discrete distribution with two parameters in R [closed]

I want to estimate the MLE of a discrete distribution in R using a numeric method. My data looks like this: data1<-c(5,2,2,3,0,2,1 2,4,4,1) If we assume it ...
1
vote
1answer
34 views

Is there any way or function in R to find the derivatives of incomplete gamma function or is it possible to obtain its derivative manually?

I am working with a probability distribution and I have to find the derivative of incomplete gamma function as \begin{equation*} \Gamma(\frac{\theta}{\beta}x^{2},\theta) = \int_{0}^{\frac{\theta}{\...
3
votes
0answers
39 views

Correlation matrix as maximum likelihood estimator under constraint

The Problem Although it seems to be straight forward I am struggling to prove the following statement. Assume, we have $p$-variate Gaussian observations $\left\{x_1, \ldots, x_N \right\} \subset \...
0
votes
0answers
21 views

MLE estimator for Poisson Variable [closed]

Let $X_1,\dots,X_n$ be an random sample of a Poisson process with unknow mean $\mu$. How can we find an estimator for $p(0)+p(1)$? This is the first time i see an exercise like that, so i don't have ...
1
vote
0answers
43 views

Why does setting the derivative of a likelihood function equal to 0 maximize the likelihood function? [migrated]

I'm learning from a statistics tutorial which defines a likelihood function as \begin{align} L(1,3,2,2; \theta)=27 \cdot \theta^{8} (1-\theta)^{4} \tag{1} \end{align} and then the tutorial sets the ...
5
votes
1answer
70 views

Finding the MLE of Uniform distribution [duplicate]

Let $x_{1} = 2.4$ , $x_{2} = 9.2$ , $x_{3} = 5.2$ , $x_{4} = 4.1$ , $x_{5} = 2.1$, $x_{6} = 3.1$ be the observed values of a random variable of size 6 from the uniform distribution with parameters $(\...
1
vote
2answers
37 views

How do I determine the appropriate likelihood function for a custom CDF

I'm doing some work describing how things dissolve in solution, and I've determined that a particular parameterization of the 3P Weibull provides a good fit. Now what I want to do is specify target ...
0
votes
1answer
15 views

How do you find the point where two writers of text meet?

It is said that I have a text that I am familiar with, with author X writing the beginning and author Y writing the end. I'm not sure where the line between the authors should be drawn. The text is ...
0
votes
1answer
15 views

likelihood function of tobit model when y can only be 0 or 1

As you can see, , here is the likelihood function of tobit model when observed y equals 0 when it is smaller then $y_L$. What is the form when observed y equals 0 when it is smaller then $y_L$, AND y ...
0
votes
0answers
41 views

Is it possibe to find an estimator for variance of interest using least squared estimation in a linear model?

In a linear model, maximum likelihood estimation (MLE) provides estimator for both mean of insterest and variance of interest, but least squared estimation (LSE) just provides estimator for the mean ...
1
vote
0answers
9 views

Likelihood vs quasi-likelihood and restricted likelihood

I understand that the purpose of likelihood is to serve as an estimation mechanism that returns the model parameters which would most likely re-produce the observed data (given some model or ...
0
votes
1answer
64 views

Test whether two separate Weibull distributions describe the data better than a single Weibull distribution

My question is very similar to this one, but with the Weibull distribution replacing the Poisson distribution. Let's say I am analyzing the distribution of times between failures for an engine, with ...
1
vote
2answers
62 views

Numerical MLE for Rayleigh distribution

I am given a rayleigh distribution described by, =$f\left(x|\theta\right)\:=\:\frac{x}{\theta ^2}e^{-\left(\frac{x^2}{2\theta ^2}\right)}$ I need to find a numerical estimate of the MLE of $\theta^2$ ...
3
votes
1answer
50 views

Variance of the $\hat{\sigma}$ of a Maximum likelihood estimator

With a random sample having size $n$ from a normal distribution $\mathcal{N}(\mu,\sigma^2)$, and the ML estimate of $$g(\mu,\sigma^2) = \mu +\sigma$$ being $$\widehat{g(\mu,\sigma^2)}=\overline{x}+\...
0
votes
2answers
60 views

Find asymptotic efficiency of MLE to UMVUE

Let $\{X_i\}_{i=1}^n $ be a sequence of i.i.d random variables with common pdf: $$ f(x;a,\theta) =\theta a^\theta x^{-(\theta+1)} \boldsymbol 1_{(a,\infty)}(x) \, \,\text{; where } \theta, a > 0$$ ...
0
votes
0answers
11 views

Handle numerical error in bootstrapping to compute Wald confidence interval

Let $f(\theta|x)$ be the likelihood function of a random variable $x$ with parameter $\theta$. Due to complexity, I have maximized $f$ using a Newton-Raphson method with a bounded interval. This ...
4
votes
1answer
54 views

Meaning of Invariance of Maximum Likelihood Estimator

In Casella-Berger, the invariance of MLE is defined as: Assuming that $\hat{\theta}$ is MLE of $\theta$, then for any function $\tau$, $\tau(\hat{\theta})$ is MLE of $\tau(\theta)$. In the case of a ...
0
votes
0answers
35 views

Understanding difference between Maximum Likelihood and Levenberg Marquardt result

In some of my regression results I noticed a deviation between Maximum Likelihood (via Monte Carlo Markov Chain, initialised by parameter result of Nelder-Mead, median value pictured) result and ...
0
votes
0answers
9 views

Can unobserved heterogeneity with factor loading identified in MLE?

I have a question on the identifiability when I do maximum likelihood estimation with logit model. I use discrete factor random effect model for the unobserved heterogeneity. B is a binary outcome of ...
0
votes
1answer
69 views

$(x_1, x_2, x_3, x_4)=(1,0,1,1)$ means there are 4 observed values, blue for the 1st observation … What could (x_1,x_2,x_3,x_4)=(1,3,2,2) mean?

Assume a bag that contains 3 balls. Each ball is either red or blue. The number of blue balls, call it θ, might be 0, 1, 2, or 3. Choose 4 balls at random from the bag with replacement. define the i.i....
1
vote
1answer
50 views

Unknown parameter - augmenting state equation (Kalman filter)

First, we have a state space model with mean reversion and $\mu$ is unknown $y(t )= F* x_t +e_t$ $x_t- \mu = G* (x_{t-1}-\mu) +n_t$ There is a option to add unknown parameters to the state vector and ...
0
votes
0answers
12 views

Interpretation of test set negative log likelihood in neural density estimation applications

I have seen people splitting a dataset into a train and test sets and learning the parameters of a mixture density network using the negative log likelihood cost function on the train set and ...
0
votes
0answers
20 views

Understanding maximum likelihood estimate in the context without any specific distribution and different data sets

I came across following difference in probabilities vs maximum likelihood estimated (MLE) in this video: So, basically, in finding probability, we are finding the "probability of different data ...
0
votes
1answer
91 views

Doing MLE when data is missing not at random

Suppose $X, Y, S$, and $Y=f_\theta(X), S=f_\phi(Y), S \in \{0,1\}$. For $n$ data samples, the $y_i$ is only observed for those who have $s_i=1$. That is, we have $\{x_i, y_i, s_i=1\}_{i=1}^{l}$ and $\{...
2
votes
1answer
58 views

Find the MLE of $\theta$ for $X_1, X_2, ..., X_n$ with density function $f(x|\theta) = e^{-(x-\theta)}$, $x \ge \theta$ [duplicate]

Suppose that $X_1, X_2, ..., X_n$ are i.i.d with density function $$ f(x|\theta) = e^{-(x-\theta)}, \quad \quad x\ge \theta $$ and $f(x|\theta) = 0$ otherwise. Find the mle of $\theta$. (Hint: Be ...
0
votes
0answers
31 views

Maximum Likelihood For the Normal Distribution

I get using Maximum Likelihood Estimation to find unknown parameters of a function. But in the normal distribution, we know probability density function is f(x)=1/σ√2π(e^−(x−μ)2/(2σ^2)) where μ is ...
2
votes
1answer
36 views

Parameters in Naive Bayes

This is from https://scikit-learn.org/stable/modules/naive_bayes.html In the last line it says and we can use Maximum A Posteriori (MAP) estimation to estimate $P(y)$ and $P(x_i|y)$; the former is ...
2
votes
0answers
18 views

Problem Implementing MLE Estimation in bbmle Package (for R) [migrated]

I am trying to verify the MLEs obtained for $\alpha$, $\beta$ and $\lambda$ for the Logistic-Lomax distribution in the paper entitled A Study of Logistic-Lomax Distribution by Zubair et al when using ...
0
votes
0answers
13 views

Quasiprobabilities as weights in statistical analysis, with a motivating example for mixed distributions

$\def\b{\mathbf}\def\D{\DeclareMathOperator}\D{\Ind}{\b Δ}\D{\sWeights}{\hat{\b Δ}}\D{\sWeightsAlt}{\hat{\b δ}}\D{\diag}{diag}$Quasiprobability distributions, originating in some formulations of ...
5
votes
0answers
70 views

Maximum likelihood estimate for multivariate sum of normal distributions

For each $j = 1,\dots,N$, let $\mu_j \in \mathbb{R}^N$ denote a known column vector, $\Sigma_j \in \mathbb{R}^{N\times N}$ a known covariance matrix, and $\theta_j \in \mathbb{R}$ an unknown parameter,...
2
votes
1answer
31 views

Problem in Reproducing the MLEs for a Given Distribution

I am testing a code I am using in R to compute estimates for the MLEs of the parameters for a given distribution. As an example, to check if the code works, I have chosen the paper A New Two-parameter ...
3
votes
0answers
78 views

Label smoothing and KL divergence

I am reading the paper Regularizing Neural Networks by Penalizing Confident Output Distributions where the authors introduce label smoothing in section 3.2. For a neural network that produces a ...
2
votes
1answer
97 views

Better understanding Maximum Likelihood parameter estimation

Suppose I am trying to model the dependence of a variable $B$ on another variable $A$ by a function $B=f(A;k)$, where $k$ is a parameter, whose value I would like to estimate. Given $n$ observations $\...
3
votes
1answer
176 views

Students are assigned numbers 1 to n; 3 are chosen randomly which turn out to be 1,3,7. Given max(n) = 30 what is the MLE of n?

The students are drawn from a discrete uniform distribution. This is one of the exercise questions from mit.ocw. I can't quite figure out how to use the numbers selected (1,3,7) into the likelihood ...
2
votes
0answers
53 views

Maximum likelihood estimator and KL divergence

Let $\mathbf{X}$ be a continuous random vector, $\mathbf{d}$ a sample of size $m$, and $\mathbb{P}_{\mathbf{X}|\mathbf{\theta}}$ a parametric model for the distribution of $\mathbf{X}$. We can write ...
1
vote
0answers
48 views

Using a different (but related) hypothesis for the prior in MAP

Say we have a general set of data $\mathcal{D} = \{\mathbf{x}_i, \mathbf{y}_i \}_{i \in N}$ of covariates $\mathbf{x}$ and observations $\mathbf{y}$. Our problem is in fitting a known model $\mathbf{y}...
1
vote
0answers
37 views

Is it impossible to use P-spline with a small number of basic functions such as 5 and an internal knot?

I want to maximize penalized loglikelihood. At first, I set penalty matrix by integrating the square of second derivative of B-spline functions and I didn`t get good result. I used cubic B-spline and ...
0
votes
0answers
15 views

Derivation of estimates in Factor Analysis

Are the derivations made below regarding estimates for factors and the factor weights in Factor Analysis correct? Assume the following model for factor analysis: $$ y_i = Wx_i + \epsilon_i$$ where $...
0
votes
1answer
31 views

MLE of Variance of Normal Distribution Asymptotically Unbiased?

So the MLE of the variance of a normal distribution, $\sigma^2$, is just the mean squared error, i.e., $\frac{1}{N}\sum_{i=1}^{N} (\hat{y_i} - y_i)^2$. Clearly, this goes to $0$ as $n \rightarrow \...
0
votes
0answers
23 views

Do GANs compute a posterior distribution, and if not how do they have such good results with just MLE?

As the title states, do GANs (Generative Adversarial Networks) compute a posterior distribution? If they do not, how do they have such good results with just using MLE? Wouldn't they run into issues ...
0
votes
1answer
54 views

Estimate dispersion parameters in negative binomial distribution

A popular parameterization of the negative binomial distribution is by $\mu$ and $r$, which represent mean and dispersion, respectively. The probability mass function states: $$ \text{P(x = k)} = \...
0
votes
0answers
19 views

Where to find information on likelihood ratio tests [duplicate]

Can someone please recommend a good resource that explains the theory and logic behind likelihood ratio tests?
2
votes
1answer
41 views

Bias of MLE scales with $1/N$?

I was reading this paper (link) and it gave me some confusion. $P(r|\theta)$ is a distribution that generates sample $r$ based on some Poisson distribution, whose mean and variance are defined as some ...
0
votes
1answer
37 views

Using PDF values for Likelihood [duplicate]

Given that PDF value $𝑓_𝑋(𝑥)$ for a particular $𝑥=𝑥_1$ does not have any probabilistic meaning (by definition $𝑝(𝑥=𝑥_1)=0$). We still see the use of $𝑓_𝑋(𝑥_1)$ as its likelihood. My ...
0
votes
0answers
19 views

Using Sigmoid in Maximum Likelihood Estimates [duplicate]

I have two questions regarding the use of Sigmoid in MLE: Clearly, the Sigmoid Function is not a PDF. But in the MLE of Logistic Regression, we see Sigmoid being used as if it is a PDF. Is my ...
0
votes
1answer
21 views

Quasi-likelihood can't be generated by any valid probability distribution

I am learning about quasi-Poisson and i'm stuck at the concept of quasi-likelihood function. In wikipedia, it is said that: The term quasi-likelihood function was introduced by Robert Wedderburn in ...
1
vote
1answer
59 views

What does "Expectation with respect to true unknown parameter" mean?

I am trying to study the asymptotic properties of MLE, but I am having trouble understanding an expression that seems to be consistently used in all lecture notes available online (page 93,page 18,...

1
2 3 4 5
58