Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
2,855
questions
0
votes
0answers
10 views
In theory, does the form of the latent dynamics matter?
I am experimenting with latent variable models of the form
$\mathbf{x}_t = \mathbf{W}\mathbf{z}_t + \mathcal{E}_t$ where $\mathcal{E}_t \sim \mathcal{N}(0,\sigma^2\mathbf{I})$ and $\mathbf{z}_t \sim \...
2
votes
0answers
12 views
How do I calculate the log-likelihood for prospect theory function?
I want to estimate parameters of prospect theory using Maximum Likelihood. My data structure and my estimation procedure closely follow this paper. I attached my R code below.
...
0
votes
0answers
20 views
Maximum Likelihood in the Markov Switching GARCH(1,1) Model
In the standard GARCH(1, 1) model with normal innovations:
$${\displaystyle ~\epsilon _{t}=\sigma _{t}z_{t}},$$
$$\sigma^2_t=\omega+\alpha\epsilon^2_{t-1}+\beta\sigma^2_{t-1}.$$
The (negative) log-...
3
votes
1answer
56 views
Estimate MLE of discrete distribution with two parameters in R [closed]
I want to estimate the MLE of a discrete distribution in R using a numeric method.
My data looks like this:
data1<-c(5,2,2,3,0,2,1 2,4,4,1)
If we assume it ...
1
vote
1answer
34 views
Is there any way or function in R to find the derivatives of incomplete gamma function or is it possible to obtain its derivative manually?
I am working with a probability distribution and I have to find the derivative of incomplete gamma function as
\begin{equation*}
\Gamma(\frac{\theta}{\beta}x^{2},\theta) = \int_{0}^{\frac{\theta}{\...
3
votes
0answers
39 views
Correlation matrix as maximum likelihood estimator under constraint
The Problem
Although it seems to be straight forward I am struggling to prove the following statement.
Assume, we have $p$-variate Gaussian observations $\left\{x_1, \ldots, x_N \right\} \subset \...
0
votes
0answers
21 views
MLE estimator for Poisson Variable [closed]
Let $X_1,\dots,X_n$ be an random sample of a Poisson process with unknow mean $\mu$. How can we find an estimator for $p(0)+p(1)$?
This is the first time i see an exercise like that, so i don't have ...
1
vote
0answers
43 views
Why does setting the derivative of a likelihood function equal to 0 maximize the likelihood function? [migrated]
I'm learning from a statistics tutorial which defines a likelihood function as
\begin{align}
L(1,3,2,2; \theta)=27 \cdot \theta^{8} (1-\theta)^{4} \tag{1}
\end{align}
and then the tutorial sets the ...
5
votes
1answer
70 views
Finding the MLE of Uniform distribution [duplicate]
Let $x_{1} = 2.4$ , $x_{2} = 9.2$ , $x_{3} = 5.2$ , $x_{4} = 4.1$ , $x_{5} = 2.1$, $x_{6} = 3.1$ be the observed values of a random variable of size 6 from the uniform distribution with parameters $(\...
1
vote
2answers
37 views
How do I determine the appropriate likelihood function for a custom CDF
I'm doing some work describing how things dissolve in solution, and I've determined that a particular parameterization of the 3P Weibull provides a good fit. Now what I want to do is specify target ...
0
votes
1answer
15 views
How do you find the point where two writers of text meet?
It is said that I have a text that I am familiar with, with author X writing the beginning and author Y writing the end. I'm not sure where the line between the authors should be drawn. The text is ...
0
votes
1answer
15 views
likelihood function of tobit model when y can only be 0 or 1
As you can see, , here is the likelihood function of tobit model when observed y equals 0 when it is smaller then $y_L$.
What is the form when observed y equals 0 when it is smaller then $y_L$, AND y ...
0
votes
0answers
41 views
Is it possibe to find an estimator for variance of interest using least squared estimation in a linear model?
In a linear model, maximum likelihood estimation (MLE) provides estimator for both mean of insterest and variance of interest, but least squared estimation (LSE) just provides estimator for the mean ...
1
vote
0answers
9 views
Likelihood vs quasi-likelihood and restricted likelihood
I understand that the purpose of likelihood is to serve as an estimation mechanism that returns the model parameters which would most likely re-produce the observed data (given some model or ...
0
votes
1answer
64 views
Test whether two separate Weibull distributions describe the data better than a single Weibull distribution
My question is very similar to this one, but with the Weibull distribution replacing the Poisson distribution.
Let's say I am analyzing the distribution of times between failures for an engine, with ...
1
vote
2answers
62 views
Numerical MLE for Rayleigh distribution
I am given a rayleigh distribution described by,
=$f\left(x|\theta\right)\:=\:\frac{x}{\theta ^2}e^{-\left(\frac{x^2}{2\theta ^2}\right)}$
I need to find a numerical estimate of the MLE of $\theta^2$ ...
3
votes
1answer
50 views
Variance of the $\hat{\sigma}$ of a Maximum likelihood estimator
With a random sample having size $n$ from a normal distribution $\mathcal{N}(\mu,\sigma^2)$, and the ML estimate of $$g(\mu,\sigma^2) = \mu +\sigma$$ being $$\widehat{g(\mu,\sigma^2)}=\overline{x}+\...
0
votes
2answers
60 views
Find asymptotic efficiency of MLE to UMVUE
Let $\{X_i\}_{i=1}^n $ be a sequence of i.i.d random variables with common pdf:
$$ f(x;a,\theta) =\theta a^\theta x^{-(\theta+1)} \boldsymbol 1_{(a,\infty)}(x) \, \,\text{; where } \theta, a > 0$$
...
0
votes
0answers
11 views
Handle numerical error in bootstrapping to compute Wald confidence interval
Let $f(\theta|x)$ be the likelihood function of a random variable $x$ with parameter $\theta$. Due to complexity, I have maximized $f$ using a Newton-Raphson method with a bounded interval. This ...
4
votes
1answer
54 views
Meaning of Invariance of Maximum Likelihood Estimator
In Casella-Berger, the invariance of MLE is defined as:
Assuming that $\hat{\theta}$ is MLE of $\theta$, then for any function
$\tau$, $\tau(\hat{\theta})$ is MLE of $\tau(\theta)$.
In the case of a ...
0
votes
0answers
35 views
Understanding difference between Maximum Likelihood and Levenberg Marquardt result
In some of my regression results I noticed a deviation between Maximum Likelihood (via Monte Carlo Markov Chain, initialised by parameter result of Nelder-Mead, median value pictured) result and ...
0
votes
0answers
9 views
Can unobserved heterogeneity with factor loading identified in MLE?
I have a question on the identifiability when I do maximum likelihood estimation with logit model. I use discrete factor random effect model for the unobserved heterogeneity.
B is a binary outcome of ...
0
votes
1answer
69 views
$(x_1, x_2, x_3, x_4)=(1,0,1,1)$ means there are 4 observed values, blue for the 1st observation … What could (x_1,x_2,x_3,x_4)=(1,3,2,2) mean?
Assume a bag that contains 3 balls. Each ball is either red or blue. The number of blue balls, call it θ, might be 0, 1, 2, or 3. Choose 4 balls at random from the bag with replacement. define the i.i....
1
vote
1answer
50 views
Unknown parameter - augmenting state equation (Kalman filter)
First, we have a state space model with mean reversion and $\mu$ is unknown
$y(t )= F* x_t +e_t$
$x_t- \mu = G* (x_{t-1}-\mu) +n_t$
There is a option to add unknown parameters to the state vector and ...
0
votes
0answers
12 views
Interpretation of test set negative log likelihood in neural density estimation applications
I have seen people splitting a dataset into a train and test sets and learning the parameters of a mixture density network using the negative log likelihood cost function on the train set and ...
0
votes
0answers
20 views
Understanding maximum likelihood estimate in the context without any specific distribution and different data sets
I came across following difference in probabilities vs maximum likelihood estimated (MLE) in this video:
So, basically, in finding probability, we are finding the "probability of different data ...
0
votes
1answer
91 views
Doing MLE when data is missing not at random
Suppose $X, Y, S$, and $Y=f_\theta(X), S=f_\phi(Y), S \in \{0,1\}$. For $n$ data samples, the $y_i$ is only observed for those who have $s_i=1$.
That is, we have $\{x_i, y_i, s_i=1\}_{i=1}^{l}$ and $\{...
2
votes
1answer
58 views
Find the MLE of $\theta$ for $X_1, X_2, ..., X_n$ with density function $f(x|\theta) = e^{-(x-\theta)}$, $x \ge \theta$ [duplicate]
Suppose that $X_1, X_2, ..., X_n$ are i.i.d with density function
$$
f(x|\theta) = e^{-(x-\theta)}, \quad \quad x\ge \theta
$$
and $f(x|\theta) = 0$ otherwise.
Find the mle of $\theta$. (Hint: Be ...
0
votes
0answers
31 views
Maximum Likelihood For the Normal Distribution
I get using Maximum Likelihood Estimation to find unknown parameters of a function.
But in the normal distribution, we know probability density function is f(x)=1/σ√2π(e^−(x−μ)2/(2σ^2)) where μ is ...
2
votes
1answer
36 views
Parameters in Naive Bayes
This is from https://scikit-learn.org/stable/modules/naive_bayes.html
In the last line it says
and we can use Maximum A Posteriori (MAP) estimation to estimate
$P(y)$ and $P(x_i|y)$; the former is ...
2
votes
0answers
18 views
Problem Implementing MLE Estimation in bbmle Package (for R) [migrated]
I am trying to verify the MLEs obtained for $\alpha$, $\beta$ and $\lambda$ for the Logistic-Lomax distribution in the paper entitled A Study of Logistic-Lomax Distribution by Zubair et al when using ...
0
votes
0answers
13 views
Quasiprobabilities as weights in statistical analysis, with a motivating example for mixed distributions
$\def\b{\mathbf}\def\D{\DeclareMathOperator}\D{\Ind}{\b Δ}\D{\sWeights}{\hat{\b Δ}}\D{\sWeightsAlt}{\hat{\b δ}}\D{\diag}{diag}$Quasiprobability distributions, originating in some formulations of ...
5
votes
0answers
70 views
Maximum likelihood estimate for multivariate sum of normal distributions
For each $j = 1,\dots,N$, let $\mu_j \in \mathbb{R}^N$ denote a known column vector, $\Sigma_j \in \mathbb{R}^{N\times N}$ a known covariance matrix, and $\theta_j \in \mathbb{R}$ an unknown parameter,...
2
votes
1answer
31 views
Problem in Reproducing the MLEs for a Given Distribution
I am testing a code I am using in R to compute estimates for the MLEs of the parameters for a given distribution. As an example, to check if the code works, I have chosen the paper A New Two-parameter ...
3
votes
0answers
78 views
Label smoothing and KL divergence
I am reading the paper Regularizing Neural Networks by Penalizing Confident Output Distributions where the authors introduce label smoothing in section 3.2. For a neural network that produces a ...
2
votes
1answer
97 views
Better understanding Maximum Likelihood parameter estimation
Suppose I am trying to model the dependence of a variable $B$ on another variable $A$ by a function $B=f(A;k)$, where $k$ is a parameter, whose value I would like to estimate. Given $n$ observations $\...
3
votes
1answer
176 views
Students are assigned numbers 1 to n; 3 are chosen randomly which turn out to be 1,3,7. Given max(n) = 30 what is the MLE of n?
The students are drawn from a discrete uniform distribution.
This is one of the exercise questions from mit.ocw. I can't quite figure out how to use the numbers selected (1,3,7) into the likelihood ...
2
votes
0answers
53 views
Maximum likelihood estimator and KL divergence
Let $\mathbf{X}$ be a continuous random vector, $\mathbf{d}$ a sample of size $m$, and $\mathbb{P}_{\mathbf{X}|\mathbf{\theta}}$ a parametric model for the distribution of $\mathbf{X}$.
We can write ...
1
vote
0answers
48 views
Using a different (but related) hypothesis for the prior in MAP
Say we have a general set of data $\mathcal{D} = \{\mathbf{x}_i, \mathbf{y}_i \}_{i \in N}$ of covariates $\mathbf{x}$ and observations $\mathbf{y}$. Our problem is in fitting a known model $\mathbf{y}...
1
vote
0answers
37 views
Is it impossible to use P-spline with a small number of basic functions such as 5 and an internal knot?
I want to maximize penalized loglikelihood. At first, I set penalty matrix by integrating the square of second derivative of B-spline functions and I didn`t get good result. I used cubic B-spline and ...
0
votes
0answers
15 views
Derivation of estimates in Factor Analysis
Are the derivations made below regarding estimates for factors and the factor weights in Factor Analysis correct?
Assume the following model for factor analysis:
$$ y_i = Wx_i + \epsilon_i$$
where $...
0
votes
1answer
31 views
MLE of Variance of Normal Distribution Asymptotically Unbiased?
So the MLE of the variance of a normal distribution, $\sigma^2$, is just the mean squared error, i.e., $\frac{1}{N}\sum_{i=1}^{N} (\hat{y_i} - y_i)^2$. Clearly, this goes to $0$ as $n \rightarrow \...
0
votes
0answers
23 views
Do GANs compute a posterior distribution, and if not how do they have such good results with just MLE?
As the title states, do GANs (Generative Adversarial Networks) compute a posterior distribution?
If they do not, how do they have such good results with just using MLE? Wouldn't they run into issues ...
0
votes
1answer
54 views
Estimate dispersion parameters in negative binomial distribution
A popular parameterization of the negative binomial distribution is by $\mu$ and $r$, which represent mean and dispersion, respectively. The probability mass function states:
$$
\text{P(x = k)} = \...
0
votes
0answers
19 views
Where to find information on likelihood ratio tests [duplicate]
Can someone please recommend a good resource that explains the theory and logic behind likelihood ratio tests?
2
votes
1answer
41 views
Bias of MLE scales with $1/N$?
I was reading this paper (link) and it gave me some confusion.
$P(r|\theta)$ is a distribution that generates sample $r$ based on some Poisson distribution, whose mean and variance are defined as some ...
0
votes
1answer
37 views
Using PDF values for Likelihood [duplicate]
Given that PDF value $𝑓_𝑋(𝑥)$ for a particular $𝑥=𝑥_1$ does not have any probabilistic meaning (by definition $𝑝(𝑥=𝑥_1)=0$). We still see the use of $𝑓_𝑋(𝑥_1)$ as its likelihood.
My ...
0
votes
0answers
19 views
Using Sigmoid in Maximum Likelihood Estimates [duplicate]
I have two questions regarding the use of Sigmoid in MLE:
Clearly, the Sigmoid Function is not a PDF. But in the MLE of Logistic Regression, we see Sigmoid being used as if it is a PDF. Is my ...
0
votes
1answer
21 views
Quasi-likelihood can't be generated by any valid probability distribution
I am learning about quasi-Poisson and i'm stuck at the concept of quasi-likelihood function.
In wikipedia, it is said that:
The term quasi-likelihood function was introduced by Robert Wedderburn in ...
1
vote
1answer
59 views
What does "Expectation with respect to true unknown parameter" mean?
I am trying to study the asymptotic properties of MLE, but I am having trouble understanding an expression that seems to be consistently used in all lecture notes available online (page 93,page 18,...