Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
2
votes
1answer
91 views
Question regarding MLE
I have a question regarding maximum likelihood estimators/functions. I thought I understood the concept. But now i found an example of an Maximum likelihood function that i don’t get.
The question is ...
-1
votes
0answers
11 views
Error of MLE of Gamma distribution [on hold]
Error in the MLE, I don't know how to fix that, please help me!!!!!
0
votes
0answers
14 views
Compartion of GLM models through log-likelihood, deviance and chi square
I'm studying GLM models in software R. I have a dataset with the follow distribution: age, sex, years of study (ys), road or hightway (usop), and
claims. I'm adjusting my model to claimns where it is ...
2
votes
1answer
36 views
On a mistake computing the Kullback Liebler Information Criterion
THE FRAMEWORK:
Let $X_1$ be an observation from a normal random variable with mean zero and variance $\sigma^2$ and lets call the PDF $f(x)$.
I want to minimize the Kullback Liebler Information ...
2
votes
1answer
182 views
MLE of the unknown radius
Consider this question,
Suppose that $(X_1, Y_1),(X_2, Y_2), . . . ,(X_n, Y_n)$ are the
coordinates of $n$ points chosen independently and uniformly at random
within a circle with center $(0, 0)...
2
votes
1answer
35 views
Transformation of MLE parameters standard deviations calculation
I have run an MLE estimation using R to estimate parameters $\theta_1, \theta_2, \theta_3$, which are:
0.0002022146 0.0222026625 -6.1910421067
the variance ...
0
votes
1answer
25 views
Iterative optimization of alternative glm family
I'm setting up an alternative response function to the commonly used exponential function in poisson glms, which is called softplus and defined as $\frac{1}{c} \log(1+\exp(c \eta))$, where $\eta$ ...
0
votes
0answers
19 views
Censored regression with Poisson distribution
I am trying to fit a Poisson distribution for left censored data. Let $x_1,x_2,...x_n$ be the observations with the first $r$ observations being less than the threshold of $c$ and hence censored. The ...
1
vote
2answers
55 views
Finding the MLE of Poisson in R [closed]
I'm trying to determine the MLE of $\lambda$ in a Poisson distribution using R. I'm aware that the MLE is $\hat{\lambda}=\bar{x}$ but I want to demonstrate this using Rmarkdown. My experience with R ...
1
vote
0answers
18 views
log-likelihood of normal distributed fitted using MLE
Suppose we fit a normal using MLE which means we have parameters $$\mu = \frac{1}{n}\sum_{i=1}^n x_i$$ and $$\sigma^2 = \frac{1}{n}\sum_{i=1}^n(x_i - \mu)^2$$
Then we calculate the log-likelihood as
...
1
vote
1answer
19 views
Parameter estimation in Dynamic Linear Models
I am currently developing a DLM of the following form
$$\underset{k \times 1} {y_t} = \underset{k \times n}A \underset{n \times 1}{\theta_t} + \epsilon_t$$
$$\theta_t = \mu + \underset{n \times n}B\...
0
votes
0answers
22 views
MLE for Beta distribution, with $\beta$ = 3
I'm trying to calculate the Maximum-Likelihood Estimator for $\alpha$, using the beta distribution with $\beta = 3$. I'm kind of stuck at the last bit. Perhaps I've made a mistake somewhere, or this ...
0
votes
0answers
22 views
Non-positive variance-covariance in a linear mixed model
I have some problem with my lienar mixed model.
When I do my model with the maximum likelihood, I have no error. But, when I want the confidence intervals, I get the Error : " cannot get confidence ...
-1
votes
0answers
36 views
Cross-validation on S-curve [closed]
Which approach/R package would you recommend for cross-validation on a S-curve where almost all of the data is in the lower half of the S.
I use piecewise linear sloped step as alternative model ...
1
vote
1answer
40 views
Is my understanding on how to estimate the parameters in a GARCH model correct?
Assume (for the sake of simplicity) we have observed only $X_1,X_2$ and we want to estimate the parameters of a GARCH(1,1) that tells us the variance of $X_t$ (that is normally distributed) evolves ...
0
votes
0answers
46 views
Plot the approximate the likelihood function around the MLE using Gaussian
Say we are looking to estimate $\theta$ for a variable with distribution
$f(x|\theta) = \frac{\theta}{x^{\theta+1}}$, $\theta>0$, $x>1$
The method of moments estimate for ${\theta}$ is $\hat{\...
0
votes
0answers
48 views
Kalman Filter with MLE giving bad estimates
I am trying to learn and implement the Kalman filter. Yesterday I successfully implemented a non linear kalman filter of the form:
$$
x_t = a(x_{t-1}) + u_t \\
y_t = Gy_{t-1} + v_t
$$
$u_t$ and $v_t$...
0
votes
0answers
24 views
Bayesian Information Criterion Formula Proof
while I was digging arima model I saw that BIC value is given as $k*log(n)-2*log(L)$ where $L$ is the maximized value of likelihood function whereas $k$ is number of parameters. I wonder how it is ...
0
votes
1answer
19 views
How do I estimate probability of success with no successes? [duplicate]
My $6$ friends and I tried buying tickets to a popular event. Everyone who wanted a ticket got a random number and if your number is less than or equal the number of tickets available, you can buy a ...
1
vote
1answer
27 views
Build an approximated confidence interval for $\sigma$ based on its maximum likelihood estimator
Let $X_{1},X_{2},\ldots,X_{n}$ be a random sample whose distribution is given by $\mathcal{N}(0,\sigma^{2})$. Build an approximated confidence interval for $\sigma$ based on its maximum likelihood ...
0
votes
1answer
139 views
Sample mean is always an optimal estimator of the mean?
Suppose we have $T_i,i=1..n$ i.i.d. with unknown distribution and we want to estimate $E[T]$. Note that in this setting we are not estimating E[T] as a parameter of a parameter-dependent family of ...
0
votes
0answers
6 views
What should the form of error be on CrossEntropy or KL-divergence loss function across samples of distributions?
Suppose your model produces (discrete) probability distributions and you have some truth distributions you want to compare to.
For each sample $i$, you can compute the loss as the KL divergence or ...
0
votes
0answers
8 views
Estimate weight parameter between two levels and make profile likelihood in R [closed]
I have to estimate the weight parameter between two levels in a feature (men and women) in a Gamma function with an identity linkfunction. Though I have not tried this before and therefore not very ...
3
votes
3answers
304 views
Is it correct to say the Neural Networks are an alternative way of performing Maximum Likelihood Estimation? if not, why? [duplicate]
We often say that minimizing the (negative) cross-entropy error is the same as maximizing the likelihood. So can we say that NN are just an alternative way of performing Maximum Likelihood Estimation? ...
0
votes
0answers
13 views
What are the validity test of the Maximum likelihood estimation?
I know that the OLS has the standard validity test for assumption violation but when it come to the MLE how can we tell that the coefficients are valid for the conclusion
2
votes
1answer
34 views
How does L2 penalize large weights
The L2 regularization term is useful because it penalizes large weights over smaller weights which is good to prevent overfitting. I'm having a hard time understanding how exactly it does this.
This ...
0
votes
1answer
12 views
How to make a prediction with Bayes Classifier after computing MLE?
I'm trying to figure out the role of computing the MLE for classification/prediction purposes with the Bayes Classifier.
Let's say I'm given a set of data assumed to be Gaussian. I can then compute ...
0
votes
1answer
15 views
Evaluating the rate of convergence for MLE of multinomial distribution
I'd like quantify how the number of trials $n$ influences the quality of an MLE estimate of multinomial distribution with even probabilities $p_1,...,p_k$. So we know that we can estimate the event ...
0
votes
1answer
37 views
How can you constrain values to be positive when fitting a model?
I'm currently fitting a model using maximum likelihood estimation on biological data (electroencephalography). Basically, I'm fitting normal distributions to several subsets of data (experimental ...
0
votes
0answers
47 views
R - Moving average; MA(2), Maximum-Likelihood estimation through optim routine
I am trying to complete my assignment for time-series where I have to use Nile data to fit MA(2) model and estimate theta coefficients through creation of new function and optimizing it to get ...
0
votes
0answers
25 views
How is the L2 regularization derived? [duplicate]
I just proved to myself why the regularization is added rather than multiplied to loss function.
I did so by taking the MLE formula...
$$\operatorname{argmax}\sum \log(P(x_i\mid\Theta ))$$
and ...
9
votes
2answers
552 views
Maximum likelihood parameters deviate from posterior distributions
I have a likelihood function $\mathcal{L}(d | \theta)$ for the probability of my data $d$ given some model parameters $\theta \in \mathbf{R}^N$, which I would like to estimate. Assuming flat priors on ...
0
votes
0answers
15 views
Using dlmMLE to estimate state space parameters
I have been trying to use the dlmMLE function from the R package dlm to estimate parameters ...
0
votes
1answer
34 views
When shall we use Expectation Maximization (EM) instead of Maximum Likelihood Estimation (MLE)?
I saw in many articles that EM is an algorithm to do MLE, and we usually use it when a direct MLE is not possible.
Can someone tell me what is the meaning of "direct MLE is not possible" (and what ...
0
votes
0answers
23 views
parameter estimation using censored data by fitting a Maximum likelihood to a differential equation
I have a population data ($N$) measured over certain time points ($t$). The rate of change of the population is modelled as a ODE as,
${dN\over dt}=-\lambda N$
My intention is to estimate the ...
0
votes
1answer
16 views
Can we say the Expectation Maximization (EM) algorithm is supposed to be used for unsupervised or semi-supervised learning?
From what I read and understood, when we have a discrete hidden variable that we already know its particular value (instead of summing/marginalizing over them) associated with data then it is ...
3
votes
2answers
45 views
EM algorithm for mixture of Gaussians - is it ok to use my updated mu's in my new estimate of Sigma, within a single M-step?
Here is a screenshot from an assignment I am currently working on - these are the Expectation-Maximization update rules for the parameters $\omega$ (latent component "responsibilities"), $\mu$, and $\...
1
vote
1answer
71 views
Questions about the likelihood in probabilities?
Many define the likelihood of the data something like $\prod_{x} p(x|\theta)$ others like $p(x|\theta)$. Is the likelihood defined for one sample point/data element (like one document from a ...
1
vote
0answers
6 views
Parametric estimator for straightforward interval-censored data
$X_i$ is iid from some distribution, such as $N(\mu, \sigma^2)$. All I want is to estimate the parameters of the distribution. However, I don't observe $x_i$, instead, I observe $(a_i, b_i)$ such that ...
-2
votes
0answers
33 views
to prove equation [duplicate]
X1…xn iid U(-θ,θ) find the MLE of θ
F(x,θ)= 1/2θ -θ≤x≤θ
0 o.w
F(x,θ)= 1/(2θ ) │x│≤θ
0 o.w
L(θ│x`s)=1/2^n 1/θ^...
-1
votes
1answer
34 views
MLE : Getting wrong answer
I am trying to solve the critical point for an MLE but my answer is wrong
PDF is as follows : $\frac{x}{\theta^2}e^{\frac{-x^2}{2\theta^2}}$ 1 - >( X>=0) , theta > 0 ( 1 is the indicator variable)
...
1
vote
1answer
47 views
Fraction of Missing Information with linear mixed models
I have a daily diary dataset (140obs for 110 persons) which I've analysed using a random slopes and intercept linear mixed model (using FIML). The model has 1 dependent variable, and 5 fixed effects.
...
0
votes
0answers
9 views
0
votes
0answers
20 views
How to compute a combined likelihood involving multiple datasets?
I have a parametric model that generates $N$ time series, and my goal is to try to find a model where the $N$ predicted time series match $N$ observed time series relatively well. My physics-based ...
0
votes
3answers
109 views
What's the difference between estimating on a dataset $P(X|Y)$ and $P(Y)$ vs $P(Y|X)$? [closed]
In chapter 3 of his excellent book ("Generative and discriminative classifiers: Naive Bayes and logistic regression") , Tom Mitchell says that, when learning classifiers based on Bayes rule, one can ...
3
votes
2answers
85 views
95% Confidence interval of $\lambda$ for $X_1,…,X_n$ IID exponential with rate $\lambda$
I know how how to find the estimation of $\hat{\lambda}$ using the method of moments. I can take the first moment and equate it to the empirical to get,
$E(X) = \frac{1}{\lambda} = \frac{\sum_{i=1}^{...
0
votes
0answers
18 views
Asymptotic covariance matrix of the maximum likelihood estimator of the parameters of a multivariate normal distribution
I would like to know how to derive the asymptotic covariance matrix of the maximum likelihood estimator of the two parameters (mean vector and covariance matrix) of a multivariate normal distribution.
...
1
vote
0answers
12 views
What are recommended practices to avoid overestimation of size/dispersion from small samples of negative-binomially distributed data?
I want to estimate mu and size/dispersion accurately for a large number of small samples (n = 6-8 is typical).
When I try to do maximum-likelihood inference (or related, Bayesian inference with ...
1
vote
0answers
34 views
Adjusted Pearson Goodness-of-Fit Test - Rugarch Package
I fitted a GARCH(1,1), GARCH-M and EGARCH of first order (using maximum likelihood) to my return dataset using both, Gaussian normal and Student-t distribution assumption for the error term.
When ...
0
votes
0answers
21 views
Iteratively Reweighted Least Squares Poisson Regression with Non-Canonical Link, eventually in R
I am attempting to implement iteratively reweighted least squares (IRLS) for a poisson regression with a non-canonical link function. My understanding of IRLS is only basic at this point, so please ...
