New answers tagged maximum-likelihood
1
We have the likelihood of $m$ by $$f(m)=\frac{m!}{(m-X)!X!}p^X(1-p)^{m-X}$$
Here $m\geq X$. Since when $m\rightarrow\infty$ and $m\rightarrow X$, $f(m)\rightarrow -\infty$, we know MLE exists.
Now, let's consider when $\frac{f(m+1)}{f(m)}$ is less than 1. When it is less than 1, we know that function $f$ reached a (local) maximum. After calculation we find
$$...
1
I don't see how to do this with only one observation, if $p$ is unknown.
If $p$ is known, here are some clues. [More generally, this is a much-studied problem; perhaps see this paper and its references.]
If $p = 0.3$ and your observation is $X = 12,$ then
the method of moments estimator is $X/p = 40.$
Perhaps it is reasonable to guess that the MLE will
be ...
0
You are looking at a multivariate binomial distribution where all the variables are independent. The ML may be calculated for each variable separately.
Consider the $i$ variable and let $\theta_1 = p$. Let $y_1,\ y_2 \ \dots,y_n$ be a random sample of size $n$ from this distribution. Then let $j=\sum_{k=1}n y_k$. Then you may show that the Liklihood ...
3
No, the $z_t$ are not independent, so that doesn't work.
The likelihood is a joint probability, so start there rather than from a single observation:
$$\mathcal{L}(\theta) = p(z_1, ..., z_T| \theta)$$
The recursive definition of the process (the first equation you show) gives you the transition distribution directly:
$$z_t | (z_1,...,z_{t-1}, \theta) \sim \...
3
I think this paper answers your question! I will briefly describe the main result of the paper.
The paper suggests a family of distributions that are 'in between' the uniform and triangular distribution. That is, at $\theta = 1$ the distribution is the triangular distribution and for $\theta = 0$ the distribution is uniform. For in between values the ...
1
You’re asking how to estimate the maximum likelihood parameters of a normal distribution from data. That is—you want the parameters with highest likelihood, given the data.
How you find these is up to you; there are several optimization procedures you could turn to. You could start at some initial guess of $\mu$ and $\sigma$, then climb the gradient of the ...
2
As an overarching framework, indeed MLE is much better. It shows when you really want to use LSE, and when you want to use other estimators, such as LAE, Poisson regression, logistic regression, WLS, GLS, etc. It also leads naturally and seamlessly to Bayesian methods. All quantitative disciplines would be much better off to replace LSE with MLE.
These days,...
2
I'll use $\mu_i = \eta_1 - 2\theta\eta_2x_i + \eta_2 x_i^2$ for convenience. If we're thinking of $\mu_i$ as a function of $\theta$, so only $\eta_1$ and $\eta_2$ are parameters, then we can write this as
$$
\mu_i = \eta_1 + \eta_2(-2\theta x_i + x_i^2) = \eta_1 + \eta_2 z_i
$$
for $z_i = -2\theta x_i + x_i^2$. This is just a simple linear regression now so
$...
1
At the end of my paper with James MacKinnon and others, you'll find a derivation of a generalization of LIML that I figured out: restricted LIML, meaning LIML with linear constraints on coefficients. The derivation is itself derived from Davidson and MacKinnon (1993), which several respondents have already mentioned. But it is online and free, and might help ...
1
The traditional (Wald) 95% CI for success probability $p$ uses the MLE $\hat p = x/n,$ where $x$ is the number of successes in $n$ trials. It is an asymptotic
CI intended for use with large $n$ where the normal approximation is accurate. It is of the form
$$\hat p \pm 1.96\sqrt{\frac{\hat p(1-\hat p)}{n}}.$$
In judging the $n$ required for a given margin of ...
0
The product is indexed over the countable list of your random sample, as indicated by the superscript $n$ over the Pi operator.
You are confusing the case in probability theory where a random variable following a known distribution can summarize the probability of a range of events. When those random variables have a continuous distribution, you integrate ...
0
Here is my answer: Think about the data generation process. Suppose we have a random variable $X$ depending on a parameter $\theta$. We assume $\theta$ follows some distribution as in the Bayesian philosophy.
In this case, consider the joint distribution of $(X, \theta)$. Each time $X$ is generated, a $\theta$ is generated as well. However, you cannot see ...
1
Using a uniform prior is equivalent to the MLE estimate. When you say that the asymptotic variance of $σ^{2}$ is normal, that's not quite correct. The quantity $(n-1)×s^{2}/σ^{2}$ is distributed as $χ^{2}_{n-1}$, with $s$ the sample standard deviation, $σ$ the true population standard deviation, and $n$ the sample size. With large $n$, a $χ^{2}$ distribution ...
1
The likelihood function $L(\theta|X)$ is indeed a random function (since $X$ is random) in the set of likelihood functions
$$\{L(\cdot|x);\ x\in\mathsf X\} $$
Its distribution depends (obviously) on the statistical model. For instance
if $L(\theta|X)$ is the likelihood function attached to a Normal $\mathcal N(\theta,1)$ $n$-sample, and if $\theta^*$ is the ...
1
The fact that cross entropy has an interpretation as a probability distribution or as a the Kullback-Leibler divergence between's the model estimated probability distribution and the empirical distribution of labels does not mean that it is optimal for all cases. As you point out, classification with focal loss actually works even better.
The original paper (...
0
The main reason for using conditional maximum likelihood is the resulting distribution. For Y|X ~N(x'B,Var(eps)) holds because the variation of Y only depends on the (normal) variation of the eps. As you know the correct assumption of the underlying density is a crucial point in MML estimation and hence, with unconditional MML you would run into problems ...
0
You have to define "best parameters" and "better performance". Normally, you choose a loss function that's differentiable, but you actually care about some other metric, like accuracy, AUC, or average precision. If, for example, you care about AUC, and using focal loss produces higher validation AUC than cross entropy loss does, then use ...
0
There may be a problem with the likelihood calculation. Essentially, you need to integrate out the latent variables from the likelihood itself rather than the loglikelihood.
Let $Y=(Y_t)_{t=1}^T$ denote the vector of observed variables, let $X=(X_t)_{t=1}^T$ denote the vector of unobserved variables. Then $$P(Y|X)=\prod_{t=1}^T {N\choose y_t} {p_t}^{y_t}(1-...
1
There are multiple concepts in this question. In statistics, you will find there are theorems for certain models about what methods produces the "best" parameter estimates. In linear regression with normally distributed errors, you can prove that the normal statistical parameter estimates are the BLUE (Best linear unbiased estimates) of the ...
1
I think there is a lot of confusion here. First, I want to remind you that OLS and MLE are statistical algorithms for estimating parameters from data. OLS says, to get the parameters estimates for a linear model, find those that minimize the sum of the squared residuals. MLE says, to get the parameter estimates for a model, find those that maximize the ...
Top 50 recent answers are included
