Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
0
votes
0answers
31 views
How to determine the right answers from different sources [on hold]
Situation:
There are n number of questions Q1..Qn
Each question has one and only one correct answer (unknown to me, and when the answer is B, Z is as wrong as C is)
I have some answers to those ...
1
vote
1answer
19 views
How to fit a superimposed distribution (\eg a Gaussian distribution + a Uniform distribution)
Suppose we have a set of independent observations of a random variable X, which is a Superimposition of two mutual independent random variables (i.e. X = Y + Z), Y follows a uniform distribution, ...
2
votes
0answers
20 views
Model selection with different fixed effects and different corARMA structures
I analyzed the effect of temperature (4 different areas) on laying date: LDT ~ Aa3+Bb+Cc+Dd. Because of autocorrelation in residuals I used ...
3
votes
1answer
50 views
The form of the Log-Likelihood Function in Mixed Linear Models
Let us assume the following mixed effects model:
$y = X\beta+Zu+e$
where $y$ is a vector of n observable random variables, $\beta$ is a vector of $p$ fixed effects, $X$ and $Z$ are known matrices, ...
1
vote
2answers
38 views
Maximum Likelihood Estimator (MLE) for $2 \theta^2 x^{-3}$
I'm having a bit of trouble solving this.
$$
f(x_i; \theta) = 2 \theta^2 x_i^{-3}, 0 \le \theta \le x_i \lt \infty
$$
I start by finding $f(\textbf{x}; \theta)$:
$$
f(\textbf{x}; \theta) = \prod{f(...
0
votes
0answers
13 views
Likelihood ratio test, Wald test and LM test for variance of a normal distribution
Let y1, y2....yt follow a N(0,sigma^2) distribution. [Note that the mean is zero and you know that it is zero]. Derive the LR, LM and Wald test of hypothesis sigma^2 = 1.
I have got the MLE, the ...
0
votes
0answers
38 views
Find the distribution function $F$ for $min_{1 \le i \le n}{X_i}$ [duplicate]
Given a random sample $X_1, X_2, ..., X_n$ where each $X_i$ has pdf:
$$
f(x; \theta) = 3 \theta^3 x^{-4}
$$
and $0 \lt \theta \le x \le \infty$. Show that the distribution function $F$ for $min_{1 \...
0
votes
2answers
34 views
What does $\sum_{\{\textbf{x}:T(\textbf{x})=t\}} f(\textbf{x}; \theta)$ mean?
This is in the following context:
$$
q(t;\theta) = P(T=t;\theta) = \sum_{\{\textbf{x}:T(\textbf{x})=t\}} f(\textbf{x}; \theta)
$$
Where $T=T(\textbf{X})$ is a statistic, $q(t; \theta)$ is the pmf of ...
0
votes
2answers
27 views
Maximum likelihood in Naive Bayes classifier
With regards to the Naive Bayes classificator, I have read the following in Wikipedia and wanted to know why it is like that:
"In many practical applications, parameter estimation for naive Bayes ...
1
vote
0answers
25 views
Trouble with MLE [closed]
I have a random sample $X_1, X_2, ..., X_n$ with $X_i$ having a pdf
$$
f(x;\theta) = 2\theta^2x^{-2}
$$
I'd like to find the MLE of $\theta$.
First, because this is a random sample, all $X_i$ are ...
2
votes
1answer
40 views
Two approaches for finding a MLE in a binomial setting
I'm learning towards an exam in mathematical statistics and I came across the following question. I was wondering if the second approach of solving the question is legitimate. If both are correct, is ...
1
vote
1answer
25 views
Recursive Bayes Learning
I'm trying to work through an example from Richard Dudas Pattern Classification on Recursive Bayes Learning. My main question is why do we choose the $max[D^n] $ in: $$max[D^n] \le \theta \le 10 $$
...
3
votes
0answers
32 views
Does any `R` package offer `gnorm`, `hnorm`, and similar? What about other languages?
R typically offers functions prefixed by d, p, q, and r ...
1
vote
0answers
10 views
Should lower and upper bounds for a distribution count as parameters in AIC model selection
Suppose we want a random variable $X$ to be constrained and thereby to lie within specified bounds other than the natural bounds of the underlying distribution.
This should be understood such that if ...
2
votes
1answer
38 views
Independent and Identically distributed assumption in Maximum likelihood estimation
I was reading about Maximum likelihood estimation from various sources on the internet and I noticed that MLE makes an assumption about the data known as IID but I didn't completely understand why is ...
2
votes
1answer
66 views
Interpreting matrix notation to run MLE in R
I am trying to re-create some indicators from the World Bank, using the methodology described in this paper, and I need to do maximum likelihood estimation, preferably using R.
The aim is to get an ...
13
votes
1answer
713 views
Why does MLE make sense, given the probability of an individual sample is 0?
This is kind of an odd thought I had while reviewing some old statistics and for some reason I can't seem to think of the answer.
A continuous PDF tells us the density of observing values in any ...
0
votes
1answer
44 views
Likelihood function for linear regression
For linear regression, the likelihood function can be found with:
However if your data points are multi-dimensional such that x,...
0
votes
0answers
21 views
Additive Gaussian Processes with Penalized Likelihood
I have a problem with many - say $D$ - input variables, $\mathbf x=(x_1,\dotsc,x_D)^\top$. I have have dataset $\mathcal D$ of $n$ input/outputs, with $n<D$. Only $\delta<<D$ should suffice ...
0
votes
0answers
17 views
Anyone can explain simply targeted learninig? [duplicate]
I am trying to understand Targeted Learning (Mark van der Laan), can anyone explain this method simply, please?
0
votes
0answers
30 views
Log likelihood function for (neural networks) regression
My question is about how we calculate the loglikelihood function for regression when you have multiple standard deviations instead of a single standard deviation.
For a standard linear regression (...
3
votes
1answer
76 views
Is the distribution of the logarithm of the mean of Bernoulli random variables ($\log \overline X$) still asymptotically normal?
Let $\overline X$ be the mean of a Bernoulli random variable (r.v.)
$$\overline X = \frac{1}{n}\sum_{i=1}^{n} X_i$$
where $X_i \in \{0, 1\}$. So based on Central Limit Theoreom,
$$\overline X \sim \...
0
votes
0answers
9 views
Does maximizing model generativeness maximize its discriminative-ness?
If I train a model M in a way that maximizes the penalized likelihood of M given some data, is this equivalent to maximizing its ...
1
vote
1answer
61 views
Deriving the sampling distribution of MLE for Normal distribution
Let $X_1,\ldots,X_n$ be an observed random sample from $N_p(\mu, \Sigma)$.
I know that the MLE of $\Sigma$ is $\frac{1}{n} \sum_i^n(X_i -\bar X)(X_i -\bar X)^T$, which is biased.
We define $S = \...
2
votes
3answers
140 views
Deriving likelihood function of binomial distribution, confusion over exponents
This question focuses on a specific aspect of this one:
How to derive the likelihood function for binomial distribution for parameter estimation?
In my own derivation, I start with:
$$f(x\mid p) = ...
0
votes
1answer
40 views
Maximum likelihood joint probability distribution (discrete & continuous)
I am trying to find the values $v_1$ and $v_2$ that maximizes the likelihood of some observations.
I have information about $v_1$ and $v_2$ from a set of 'experiments'. In each experiment, $v_1$ and ...
4
votes
1answer
76 views
Is the Quadratic Approximation of Log-Likelihood Equivalent to the Normal Approximation of the MLE?
Let $X_1, X_2, ..., X_n \sim \text{IID N}(\theta, \sigma^2)$ with $\sigma^2$ known, and let $\hat{\theta}$ be the MLE of the mean.
(1) How can I show that in this case, the following is true?
$$\...
5
votes
1answer
72 views
Should log-likelihood values increase when the sample size of a simulation increases?
If one simulates a process (such as an ARMA-GARCH process) with sample size $n$ and log-likelihood $x$, should this log-likelihood increase when the sample size increases to $2n$ for example?
2
votes
0answers
15 views
Estimating the number of tokens in a pile
We have a pile of tokens numbered from $1$ to $n$, where $n$ is unknown. $k$ tokens are drawn from the pile at random without replacement(i.e. the numbers that we get from tokens are unique). Say the ...
8
votes
4answers
113 views
Likelihood-free inference - what does it mean?
Recently I have become aware of 'likelihood-free' methods being bandied about in literature. However I am not clear on what it means for an inference or optimization method to be likelihood-free.
In ...
2
votes
2answers
92 views
Standard error from Hessian matrix when likelihood is used (rather than Ln L)
I understand that at MLE point, the inverse of the Hessian matrix can be used as approximation of V-Cov matrix:
...
0
votes
0answers
17 views
Epsilon from Bivariate Normal Distribution [duplicate]
I came across the following example from a book. I am given a dataset generated from a bivariate normal distribution:
Among the data, there are missing values for the last 20 of x2i (but not for x1i)....
0
votes
0answers
23 views
Should I try to estimate logistic regression by F1 maximization, rather than Liklihood?
Assume I have a dataset of covariates $x_i$ and binary outcomes $y_i \in \{0,1\}$. I want to predict outcome for unknown a unknown $y_k$ given $y$.
Quite common is to do this with logistic regression,...
2
votes
0answers
35 views
Obtaining Standard Errors in Optim() in R [duplicate]
I'm using a maximum likelihood estimation and I'm using the optim() function in R in a similar way as follows:
...
1
vote
0answers
27 views
How does one estimate parameters in a GARCH-M(1,1) model?
Say you have a GARCH-M(1,1) model as follows:
$y_t = \beta y_{t-1} + \delta h_t + \epsilon_t, \quad \epsilon_t \sim N(0, h_t) $
$h_t = a_0 + a_1 \epsilon^2_{t-1} + b_1 h_{t-1}.$
How exactly does ...
0
votes
1answer
30 views
Estimating the variance of a function of MLE
Say I have the following likelihood :
$$
l(\alpha, \lambda) = n(\log \alpha + \log \lambda) + (\alpha -1 )\sum x_i - \lambda \sum x_i^\alpha
$$
which is that of Weibull distribution.
The question ...
1
vote
0answers
37 views
AIC formula in R vs Python [closed]
I have been trying to calculate a GLM's AIC both in python (package Statsmodels) and R (native glm function). For exactly the same model I get two different AIC estimates. The formula for AIC is:
-...
1
vote
1answer
42 views
Closed form ML estimation of GMM with known class assignments
In Andrew Ng's CS229 notes, Gaussian mixture model and its likelihood function are given as follows:
\begin{eqnarray}
z^{(i)} \sim \textrm{Multinomial}(\phi)\\
\phi_j \geq 0\\
\sum_{j=1}^k \phi_j = 1\...
0
votes
0answers
15 views
MAP/MLE/Neyman Pearson Example Problem
I'm currently attempting a practice problem and wanted help checking my work:
The random variable X is such that P(X=1) = 2/3 and P(X=0) = 1/3. When X = 1, the random variable Y is exponentially ...
0
votes
1answer
41 views
Exemplar MLE for negative binomial?
I recently compared MLE estimates for a negative binomial fit using two different pieces of software, and got different results. I'd like to determine which (if either) is correct.
To do that, I'd ...
0
votes
0answers
26 views
Suggesting a method of moments estimator for the chance that some event happens
Let $X_i$~ $\text{Pois}(\lambda)$ be the number of breakdowns a certain ATM machine experiences in the $i^{th}$ week. $\implies$ Let $ \{X_i\}_{i=1}^n$ be iid of the number of breakdowns the machine ...
0
votes
1answer
75 views
Optimization using the optim function in R with a two parameter exponential distribution
I'm having trouble trying to optimize a two-parameter exponential distribution, by finding the maximum likelihood function and then using the function optim() in R
...
0
votes
1answer
29 views
$2D$ Maximum Likelihood Fit
I have read a couple of places that it is possible to do a $2D$ (or $3D$) maximum likelihood fit, but I can't seem to understand how this would work. Suppose I'm considering a probability distribution ...
2
votes
2answers
125 views
Calculating a MLE for the combinatorial probability distribution: $\sum_k p^k(1-p)^{N-j-k}\binom{N-j}{k} \cdot(1-p)^{i-k}p^{j-i+k}\binom{j}{i-k}$
I have a relatively complicated discrete probability distribution:
$$\begin{aligned}
P(i;j) &= \sum_k p^k(1-p)^{N-j-k}\binom{N-j}{k}\cdot(1-p)^{i-k}p^{j-i+k}\binom{j}{i-k} \\
&= \frac{p^j}{p^...
1
vote
0answers
40 views
How to construct a cross-entropy loss for general regression targets?
It's common short-hand in neural networks literature to refer to categorical cross-entropy loss as "cross-entropy," even though there are a number of loss functions which could properly be described ...
0
votes
0answers
11 views
What are level-2 covariance parameters within Iterative Generalised Least Squares Models to estimate multilevel models?
I am referring to Goldstein & Rasbash (1992): Efficient computational procedures for the estimation of parameters in multilevel models based on iterative generalised least squares. Computational ...
2
votes
1answer
32 views
Is it ever convenient to maximize different functions of the likelihood than the logarithm?
We all know that it's often much more convenient to maximize the log-likelihood rather than the likelihood to get a parameter estimate, since it amounts to the same thing by the fact that the ...
0
votes
0answers
16 views
Choose between ratio of estimators or estimator from the ratio of data
I have to estimate a function which is, say, the proportion of people under 20 in each point of a given territory. Let's call that $h(x,y)$ for $(x,y)$ in my territory.
To get that, I have, for every ...
3
votes
0answers
54 views
Maximum likelihood: Why is the number of non-zero eigenvalues equal to $x^T \hat{\Sigma}^{-1} x$
I've been reading this code (based on this R package) and I found that the number of non-zero eigenvalues of the estimated covariance is roughly equal to $x_i^T \hat{\Sigma}^{-1} x_i$. I want to know ...
1
vote
0answers
51 views
Confusion about the use of the MLE & the posterior in parameter estimation for logistic regression
In classification one usually computes
$$
C = \operatorname*{argmax}_k p(C=k\mid X)
$$
where $p(C=k\mid X)$ is the posterior distribution.
In a simple logistic regression setting with $C \in \{0, 1\}...
