Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

0
votes
0answers
2 views

Model selection with different fixed effects and different corARMA structures

I analyzed the effect of temperature (4 different areas) on laying date: LDT ~ Aa3+Bb+Cc+Dd. Because of autocorrelation in residuals I used ...
1
vote
1answer
18 views

The form of the Log-Likelihood Function in Mixed Linear Models

Let us assume the following mixed effects model: $y = X\beta+Zu+e$ where $y$ is a vector of n observable random variables, $\beta$ is a vector of $p$ fixed effects, $X$ and $Z$ are known matrices, ...
1
vote
1answer
6 views

Finding logged partial difference and the loglikelihood of GED PDF

I'm not so good at math, and i am trying to learn. I have a pdf function of GED (Generalized Error Distribution) as showed: I need to find the partial differential of the logged pdf with respect to ...
1
vote
2answers
32 views

Maximum Likelihood Estimator (MLE) for $2 \theta^2 x^{-3}$

I'm having a bit of trouble solving this. $$ f(x_i; \theta) = 2 \theta^2 x_i^{-3}, 0 \le \theta \le x_i \lt \infty $$ I start by finding $f(\textbf{x}; \theta)$: $$ f(\textbf{x}; \theta) = \prod{f(...
0
votes
0answers
9 views

Likelihood ratio test, Wald test and LM test for variance of a normal distribution

Let y1, y2....yt follow a N(0,sigma^2) distribution. [Note that the mean is zero and you know that it is zero]. Derive the LR, LM and Wald test of hypothesis sigma^2 = 1. I have got the MLE, the ...
0
votes
0answers
38 views

Find the distribution function $F$ for $min_{1 \le i \le n}{X_i}$ [duplicate]

Given a random sample $X_1, X_2, ..., X_n$ where each $X_i$ has pdf: $$ f(x; \theta) = 3 \theta^3 x^{-4} $$ and $0 \lt \theta \le x \le \infty$. Show that the distribution function $F$ for $min_{1 \...
0
votes
2answers
34 views

What does $\sum_{\{\textbf{x}:T(\textbf{x})=t\}} f(\textbf{x}; \theta)$ mean?

This is in the following context: $$ q(t;\theta) = P(T=t;\theta) = \sum_{\{\textbf{x}:T(\textbf{x})=t\}} f(\textbf{x}; \theta) $$ Where $T=T(\textbf{X})$ is a statistic, $q(t; \theta)$ is the pmf of ...
0
votes
2answers
25 views

Maximum likelihood in Naive Bayes classifier

With regards to the Naive Bayes classificator, I have read the following in Wikipedia and wanted to know why it is like that: "In many practical applications, parameter estimation for naive Bayes ...
1
vote
0answers
25 views

Trouble with MLE [on hold]

I have a random sample $X_1, X_2, ..., X_n$ with $X_i$ having a pdf $$ f(x;\theta) = 2\theta^2x^{-2} $$ I'd like to find the MLE of $\theta$. First, because this is a random sample, all $X_i$ are ...
2
votes
1answer
40 views

Two approaches for finding a MLE in a binomial setting

I'm learning towards an exam in mathematical statistics and I came across the following question. I was wondering if the second approach of solving the question is legitimate. If both are correct, is ...
1
vote
1answer
23 views

Recursive Bayes Learning

I'm trying to work through an example from Richard Dudas Pattern Classification on Recursive Bayes Learning. My main question is why do we choose the $max[D^n] $ in: $$max[D^n] \le \theta \le 10 $$ ...
3
votes
0answers
30 views

Does any `R` package offer `gnorm`, `hnorm`, and similar? What about other languages?

R typically offers functions prefixed by d, p, q, and r ...
1
vote
0answers
10 views

Should lower and upper bounds for a distribution count as parameters in AIC model selection

Suppose we want a random variable $X$ to be constrained and thereby to lie within specified bounds other than the natural bounds of the underlying distribution. This should be understood such that if ...
2
votes
1answer
38 views

Independent and Identically distributed assumption in Maximum likelihood estimation

I was reading about Maximum likelihood estimation from various sources on the internet and I noticed that MLE makes an assumption about the data known as IID but I didn't completely understand why is ...
2
votes
1answer
65 views

Interpreting matrix notation to run MLE in R

I am trying to re-create some indicators from the World Bank, using the methodology described in this paper, and I need to do maximum likelihood estimation, preferably using R. The aim is to get an ...
13
votes
1answer
708 views

Why does MLE make sense, given the probability of an individual sample is 0?

This is kind of an odd thought I had while reviewing some old statistics and for some reason I can't seem to think of the answer. A continuous PDF tells us the density of observing values in any ...
0
votes
1answer
43 views

Likelihood function for linear regression

For linear regression, the likelihood function can be found with: However if your data points are multi-dimensional such that x,...
0
votes
0answers
21 views

Additive Gaussian Processes with Penalized Likelihood

I have a problem with many - say $D$ - input variables, $\mathbf x=(x_1,\dotsc,x_D)^\top$. I have have dataset $\mathcal D$ of $n$ input/outputs, with $n<D$. Only $\delta<<D$ should suffice ...
0
votes
0answers
17 views

Anyone can explain simply targeted learninig? [duplicate]

I am trying to understand Targeted Learning (Mark van der Laan), can anyone explain this method simply, please?
0
votes
0answers
28 views

Log likelihood function for (neural networks) regression

My question is about how we calculate the loglikelihood function for regression when you have multiple standard deviations instead of a single standard deviation. For a standard linear regression (...
3
votes
1answer
76 views

Is the distribution of the logarithm of the mean of Bernoulli random variables ($\log \overline X$) still asymptotically normal?

Let $\overline X$ be the mean of a Bernoulli random variable (r.v.) $$\overline X = \frac{1}{n}\sum_{i=1}^{n} X_i$$ where $X_i \in \{0, 1\}$. So based on Central Limit Theoreom, $$\overline X \sim \...
0
votes
0answers
9 views

Does maximizing model generativeness maximize its discriminative-ness?

If I train a model M in a way that maximizes the penalized likelihood of M given some data, is this equivalent to maximizing its ...
1
vote
1answer
59 views

Deriving the sampling distribution of MLE for Normal distribution

Let $X_1,\ldots,X_n$ be an observed random sample from $N_p(\mu, \Sigma)$. I know that the MLE of $\Sigma$ is $\frac{1}{n} \sum_i^n(X_i -\bar X)(X_i -\bar X)^T$, which is biased. We define $S = \...
2
votes
3answers
136 views

Deriving likelihood function of binomial distribution, confusion over exponents

This question focuses on a specific aspect of this one: How to derive the likelihood function for binomial distribution for parameter estimation? In my own derivation, I start with: $$f(x\mid p) = ...
0
votes
1answer
39 views

Maximum likelihood joint probability distribution (discrete & continuous)

I am trying to find the values $v_1$ and $v_2$ that maximizes the likelihood of some observations. I have information about $v_1$ and $v_2$ from a set of 'experiments'. In each experiment, $v_1$ and ...
4
votes
1answer
73 views

Is the Quadratic Approximation of Log-Likelihood Equivalent to the Normal Approximation of the MLE?

Let $X_1, X_2, ..., X_n \sim \text{IID N}(\theta, \sigma^2)$ with $\sigma^2$ known, and let $\hat{\theta}$ be the MLE of the mean. (1) How can I show that in this case, the following is true? $$\...
5
votes
1answer
72 views

Should log-likelihood values increase when the sample size of a simulation increases?

If one simulates a process (such as an ARMA-GARCH process) with sample size $n$ and log-likelihood $x$, should this log-likelihood increase when the sample size increases to $2n$ for example?
2
votes
0answers
15 views

Estimating the number of tokens in a pile

We have a pile of tokens numbered from $1$ to $n$, where $n$ is unknown. $k$ tokens are drawn from the pile at random without replacement(i.e. the numbers that we get from tokens are unique). Say the ...
8
votes
4answers
111 views

Likelihood-free inference - what does it mean?

Recently I have become aware of 'likelihood-free' methods being bandied about in literature. However I am not clear on what it means for an inference or optimization method to be likelihood-free. In ...
2
votes
2answers
90 views

Standard error from Hessian matrix when likelihood is used (rather than Ln L)

I understand that at MLE point, the inverse of the Hessian matrix can be used as approximation of V-Cov matrix: ...
0
votes
0answers
17 views

Epsilon from Bivariate Normal Distribution [duplicate]

I came across the following example from a book. I am given a dataset generated from a bivariate normal distribution: Among the data, there are missing values for the last 20 of x2i (but not for x1i)....
0
votes
0answers
23 views

Should I try to estimate logistic regression by F1 maximization, rather than Liklihood?

Assume I have a dataset of covariates $x_i$ and binary outcomes $y_i \in \{0,1\}$. I want to predict outcome for unknown a unknown $y_k$ given $y$. Quite common is to do this with logistic regression,...
2
votes
0answers
34 views

Obtaining Standard Errors in Optim() in R [duplicate]

I'm using a maximum likelihood estimation and I'm using the optim() function in R in a similar way as follows: ...
1
vote
0answers
26 views

How does one estimate parameters in a GARCH-M(1,1) model?

Say you have a GARCH-M(1,1) model as follows: $y_t = \beta y_{t-1} + \delta h_t + \epsilon_t, \quad \epsilon_t \sim N(0, h_t) $ $h_t = a_0 + a_1 \epsilon^2_{t-1} + b_1 h_{t-1}.$ How exactly does ...
0
votes
1answer
30 views

Estimating the variance of a function of MLE

Say I have the following likelihood : $$ l(\alpha, \lambda) = n(\log \alpha + \log \lambda) + (\alpha -1 )\sum x_i - \lambda \sum x_i^\alpha $$ which is that of Weibull distribution. The question ...
1
vote
0answers
36 views

AIC formula in R vs Python [closed]

I have been trying to calculate a GLM's AIC both in python (package Statsmodels) and R (native glm function). For exactly the same model I get two different AIC estimates. The formula for AIC is: -...
1
vote
1answer
42 views

Closed form ML estimation of GMM with known class assignments

In Andrew Ng's CS229 notes, Gaussian mixture model and its likelihood function are given as follows: \begin{eqnarray} z^{(i)} \sim \textrm{Multinomial}(\phi)\\ \phi_j \geq 0\\ \sum_{j=1}^k \phi_j = 1\...
0
votes
0answers
15 views

MAP/MLE/Neyman Pearson Example Problem

I'm currently attempting a practice problem and wanted help checking my work: The random variable X is such that P(X=1) = 2/3 and P(X=0) = 1/3. When X = 1, the random variable Y is exponentially ...
0
votes
1answer
39 views

Exemplar MLE for negative binomial?

I recently compared MLE estimates for a negative binomial fit using two different pieces of software, and got different results. I'd like to determine which (if either) is correct. To do that, I'd ...
0
votes
0answers
25 views

Suggesting a method of moments estimator for the chance that some event happens

Let $X_i$~ $\text{Pois}(\lambda)$ be the number of breakdowns a certain ATM machine experiences in the $i^{th}$ week. $\implies$ Let $ \{X_i\}_{i=1}^n$ be iid of the number of breakdowns the machine ...
0
votes
1answer
74 views

Optimization using the optim function in R with a two parameter exponential distribution

I'm having trouble trying to optimize a two-parameter exponential distribution, by finding the maximum likelihood function and then using the function optim() in R ...
0
votes
1answer
29 views

$2D$ Maximum Likelihood Fit

I have read a couple of places that it is possible to do a $2D$ (or $3D$) maximum likelihood fit, but I can't seem to understand how this would work. Suppose I'm considering a probability distribution ...
2
votes
2answers
125 views

Calculating a MLE for the combinatorial probability distribution: $\sum_k p^k(1-p)^{N-j-k}\binom{N-j}{k} \cdot(1-p)^{i-k}p^{j-i+k}\binom{j}{i-k}$

I have a relatively complicated discrete probability distribution: $$\begin{aligned} P(i;j) &= \sum_k p^k(1-p)^{N-j-k}\binom{N-j}{k}\cdot(1-p)^{i-k}p^{j-i+k}\binom{j}{i-k} \\ &= \frac{p^j}{p^...
1
vote
0answers
38 views

How to construct a cross-entropy loss for general regression targets?

It's common short-hand in neural networks literature to refer to categorical cross-entropy loss as "cross-entropy," even though there are a number of loss functions which could properly be described ...
0
votes
0answers
11 views

What are level-2 covariance parameters within Iterative Generalised Least Squares Models to estimate multilevel models?

I am referring to Goldstein & Rasbash (1992): Efficient computational procedures for the estimation of parameters in multilevel models based on iterative generalised least squares. Computational ...
2
votes
1answer
31 views

Is it ever convenient to maximize different functions of the likelihood than the logarithm?

We all know that it's often much more convenient to maximize the log-likelihood rather than the likelihood to get a parameter estimate, since it amounts to the same thing by the fact that the ...
0
votes
0answers
16 views

Choose between ratio of estimators or estimator from the ratio of data

I have to estimate a function which is, say, the proportion of people under 20 in each point of a given territory. Let's call that $h(x,y)$ for $(x,y)$ in my territory. To get that, I have, for every ...
3
votes
0answers
54 views

Maximum likelihood: Why is the number of non-zero eigenvalues equal to $x^T \hat{\Sigma}^{-1} x$

I've been reading this code (based on this R package) and I found that the number of non-zero eigenvalues of the estimated covariance is roughly equal to $x_i^T \hat{\Sigma}^{-1} x_i$. I want to know ...
1
vote
0answers
50 views

Confusion about the use of the MLE & the posterior in parameter estimation for logistic regression

In classification one usually computes $$ C = \operatorname*{argmax}_k p(C=k\mid X) $$ where $p(C=k\mid X)$ is the posterior distribution. In a simple logistic regression setting with $C \in \{0, 1\}...
0
votes
0answers
4 views

Duration Analysis with Clumping at Infinity?

I am currently trying to build a model to analyze how price setting today affects how long it takes for a customer to return. My first cut was to fit a Weibull regression where the log of the scale ...