Skip to main content

Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

How are the MLE/MAP distinction and the generative/discriminative distinction related?

What is the relationship between Maximum Likelihood Estimation versus Maximum A Posteriori Estimation and generative modeling versus discriminative modeling? Is MLE an example of a generative model ...
user10478's user avatar
  • 123
0 votes
0 answers
37 views

How to fit data to a parametric curve/model (x(t), y(t)?

I've got data of x and y pairs and I'd like to fit it to a model that is parametrized as f = (x(t), y(t)). Unfortunately, there is no way for me to analytically solve for t and get a direct ...
StatAnomaly123's user avatar
1 vote
0 answers
76 views

Estimation in generalized additive models

I'm currently trying to learn about generalized additive models (GAMs) with the book Generalized Additive Models An Introduction with R by Simon N. Wood. However, I have some questions regarding the ...
Dude3400's user avatar
0 votes
0 answers
30 views

What is the difference between estimating parameters via MLE versus minimizing deviations from expectation?

What is the difference between estimating parameters using MLE (or MAP with uniform priors): $$\theta^* = \arg \max_\theta p(X|\theta)$$ and estimating them according to which setting would engender ...
actinidia's user avatar
  • 145
4 votes
3 answers
226 views

biased MLEs in negative binomial models

I found the maximum likelihood parameters of negative binomial models are biased. An example code is provided below. Is this normal? Are there ways to obtain unbiased estimates? ...
quibble's user avatar
  • 1,664
0 votes
0 answers
28 views

Fitting Hypergeometric distribution requires non-integer arguments?

I have a vector (length s) of observations, x are class "0" and s-x are class "1" and are drawn from a population of size N. Hence, they follow the hypergeometric distribution: $$H(...
Jesús Castrejón's user avatar
0 votes
0 answers
29 views

Which log-likelihood is to be maximized for left-truncated count data?

What is to be done, if the count data is missing the counts on the zeros (i.e. left truncated data)? Say one wants to estimate a Poisson regression and the goal is to derive the log-likelihood to be ...
Marlon Brando's user avatar
1 vote
0 answers
25 views

Convexity of negative log-likelihood of exponential family distribution [closed]

Let $p(y; \theta^Tx) = b(y) \space \exp\big(\theta^T x y - a(\theta^T x)\big)$, where $x$ and $\theta$ are $d$-dimensional vectors and $y$ a scalar. If I'm not mistaken, the negative log-likelihood ...
Arturo Sbr's user avatar
0 votes
0 answers
22 views

Likelihood test on simple regression

I'm working through Introduction to Econometrics by Gary Koop, but I'm slightly struggling to make sense of a question from chapter 4. It asks you to carry out a liklihood ratio test of $H_0 :\beta=0$ ...
Student123's user avatar
0 votes
0 answers
14 views

Manual MLE of AR(1) yields a weird initial value $y_0$

I am playing with a manual implementation of the maximum likelihood estimator (MLE) of the parameters in an AR(1) model $$ y_t = c + \varphi_1 y_{t-1} + \varepsilon_t $$ with $\text{Var}(\varepsilon_t)...
Richard Hardy's user avatar
3 votes
1 answer
86 views

likelihood ratio tests on bounded parameters

I am confused by the likelihood ratio test's boundary condition limitation. A commonly stated is that it causes problem for variance parameter because it is bounded below by 0. Can these models ...
quibble's user avatar
  • 1,664
3 votes
0 answers
27 views

Is there a likelihood penalization or (im)proper prior to remove estimation bias for gamma parameters?

So I am learning that maximum likelihood estimation of the parameters for a gamma distribution are biased. As far as I understand there is no guarantee in general that there exists a prior (or base ...
Galen's user avatar
  • 9,229
0 votes
0 answers
31 views

MLE of marginal distribution for continuous random variable

Let $\mathcal{F}$ be a family of multivariate probability densities such that for a sufficiently large data sample, there always exists a unique MLE. Assume also that all marginal and conditional ...
12345's user avatar
  • 213
0 votes
0 answers
7 views

Precision of estimates of lower bit error probabilities at higher SNR

For my university lab in wireless communications, I simulated a simple uncoded BPSK (binary phase shift keying) channel with AWGN (additive white gaussian noise) to estimate the BER (bit error rate) ...
Scarab's user avatar
  • 1
0 votes
0 answers
15 views

Optimisation of Polynomial Fittting Process

I have built a multitvariate log link GLM model and I want to fit polynomials to some of the numerical variates (i.e. fit polynomials of order 1,2,3 etc to the relativities of the model). However, I ...
JHARR's user avatar
  • 1
0 votes
0 answers
15 views

When we talk about the Likelihood in general ... Are we talking about a "Joint Likelihood" or "Conditional Likelihood"? [duplicate]

I am confused about this. Suppose we collect some data and believe it came from a Normal Distribution. To estimate parameters (mu, sigma) of this Normal Distribution we create a Likelihood Function ...
user avatar
1 vote
0 answers
80 views

Manual maximum likelihood estimation of realized GARCH behaving poorly

I'm trying to estimate the maximum likelihood of a realized GARCH model. Below are the equations and the parameters I want to estimate I'm using the below function to maximise the likelihood, but it ...
user199's user avatar
  • 23
1 vote
0 answers
32 views

How to show that MLE of probit regression does not exist due to data separability

Claim The claim is the is the following: Assume we have the simple probit model $E(y_i|x_i ) = Φ(α+\beta x_i)$. Now suppose that $y_i = 1$ for all $x_i ≤ 10$ and $y_i = 0$ for all $x_i > 10$. Then $...
Marlon Brando's user avatar
0 votes
0 answers
79 views

What is the likelihood of a regression? [duplicate]

I understand linear regressions themselves have likelihoods. Is this simply the likelihood of the error? I thought it was the likelihood of the data for Y given X. In other words, $Lik(Y$~$X)=Lik(Y|...
A Friendly Fish's user avatar
3 votes
2 answers
109 views

Must maximum likelihood method be applied on a simple random sample or on a realisation?

I guess my trouble is not a big one but here it is: when one applies maximum likelihood, he considers the realization $(x_1, \dots, x_n)$ of a simple random sample (SRS), leading to ML Estimates. But ...
MysteryGuy's user avatar
3 votes
1 answer
35 views

describing binomial data in likelihood models

Binomial data can be described in various ways. Suppose we flip a fair coin twice and get one head (success). One method to calculate its negative log-likelihood is ...
quibble's user avatar
  • 1,664
0 votes
0 answers
46 views

MLE and UMVUE from an ordered sample of the exponential distribution

I am having a lot of trouble with every part of the problem below. Now, finding MLE's is simple in principle. I just find the distribution for $Y$ and then use calculus to find the value of $\sigma$ ...
Anon's user avatar
  • 155
6 votes
1 answer
346 views

Is OLS asymptotically the best estimator even without gaussian error?

It is known that MLE is consistent and asymptotically efficient. OLS under certain assumptions is asymptotically normal. If the errors are gaussian, then OLS is equivalent to MLE. If the errors are ...
user405777's user avatar
3 votes
0 answers
61 views

Probability that normally distributed variables will have a specific ranking

There are $k$ players playing a game, each gives a performance $X_k \sim N(\mu_k, 1)$ and we observe their ranking from best to worst (a permutation of the player indexes). How to calculate the ...
fhucho's user avatar
  • 131
2 votes
1 answer
40 views

Invariance property of MLE - Proof

I am reading a proof about the invariance property of the MLE but there is a equality that does not make sense to me. Suppose $\tau(\cdot)$ is a one to one function, define $\eta = \tau(\theta)$ so we ...
Lucas cantu's user avatar
0 votes
1 answer
21 views

Computing the limiting distribution of the Bayes estimator for exponential data with a Gamma prior (by using consistency?)

Let data be $X_i \sim \text{Exp}(\theta)$ iid, $i=1,...,n$. Let the prior be $\text{Gamma}(\alpha, \beta)$. The posterior is then of course $\text{Gamma}(\alpha + n, \beta + \sum X_i)$. The Bayes ...
Featherball's user avatar
2 votes
1 answer
102 views

why is MLE formula divided by sample size?

MLE for independent data samples $D = (x_1,y_1), (x_2,y_2), (x_3,y_3) ... (x_N, y_N)$ can be formulated as $$ L(D) = \prod_{i=1}^N p_i(x_i, y_i) $$ And the log likelihood being: $$ \log(L(D)) = \sum_{...
spie227's user avatar
  • 23
1 vote
0 answers
60 views

Maximum Likelihood in High Dimensions [closed]

What are some examples of high-dimensional random variables for which MLE are solved using numerical methods because we are unable to explicitly solve the equations nicely? The only example to comes ...
Nicolas Bourbaki's user avatar
1 vote
1 answer
35 views

Maximum Likelihood for Multivariate Regression (ML)

TL;DR: How do you perform likelihood maximization for a multivariate regression in the context of ML? Background: For univariate regression we can view datapoints $(x, y)$ as being sampled from a ...
asras's user avatar
  • 11
2 votes
1 answer
41 views

equivalence between the likelihood ratio test and t-tests

The linked sites (link1, link2) demonstrate that the likelihood ratio tests and the corresponding one- and two-sample t-tests are equivalent. However, based on my understanding, the null distribution ...
quibble's user avatar
  • 1,664
0 votes
0 answers
24 views

Can complicated $\mathcal{Q} (\theta; \theta^{\text{old}})$ function be replaced by log-likelihood when implementing/coding EM algorithm?

I am working on a MLE problem where one of the parameters does not have a closed-form solution. I have a proposal for $\theta^{\text{new}}$, but reject it if it does not improve $\mathcal{Q} (\theta; \...
Stew P. Doe's user avatar
1 vote
0 answers
7 views

Clarifying between sample date and input data

I'm reading section 2.6.3 from the book "The Elements of Statistical Learning" of "Trevor Hastie", "Robert Tibshirani" and "Jerome Friedman" and I have a few ...
alksdhalksjdb's user avatar
0 votes
0 answers
11 views

Analyze the variance of one adaptive process?

I'm currently interested in analyzing the variance of one adaptive process. To be more specific, suppose I have done some, let's say $n$ times, experiments where the results depend on some unknown ...
narip's user avatar
  • 187
7 votes
3 answers
303 views

Different estimates of Least Squares and Maximum Likelihood Estimates under non-normality

It is said that Least Squares estimates would differ from Maximum Likelihood estimates if the underlying data were non-normal. This should be the reason why LS estimates can be used in linear ...
Anti's user avatar
  • 159
1 vote
0 answers
13 views

Potential heteroskedasticity in maximum likelihood

I've created a bad loan classifier model using logit regression and maximum likelihood. The actual v expected comparison of the result is shown below. In order to create the chart, we binned the ...
user11209442's user avatar
1 vote
1 answer
33 views

Why can we get better asymptotic global estimators even for IID random variables?

Let $X_1,...,X_N$ be IID random variables sampled from a parametrised distribution $p_\theta$, and suppose my goal is to retrieve $\theta$ from these samples. We know that the MLE provides an ...
glS's user avatar
  • 383
0 votes
0 answers
17 views

Demonstrating $SU=U(\sigma^2 I+D^2)$ as a Sufficient Condition in Maximum Likelihood Estimation

I am working on an exercise related to maximum likelihood estimation (in the context of principal component analysis) for the distribution $$p(x) = Gauss(b, WW^T+\sigma^2I)$$ In particular, I want to ...
Andrea's user avatar
  • 153
2 votes
1 answer
39 views

Does OLS give the maximum likelihood estimation for a linear log model?

I'm fitting a model $y=a\times \log(x)+ b$ using standard scikit linear regression (wich uses OLS) and a transformation $x'=\log(x)$. My doubt is: the parameters I get for the model are the best one ...
Roger Danilo Figlie's user avatar
1 vote
1 answer
68 views

Is this Maximum Likelihood Estimate (MLE) scenario possible? [duplicate]

I'm a stats novice, and I'm currently looking into ML estimates for phylogenetic trees. I've had this "philosophical" problem with the MLE method in general ever since I learned about it. ...
Marko's user avatar
  • 11
0 votes
0 answers
49 views

Conditional likelihood, conditional independence and joint independence

Consider a sequence of data samples generated from $n$ independent random vectors $(X_1, Y_1), (X_2,Y_2), (X_3,Y_3) ...$ $$D = (x_1,y_1), (x_2,y_2), (x_3,y_3) ...$$ Where $(X_i, Y_i)$ - is a random ...
spie227's user avatar
  • 23
1 vote
3 answers
148 views

How do we get rid of $p(x|\theta)$ in MLE

Simple question... usually the intro to ML tells you the following: you want to maximize $p(\theta|D)$, thus the optimization is $\theta = \arg\max_\theta p(\theta|D)$ bayes theorem tells us that $p(\...
Alberto's user avatar
  • 1,207
1 vote
1 answer
85 views

Difference between truncated and unseen data

I have 2 related questions. Assume that we want to build a model to study of some random discrete variable $x$ that follows some known distribution with PMF $P(x)$, yet with unknown parameters that we ...
Geo's user avatar
  • 37
0 votes
1 answer
42 views

Conditional Likelihood exponential distribution [closed]

Let $X_1, ... X_n$ be iid Exp($\lambda$), where $\lambda > 0$. How does the Maximum Likelihood Estimator (MLE) of $\lambda$ change if we somehow are told that all $X_i$ overshot their mean? (i.e.: ...
JohnD's user avatar
  • 117
1 vote
0 answers
31 views

How could I fit a model of a non-homogeneous Poisson process in STAN? [closed]

I have some data $t_1, t_2, ..., t_n$ where $0 < t_i < T$ for all $t_i$. I assume that this has been generated by an inhomogeneous Poisson process with parameter $\lambda(t)$ defined again for $...
user1747134's user avatar
1 vote
0 answers
21 views

Maximum a Posteriori (MAP) in practice for machine learning

I'm a beginner in machine learning and I had a a few questions regarding MAP. From my limited understanding it seems to me a bayesian approach, specifically an MLE approach is incredibly useful when ...
Sloth Man's user avatar
  • 111
0 votes
1 answer
28 views

Maximum likelihood estimation and bayesian inference of variance given multiple datasets

I'm currently working on a problem were I have multiple normal distributed data sets $X_1, \dotsc,X_n$ with each data set having it's own mean $\bar x_i $ but all have the same variance $\sigma$. The ...
Jan's user avatar
  • 11
0 votes
0 answers
44 views

Heteroscedastic residuals of a VECM estimated by MLE

I have estimated an VEC model in Matlab, and it turns out the residuals are heteroscedastic. Now, does anyone know how to apply HAC errors to a VEC Model in Matlab? Alternatively, given the model is ...
user409978's user avatar
3 votes
0 answers
244 views

Minimum Description Length, Normalized Maximum Likelihood, and Maximum A Posteriori Estimation

TL;DR: I believe MDL using NML is a special case of the joint MAP of model and parameters, and need to verify this and find sources that have acknowledges this. This is how I understand Minimum ...
Feri's user avatar
  • 177
0 votes
2 answers
79 views

Related to consistency of MLE: why $E_{\theta_0}\log\frac{f(X,\theta)}{f(X,\theta_0)}<0$?

Assumptions. $\theta_1\ne\theta_2\Rightarrow F_{\theta_1}\ne F_{\theta_2}$ The set $(x:f(x,\theta)>0)$ does not depend on $\theta$ For a.e. $x$, $f(x,\theta)$ is a differentiable function of $\...
reyna's user avatar
  • 365
0 votes
0 answers
12 views

Mixing MLE and Hypothesis Testing

In my field of biology, we have spatial data with known distances, so we know a general formula for our error structure that we can use in a generalized least squares context, it is $$ (1-\lambda)\...
A Friendly Fish's user avatar

1
2 3 4 5
69