Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
2,672
questions
0
votes
0answers
15 views
Goodness of fit for a multi-level model
I'm currently trying to figure out how to properly express a 'goodness of fit' criteria for a Bayesian model that has several steps. I have a 2 dimension dataset of position values (x and y) and ...
0
votes
1answer
23 views
2nd derivative of $\sigma^2$ in normal MLE
Many times I differentiated the MLE of the normal distribution, but when it came to $\sigma$ I always stopped at the first derivative, showing that indeed:
$$\hat\sigma^2 = \frac{\sum(y_i-\bar y)^2}{n}...
0
votes
1answer
34 views
ML estimator of $\theta>0$ is $\hat{\theta}_n=\frac{\sum_{i=1}^nX_i^2}{n}$ and $I(\theta)=\frac{1}{\theta^2}$. Show $\hat{\theta}_n$ is consistent
In this problem we have that $I(\theta)=\frac{1}{\theta^2}$ is the Fisher information for a general probability density function $f(x;\theta)$ and $X_1,..., X_n$ are IID random variables from this ...
1
vote
1answer
19 views
Intercept in lm() and theory not agreeing in simple linear regression example
I have data $y_i,x_i$ and the model $y_i=\alpha x_i^\beta + \epsilon_i$ where $\log{\epsilon_i} \in N(0,\sigma^2)$. Thus, I define $z_i=\log{y_i}$ and $w_i=\log{x_i}$ and my normal linear model ...
0
votes
0answers
18 views
Estimate parameters of a concrete categorical mixture model (information retrieval)
Let $f_{i,d}$ be the frequency of the word $i$ in the document $d$ and $l_d$ be the length of the document $d$. Then $P(X = i \mid D = d) = \frac{f_{i,d}}{l_{d}}$ is the probability of drawing the ...
0
votes
1answer
25 views
Lost in the introduction discussing probability/likelihood [duplicate]
I have some questions about the following paragraph which introduces a masters level course.
In this unit we consider the Frequentist (i.e. counting) approach to
statistical inference and computing ...
0
votes
1answer
15 views
Show heteroskedasticity
Setup: Consider a random sample of size n with binary outcome $Y_i\in\{0,1\}$. Assume $Y_i\sim Bern(\pi_i)$. Use a linear probability model so that $\pi_i=X_i^\intercal\beta$, where $X_i$ is a ...
2
votes
0answers
28 views
MLE for the sum of independent Bernoulli trials with common factor
Suppose I am computing the sum of different bernoulli trials with probability $p_i = P s_i$, where $P$ is a common factor to all trials and $s_i$ is given, how can I compute the MLE for $P$? I realize ...
0
votes
0answers
15 views
Find Infection Rate Pattern (Rt), muT and sizeV (parameter in Covid-19 Simulation [closed]
I'm working on an article and I want to do a simulation of Iran's Covid-19 data.
The article is "A novel Monte Carlo simulation procedure for modeling COVID-19 spread over time Abstract". I ...
0
votes
0answers
13 views
Mean-square convergence of maximum likelihood estimators: Examples?
From what I've gleaned from the literature, Cràmer, in his 1947 monograph Methods of Mathematical Statistics, proved convergence in probability of an MLE under certain regularity conditions. ...
0
votes
0answers
15 views
Maximum Likelihood Estimation - parameter estimation
I must find the relation between a group of categorical features and a Target (label) variable T.
A proxy of the dataframe I am using is the following:
...
8
votes
1answer
331 views
Maximum Likelihood Estimator on birthday paradox
I am looking into some properties of some hash functions. It is rather a short hash $16bit$ which yields up to $65536$ different values.
Given that I have $M$ samples which populate $N$ out of $65536$...
0
votes
0answers
24 views
Expression for the Likelihood Function in Point Estimation
I came across this question in my statistics textbook, but I'm struggling to come up with an expression for the likelihood function. Here is the question:
Assume that there are three possible traits ...
1
vote
1answer
78 views
What is the joint distribution $(\hat{\mu_1},\hat{\mu_2},\hat{\Sigma})$?
Let $X_1,...,X_{n_1}$ be an i.i.d. sample from $N_p(\mu_1,\Sigma)$ and let $Y_1,...,Y_{n_2}$ be an independent sample from $N_p(\mu_2,\Sigma)$, for some $\mu_1,\mu_2 \in \mathbb{R}^p$ and some ...
2
votes
1answer
44 views
MLE for $\Sigma$ for MVN
Let $X_1,...,X_{n_1}$ be an i.i.d. sample from $N_p(\mu_1,\Sigma)$ and let $Y_1,...,Y_{n_2}$ be an independent sample from $N_p(\mu_2,\Sigma)$, for some $\mu_1,\mu_2 \in \mathbb{R}^p$ and some ...
1
vote
2answers
35 views
What is the Likelihood function and MLE of Binomial distribution?
I searched online, so many people mix up MLE of binomial and Bernoulli distribution.
They are saying:
"If $X$ is $Binomial(N,\theta)$ then MLE is $\hat{\theta} = X/N$."
And I don't agree ...
0
votes
1answer
24 views
Maximum likelihood observation with a constraint
I have a vector $\mathbf{x}$ of $n$ real random variables. The random variables can be assumed to follow a multivariate normal or log-normal distribution (assuming $\mathbf{x}>0$ is ok). I would ...
0
votes
0answers
25 views
R: Getting Wrong Profile Likelihood Confidence Interval Estimates
I am trying to estimate the profile likelihood confidence interval (CI) of the parameters ($\xi$, $\sigma$) of the Generalized Pareto Distribution (GPD). However, the lower estimate (left CI) of $\xi$ ...
0
votes
0answers
15 views
Are the Generalized Least Square (GLS) and Maximum Likelihood (ML) two different ways of estimation?
I was taught, that OLS and ML are two different ways of estimation. ML gives OLS estimates under met assuptions, but it doesn't change the fact the two approaches differ. If so, how is that possible ...
0
votes
0answers
58 views
How do I calculate maximum likelihood for estimating parameters when the density function is unknown (Python)?
I need to estimate parameters of a basic epsilon-greedy delta-rule reinforcement learning model using choice data of individuals- using MLE.
The model calculates expectation Q for alternative i in ...
3
votes
2answers
65 views
EM algorithm for MLE from a bivariate normal sample with missing data: Stuck on M-step
I'm trying to understand applying the EM algorithm to compute the MLE in a missing data problem.
Specifically, suppose $(x_1,y_1),\ldots,(x_n,y_n)$ is a random sample from the bivariate normal ...
0
votes
0answers
32 views
Asymptotic normality of an estimator
Suppose an exponential distribution :
$$ f(x) = \theta e^{-\theta x} $$
The MLE for theta is the inverse of the sample mean :
$$ \frac{1}{\bar{X}} $$
I want to find the asymptotic normality of this ...
18
votes
3answers
953 views
Finding most likely permutation
[Hoping that this is the right Stackexchange site; inspired from a true story seen at work]
Joe has a measuring instrument and $n$ objects to be measured (say, a scale and $n$ weights). He measures ...
2
votes
0answers
42 views
Wald Test with one-sided alternative hypothesis
it is not clear to me what happens if I want to use the Wald test to test a simple hypothesis vs a one-sided alternative. I know that if I have $H_0: \theta = \theta_0$ vs $H_1: \theta \neq \theta_0$, ...
1
vote
3answers
46 views
comparison of distributions based on the maximum likelihood
I'm new to statistics, and I'm trying to understand the concept of evaluating distribution estimates.
I have one observed data point of a normal distribution, say of a continuous random variable $x$, ...
1
vote
0answers
31 views
Is the so-called “maximum likelihood” problem for linear regression really a “conditional maximum likelihood” problem?
I am reading the highly praised “Mathematics for Machine Learning” by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong.
And in their development of the linear regression model, they write,
...
0
votes
0answers
29 views
Is there a relationship between Maximum Likelihood Estimation and the Maximum Entropy Principle?
I know that both techniques can be used to estimate distribution from the data, but I didn't see anything in common between the two and I haven't found anything yet for the internet that relates the ...
0
votes
3answers
41 views
How do we code a maximum likelihood fitting for a simple gaussian data? [closed]
I am learning about Maximum Likelihood Estimation(MLE), What I grasped about MLE is that given some data we try to find the best distribution which will most likely output values which are similar or ...
1
vote
2answers
44 views
I'm so confused about MLE
In maximum likelihood estimation, we maximise the likelihood.
I don't understand how this is possibly: for any reasonable dataset, the likelihood of hitting that EXACT data set is obviously zero! So ...
0
votes
0answers
11 views
MLE for dataset as product of PDFs confusion
I'm reading Pattern Recognition and Machine Learning by Christopher Bishop, and he writes
Because our data set x is i.i.d. from a Gaussian, we can therefore write the probability of the data set of $...
0
votes
0answers
13 views
Is mle$gradient in R is the same as the score statistics?
I'm doing a study on score statistics but every time I run the coding using mle\$gradient in the maxlik library in the formula, my answer is not satisfying. By not satisfying I mean that I calculated ...
0
votes
0answers
9 views
full information maximum likelihood for missing data in R combined with a MANOVA
i would like to do a manova with full information maximum likelihood to reduce missing data.
i dont find any help in the internet, just how to calculate a normal manova, but if i add the usual command ...
0
votes
0answers
3 views
Confidence Interval as Precision Morey et al (2015) Follow-Up Question
I have a follow-up question regarding the previous stack thread on confidence interval fallacy paper + precision.
For those familiar with the paper, it has to do with defining the likelihood as ground ...
0
votes
0answers
4 views
Number of topics - Topic Model(BTM)
I would like to know if it could be a good idea, in order to find the optimal number of topics in a topic model (in my particular case a BTM), to look at the mean of the topics coherence scores and ...
1
vote
1answer
61 views
Understanding maximum likelihood estimation
I am told
the method of maximum likelihood says we should use the model that
assigns the greatest probability to the data we have observed;
formally, the maximum likelihood estimator is found by ...
0
votes
0answers
10 views
How do we use plug-in distributions to make probability statements?
I am trying to understand the following problem.
Given a normal distribution fitted by a maximum likelihood with
$\mu_{MLE}=0.1032$ and $\hat{\sigma}_{MLE}=0.1032$ What is the
probability a person ...
0
votes
0answers
35 views
Generalized Pareto Distribution (GPD) Estimation From Scratch
I'm trying to fit a GPD to a set of log-return data from scratch, however, my function's output is totally different from ismev's ...
1
vote
0answers
13 views
Statistical Interpretation of 2 data set measures
I was adviced to post here (initialy post on physics exchange but I am going to remove it).
I have two independant experiments have measured $\tau_{1},\sigma_{1}$ and $\tau_{2},\sigma_{2}$ with $\...
1
vote
1answer
48 views
Bias corrrection for MLE when dealing with normally distributed small samples
When estimating the standard-deviation for samples of normally distributed data, it is sometimes necessary to account for bias in whatever estimator one chooses -- which is usually related to the ...
0
votes
0answers
17 views
Can you calculate the MLE of a CDF?
Consider the following question:
Suppose $X_{i}\sim N(\beta_{1}z_{i},\sigma^2)$. For a fixed $u$ and known $\beta_{1}$ calculate the probability $X_{i}$ is less than u. Derive the MLE of this ...
0
votes
1answer
48 views
MLE based on bivariate data
Let $ x \sim Exp({\lambda}_{1}) , Y \sim Exp({\lambda}_{2})$ and are independent .
We observe Z and W with
Z = min(X, Y) and $W = \begin{cases} 1 &, if Z=X \\ 0 &, if Z = Y. \end{cases} $
Now ...
0
votes
0answers
15 views
Multivariate Gamma parameter estimation
Consider $X$ a d-dimensional random variable with positive values, mean $\mu\in\mathbb{R}_+^d$, and covariance matrix $\Sigma\in\mathbb{R}^{d\times d}$. If I have $n$ samples $\{ x_1, ..., x_n \}$ ...
1
vote
0answers
23 views
Given two biased coins, how to maximize rewards whenever we get a successful head on the flip of the coins?
I am going through my undergrad courses work and came across this question.
You are given two potentially biased coins, each with an unknown and
potentially different underlying probability of heads. ...
1
vote
1answer
62 views
How to fit a unnormalized parametric distribution with MLE?
I'm somewhat familiar with parametric estimation using MLE in the context of fitting the parameters of a distribution given a sample. Is there a way of generalizing this approach to unnormalized ...
0
votes
0answers
26 views
How to calculate standard errors for a binary logistic regression?
I would like to find the standard error, z-value, and Pr(>|z|) manually.
Results from R
...
2
votes
2answers
104 views
Which definition of the likelihood function is correct?
In the online version of the Deep Learning book on chapter 5 the estimator for likelihood function is defined as:
That is the product of individual probabilities. After taking the log it arrives at ...
2
votes
1answer
46 views
Is the maximum likelihood for just a single trial reasonable, counter-intuitive or usually wrong?
Let's consider $M$ bits for which each of them has been flipped by a higher being to either $0$ or $1$. We don't know anything about the underlying random distribution, still the bits are fixed and ...
0
votes
0answers
24 views
Use of conditional logit model
I am trying to estimate a statistical model that can measure ideology on Twitter in a Danish context. I'm very unsure of what i'm doing, and i hope that you guys can help me. The basic idea that i ...
0
votes
0answers
24 views
Fixed repeated sampling
Suppose we are dealing with a linear regression $𝔼[𝑌𝑖|𝑋𝑖]=\beta_0+\beta_1X_𝑖$
The distribution of $X_i$ is fixed in repeated samples.
$\epsilon_i$ follows an unconditional i.i.d. normal ...
0
votes
2answers
28 views
Maximum Likelihood Estimator and finding parameters
I think I understand the process of determining a Maximum Likelihood Estimator as being similar to the machine learning process of Gradient Descent for Linear Regression in that GD for LR results in a ...
