Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

Filter by
Sorted by
Tagged with
0
votes
0answers
15 views

Goodness of fit for a multi-level model

I'm currently trying to figure out how to properly express a 'goodness of fit' criteria for a Bayesian model that has several steps. I have a 2 dimension dataset of position values (x and y) and ...
0
votes
1answer
23 views

2nd derivative of $\sigma^2$ in normal MLE

Many times I differentiated the MLE of the normal distribution, but when it came to $\sigma$ I always stopped at the first derivative, showing that indeed: $$\hat\sigma^2 = \frac{\sum(y_i-\bar y)^2}{n}...
0
votes
1answer
34 views

ML estimator of $\theta>0$ is $\hat{\theta}_n=\frac{\sum_{i=1}^nX_i^2}{n}$ and $I(\theta)=\frac{1}{\theta^2}$. Show $\hat{\theta}_n$ is consistent

In this problem we have that $I(\theta)=\frac{1}{\theta^2}$ is the Fisher information for a general probability density function $f(x;\theta)$ and $X_1,..., X_n$ are IID random variables from this ...
1
vote
1answer
19 views

Intercept in lm() and theory not agreeing in simple linear regression example

I have data $y_i,x_i$ and the model $y_i=\alpha x_i^\beta + \epsilon_i$ where $\log{\epsilon_i} \in N(0,\sigma^2)$. Thus, I define $z_i=\log{y_i}$ and $w_i=\log{x_i}$ and my normal linear model ...
0
votes
0answers
18 views

Estimate parameters of a concrete categorical mixture model (information retrieval)

Let $f_{i,d}$ be the frequency of the word $i$ in the document $d$ and $l_d$ be the length of the document $d$. Then $P(X = i \mid D = d) = \frac{f_{i,d}}{l_{d}}$ is the probability of drawing the ...
0
votes
1answer
25 views

Lost in the introduction discussing probability/likelihood [duplicate]

I have some questions about the following paragraph which introduces a masters level course. In this unit we consider the Frequentist (i.e. counting) approach to statistical inference and computing ...
0
votes
1answer
15 views

Show heteroskedasticity

Setup: Consider a random sample of size n with binary outcome $Y_i\in\{0,1\}$. Assume $Y_i\sim Bern(\pi_i)$. Use a linear probability model so that $\pi_i=X_i^\intercal\beta$, where $X_i$ is a ...
2
votes
0answers
28 views

MLE for the sum of independent Bernoulli trials with common factor

Suppose I am computing the sum of different bernoulli trials with probability $p_i = P s_i$, where $P$ is a common factor to all trials and $s_i$ is given, how can I compute the MLE for $P$? I realize ...
0
votes
0answers
15 views

Find Infection Rate Pattern (Rt), muT and sizeV (parameter in Covid-19 Simulation [closed]

I'm working on an article and I want to do a simulation of Iran's Covid-19 data. The article is "A novel Monte Carlo simulation procedure for modeling COVID-19 spread over time Abstract". I ...
0
votes
0answers
13 views

Mean-square convergence of maximum likelihood estimators: Examples?

From what I've gleaned from the literature, Cràmer, in his 1947 monograph Methods of Mathematical Statistics, proved convergence in probability of an MLE under certain regularity conditions. ...
0
votes
0answers
15 views

Maximum Likelihood Estimation - parameter estimation

I must find the relation between a group of categorical features and a Target (label) variable T. A proxy of the dataframe I am using is the following: ...
8
votes
1answer
331 views

Maximum Likelihood Estimator on birthday paradox

I am looking into some properties of some hash functions. It is rather a short hash $16bit$ which yields up to $65536$ different values. Given that I have $M$ samples which populate $N$ out of $65536$...
0
votes
0answers
24 views

Expression for the Likelihood Function in Point Estimation

I came across this question in my statistics textbook, but I'm struggling to come up with an expression for the likelihood function. Here is the question: Assume that there are three possible traits ...
1
vote
1answer
78 views

What is the joint distribution $(\hat{\mu_1},\hat{\mu_2},\hat{\Sigma})$?

Let $X_1,...,X_{n_1}$ be an i.i.d. sample from $N_p(\mu_1,\Sigma)$ and let $Y_1,...,Y_{n_2}$ be an independent sample from $N_p(\mu_2,\Sigma)$, for some $\mu_1,\mu_2 \in \mathbb{R}^p$ and some ...
2
votes
1answer
44 views

MLE for $\Sigma$ for MVN

Let $X_1,...,X_{n_1}$ be an i.i.d. sample from $N_p(\mu_1,\Sigma)$ and let $Y_1,...,Y_{n_2}$ be an independent sample from $N_p(\mu_2,\Sigma)$, for some $\mu_1,\mu_2 \in \mathbb{R}^p$ and some ...
1
vote
2answers
35 views

What is the Likelihood function and MLE of Binomial distribution?

I searched online, so many people mix up MLE of binomial and Bernoulli distribution. They are saying: "If $X$ is $Binomial(N,\theta)$ then MLE is $\hat{\theta} = X/N$." And I don't agree ...
0
votes
1answer
24 views

Maximum likelihood observation with a constraint

I have a vector $\mathbf{x}$ of $n$ real random variables. The random variables can be assumed to follow a multivariate normal or log-normal distribution (assuming $\mathbf{x}>0$ is ok). I would ...
0
votes
0answers
25 views

R: Getting Wrong Profile Likelihood Confidence Interval Estimates

I am trying to estimate the profile likelihood confidence interval (CI) of the parameters ($\xi$, $\sigma$) of the Generalized Pareto Distribution (GPD). However, the lower estimate (left CI) of $\xi$ ...
0
votes
0answers
15 views

Are the Generalized Least Square (GLS) and Maximum Likelihood (ML) two different ways of estimation?

I was taught, that OLS and ML are two different ways of estimation. ML gives OLS estimates under met assuptions, but it doesn't change the fact the two approaches differ. If so, how is that possible ...
0
votes
0answers
58 views

How do I calculate maximum likelihood for estimating parameters when the density function is unknown (Python)?

I need to estimate parameters of a basic epsilon-greedy delta-rule reinforcement learning model using choice data of individuals- using MLE. The model calculates expectation Q for alternative i in ...
3
votes
2answers
65 views

EM algorithm for MLE from a bivariate normal sample with missing data: Stuck on M-step

I'm trying to understand applying the EM algorithm to compute the MLE in a missing data problem. Specifically, suppose $(x_1,y_1),\ldots,(x_n,y_n)$ is a random sample from the bivariate normal ...
0
votes
0answers
32 views

Asymptotic normality of an estimator

Suppose an exponential distribution : $$ f(x) = \theta e^{-\theta x} $$ The MLE for theta is the inverse of the sample mean : $$ \frac{1}{\bar{X}} $$ I want to find the asymptotic normality of this ...
18
votes
3answers
953 views

Finding most likely permutation

[Hoping that this is the right Stackexchange site; inspired from a true story seen at work] Joe has a measuring instrument and $n$ objects to be measured (say, a scale and $n$ weights). He measures ...
2
votes
0answers
42 views

Wald Test with one-sided alternative hypothesis

it is not clear to me what happens if I want to use the Wald test to test a simple hypothesis vs a one-sided alternative. I know that if I have $H_0: \theta = \theta_0$ vs $H_1: \theta \neq \theta_0$, ...
1
vote
3answers
46 views

comparison of distributions based on the maximum likelihood

I'm new to statistics, and I'm trying to understand the concept of evaluating distribution estimates. I have one observed data point of a normal distribution, say of a continuous random variable $x$, ...
1
vote
0answers
31 views

Is the so-called “maximum likelihood” problem for linear regression really a “conditional maximum likelihood” problem?

I am reading the highly praised “Mathematics for Machine Learning” by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong. And in their development of the linear regression model, they write, ...
0
votes
0answers
29 views

Is there a relationship between Maximum Likelihood Estimation and the Maximum Entropy Principle?

I know that both techniques can be used to estimate distribution from the data, but I didn't see anything in common between the two and I haven't found anything yet for the internet that relates the ...
0
votes
3answers
41 views

How do we code a maximum likelihood fitting for a simple gaussian data? [closed]

I am learning about Maximum Likelihood Estimation(MLE), What I grasped about MLE is that given some data we try to find the best distribution which will most likely output values which are similar or ...
1
vote
2answers
44 views

I'm so confused about MLE

In maximum likelihood estimation, we maximise the likelihood. I don't understand how this is possibly: for any reasonable dataset, the likelihood of hitting that EXACT data set is obviously zero! So ...
0
votes
0answers
11 views

MLE for dataset as product of PDFs confusion

I'm reading Pattern Recognition and Machine Learning by Christopher Bishop, and he writes Because our data set x is i.i.d. from a Gaussian, we can therefore write the probability of the data set of $...
0
votes
0answers
13 views

Is mle$gradient in R is the same as the score statistics?

I'm doing a study on score statistics but every time I run the coding using mle\$gradient in the maxlik library in the formula, my answer is not satisfying. By not satisfying I mean that I calculated ...
0
votes
0answers
9 views

full information maximum likelihood for missing data in R combined with a MANOVA

i would like to do a manova with full information maximum likelihood to reduce missing data. i dont find any help in the internet, just how to calculate a normal manova, but if i add the usual command ...
0
votes
0answers
3 views

Confidence Interval as Precision Morey et al (2015) Follow-Up Question

I have a follow-up question regarding the previous stack thread on confidence interval fallacy paper + precision. For those familiar with the paper, it has to do with defining the likelihood as ground ...
0
votes
0answers
4 views

Number of topics - Topic Model(BTM)

I would like to know if it could be a good idea, in order to find the optimal number of topics in a topic model (in my particular case a BTM), to look at the mean of the topics coherence scores and ...
1
vote
1answer
61 views

Understanding maximum likelihood estimation

I am told the method of maximum likelihood says we should use the model that assigns the greatest probability to the data we have observed; formally, the maximum likelihood estimator is found by ...
0
votes
0answers
10 views

How do we use plug-in distributions to make probability statements?

I am trying to understand the following problem. Given a normal distribution fitted by a maximum likelihood with $\mu_{MLE}=0.1032$ and $\hat{\sigma}_{MLE}=0.1032$ What is the probability a person ...
0
votes
0answers
35 views

Generalized Pareto Distribution (GPD) Estimation From Scratch

I'm trying to fit a GPD to a set of log-return data from scratch, however, my function's output is totally different from ismev's ...
1
vote
0answers
13 views

Statistical Interpretation of 2 data set measures

I was adviced to post here (initialy post on physics exchange but I am going to remove it). I have two independant experiments have measured $\tau_{1},\sigma_{1}$ and $\tau_{2},\sigma_{2}$ with $\...
1
vote
1answer
48 views

Bias corrrection for MLE when dealing with normally distributed small samples

When estimating the standard-deviation for samples of normally distributed data, it is sometimes necessary to account for bias in whatever estimator one chooses -- which is usually related to the ...
0
votes
0answers
17 views

Can you calculate the MLE of a CDF?

Consider the following question: Suppose $X_{i}\sim N(\beta_{1}z_{i},\sigma^2)$. For a fixed $u$ and known $\beta_{1}$ calculate the probability $X_{i}$ is less than u. Derive the MLE of this ...
0
votes
1answer
48 views

MLE based on bivariate data

Let $ x \sim Exp({\lambda}_{1}) , Y \sim Exp({\lambda}_{2})$ and are independent . We observe Z and W with Z = min(X, Y) and $W = \begin{cases} 1 &, if Z=X \\ 0 &, if Z = Y. \end{cases} $ Now ...
0
votes
0answers
15 views

Multivariate Gamma parameter estimation

Consider $X$ a d-dimensional random variable with positive values, mean $\mu\in\mathbb{R}_+^d$, and covariance matrix $\Sigma\in\mathbb{R}^{d\times d}$. If I have $n$ samples $\{ x_1, ..., x_n \}$ ...
1
vote
0answers
23 views

Given two biased coins, how to maximize rewards whenever we get a successful head on the flip of the coins?

I am going through my undergrad courses work and came across this question. You are given two potentially biased coins, each with an unknown and potentially different underlying probability of heads. ...
1
vote
1answer
62 views

How to fit a unnormalized parametric distribution with MLE?

I'm somewhat familiar with parametric estimation using MLE in the context of fitting the parameters of a distribution given a sample. Is there a way of generalizing this approach to unnormalized ...
0
votes
0answers
26 views

How to calculate standard errors for a binary logistic regression?

I would like to find the standard error, z-value, and Pr(>|z|) manually. Results from R ...
2
votes
2answers
104 views

Which definition of the likelihood function is correct?

In the online version of the Deep Learning book on chapter 5 the estimator for likelihood function is defined as: That is the product of individual probabilities. After taking the log it arrives at ...
2
votes
1answer
46 views

Is the maximum likelihood for just a single trial reasonable, counter-intuitive or usually wrong?

Let's consider $M$ bits for which each of them has been flipped by a higher being to either $0$ or $1$. We don't know anything about the underlying random distribution, still the bits are fixed and ...
0
votes
0answers
24 views

Use of conditional logit model

I am trying to estimate a statistical model that can measure ideology on Twitter in a Danish context. I'm very unsure of what i'm doing, and i hope that you guys can help me. The basic idea that i ...
0
votes
0answers
24 views

Fixed repeated sampling

Suppose we are dealing with a linear regression $𝔼[𝑌𝑖|𝑋𝑖]=\beta_0+\beta_1X_𝑖$ The distribution of $X_i$ is fixed in repeated samples. $\epsilon_i$ follows an unconditional i.i.d. normal ...
0
votes
2answers
28 views

Maximum Likelihood Estimator and finding parameters

I think I understand the process of determining a Maximum Likelihood Estimator as being similar to the machine learning process of Gradient Descent for Linear Regression in that GD for LR results in a ...

1
2 3 4 5
54