Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

1
vote
0answers
10 views

Specifying an econometric model with unknown N

In an second-price auction setting, I observe the $y^{(2)}$ (second-order statistic) and $n$ (number of participants). The auction has a (exogenous) binding reserve price, so $n$ is a draw from a ...
1
vote
1answer
40 views

Effect of adding more sample data on Maximum Likelihood estimator [on hold]

I have samples $\{x_1, x_2, x_3, \dots , x_n\}$ of a random variable $X$. I compute Maximum Likelihood Estimator $\hat{\theta}_n$ using the sample data. Now, if I collect one more sample $x_{n+1}$ ...
0
votes
0answers
12 views

Fitting Probability Distribution to Failure Data with discrete, right censored stress

I'm a bit unfamiliar with survival analysis and I'm struggling to find examples of the particular problem I wish to tackle (which I don't think is particularly unique actually). Imagine I'm doing a ...
0
votes
0answers
16 views

How to derive the maximum *a posteriori* estimate when the prior distribution is Normal $N(m,r^2)$?

I am learning probabilities and I need a guru to help with this problem: Assume $p(y | x) = N(ax,\ s^2)$, where all quantities are scalars, $a$ and $s$ are known constants, and the prior ...
0
votes
0answers
37 views

MLE of Parameters of Bivariate Normal Distribution

I am working through find the maximum likelihood estimators of the bivariate normal distribution, without using matrices. I have the following density function: $f(Y_1,Y_2) = \frac{1}{2\pi\sigma_1\...
1
vote
0answers
31 views

R - nlm and Weibull Maximum Likelihood

Good day, I am working on an assignment where I have to Calculate the maximum likelihood estimates of $\alpha$ and $\lambda$ along with their standard erorrs on the basis of an independent and ...
9
votes
1answer
100 views

Why does the Bayesian posterior concentrate around the minimiser of KL divergence?

Consider the Bayesian posterior $\theta\mid X$. Asymptotically, its maximum occurs at the MLE estimate $\hat \theta$, which just maximizes the likelihood $\operatorname{argmin}_\theta\, f_\theta(X)$. ...
5
votes
1answer
47 views

Find maximum likelihood given Rayleigh probability function

Problem Suppose we use a Gaussian PDF to express the likelihood of light intensity prevalent on Clear, Cloudy, and Eclipse weather. The probability of a certain amount of light value (positive or ...
4
votes
1answer
62 views

Gauss Original Paper

I am looking for Gauss's 1809 paper in which he introduced least squares regression, MLE and the gaussian distribution. I cannot find it online. Can someone tell me where I may find it?
0
votes
1answer
22 views

Integrating out an extra parameter in Maximum Liklihood estimation

In estimation theory I have seen maximum likelihood being used assuming additive Gaussian AWGN where the signal is a function of multiple parameters(like frequency, time delay, phase, bit). Sometimes ...
5
votes
1answer
45 views

Is MLE intrinsically connected to logs?

My mathematical exploration led me the following claim: Claim: MLE is fundamentally connected to logs (and KL divergence, which also uses logs). It’s not correct to say log shows up simply to make ...
4
votes
3answers
148 views

MLE of $f(x\vert\theta)=1/\theta$, $x_1 , \cdots , x_n \sim U(0,\theta) \;\;, \theta>0$, [closed]

Original question $x_1 , \cdots , x_n$ are independent random variables, identically distributed as a uniform distribution over $(0,\theta)$. $$ f(x \vert \theta) = \frac{1}{\theta}, \; 0<x<\...
0
votes
0answers
69 views

Maximum likelihood estimators of $\theta$ in $U(2\theta-1,2\theta+1)$ distribution

I understand why (D) is one of the answers but i dont know about the rest?
0
votes
1answer
22 views

Train classifiers on a subset and validate on a full dataset

Let's say I want to train a classifier and use it on 20% of "known" data; the rest of the "known" data is reserved for training (60%) and testing (20%). In the end I also want to apply it on "unknown" ...
0
votes
0answers
22 views

Derive the Likelihood Ratio Test for multivariate normal and specific covariance matrix

Let $X_1,\ldots, X_n$ be i.i.d. $N(µ, C)$ random $p$-vectors. Derive the Likelihood Ratio Test for $H_0: C = σ^2(1 − ρ)I_p + ρ1_p1_p^T$, where $1_p = \begin{pmatrix} 1 \\ 1 \\ \vdots \\ 1 \end{...
0
votes
1answer
32 views

Maximum likelyhood of distribution

$L$ is the upper limit of the sample distribution $[0, L]$ which is uniform and normal. how can I show that $L=\frac{(n+1)*max(X_i)}{n}$ is unbiased. and also has a lower MSE than MLE?
0
votes
2answers
40 views

Calculating the Maximum Likelihood Estimate

I'm trying to understand MLE, but struggling with a question/example, so was hoping for an explanation if someone can help. There's a ring toss game which is repeated until it's succeeds for the ...
0
votes
1answer
46 views

Maximum Likelihood Estimator with Indicator functions

I know how to find MLE of uniform and exponential functions like maximising log likelihood etc. But I am unable to figure out the mle in the above case.
-1
votes
0answers
16 views

Understanding MLE for a Gaussian Naive Bayes classifier

I am trying to develop a text classifier and I'm reading about MLE to help me understand the process. I came across this example: and I wanted to try this myself. I'm running into a problem and so ...
2
votes
1answer
48 views

Difference between empirical distribution and the data-generating distribution? [closed]

I understand that an empirical distribution is basically sampling from the sample set with replacement. However I am not quite sure how $ \hat{p}_{data} $ and $ p_{data}$ in Maximum likelihood ...
2
votes
2answers
37 views

Maximum likelihood estimate of Gaussian given rounded observations

Suppose there is a hidden gaussian with mean $\mu$ and variance $\sigma^2$, and that $X_i \sim \mathcal{N}(\mu,\sigma^2)$ where the $X_i$ are i.i.d. If I can only oberve the rounded value of $X_i$, i....
1
vote
1answer
35 views

Invariance property

I am a bit confused regarding what exactly is the invariance property of sufficient estimators, consistent estimators and maximum likelihood estimators. As far as I know, Invariance property of ...
0
votes
1answer
75 views

how to calculate the parameter lambda in Poisson distribution?

Let's say there is a sequence: a <- c(1,2,3,1,2,1,1,3,1,2,3,5) This conforms to a Poisson distribution, the formula of which is shown as: Now I want to ...
0
votes
0answers
16 views

What is the maximum likelihood for this distribution(received signal)?

Let's assume we are sending signal $\mathbf{x}$ (which is a vector $N\times 1$) through channel $\mathbf{H}$ (a $M\times N$ matrix). Our model is $\mathbf{y}=\mathbf{Hx}+\mathbf{n}$. Note that the ...
0
votes
0answers
18 views

How would you go about estimating a friction model? (limited dependend variable)

How would you go about estimating a friction model, developed by Rosett (1959)? It is an extension of Tobit. See for instance and its references. Though it is applied in many papers in various fields,...
2
votes
1answer
26 views

Starting values in numerical algorithms?

I am estimating an ARCH(1) model, not to difficult apart from one small problems, which starting value should I use for the estimation. I am estimating it long hand, so understanding the minor details ...
1
vote
0answers
35 views

Does every loss function correspond to MLE/MAP

Many of the losses used in regression/classification tasks correspond to maximum likelihood estimation (MLE) or maximum aposteriori (MAP) under a specific data likelihood distribution $p(\mathbf{y}|X,\...
1
vote
2answers
39 views

Understand a statement about likelihood function

I'm reading Agresti - Categorical Data Analysis and it says Consider two models, $M_0$ with fitted values $\hat{\mu}_0$ and $M_1$ with fitted values $\hat{\mu}_1$ with $M_0$ a special case of $M_1$....
0
votes
1answer
17 views

Likelihood for a test data (sequence of characters) given two unigram models

I would like to find the likelihood of a sequence of characters (the test data), given two unigram models. The sequence (test data) is: A B C B B The models ...
0
votes
1answer
27 views

Do we maximize likelihood or likelihood ratio for ML estimation? [closed]

I was reading link. And I rewrite (3), here, in link to simplify notation as follows $$ \Lambda(X) = \frac{\mathcal{L}(\lambda_S | X)}{\mathcal{L}(X)} $$ Here $\lambda_S$, variance in presence of ...
0
votes
1answer
31 views

Is likelihood also defined as ratio of pdfs

My understanding of likelihood is that it is pdf except that it is a function of parameters rather than observations (as in link). I was reading link. Can likelihood be defined as ratio of ...
1
vote
1answer
39 views

Usefulness of Point Estimators: MVU vs. MLE

In a past class, two types of point estimators were introduced: minimum variance unbiased estimators (MVUs) and maximum likelihood estimators (MLEs). Supposedly, the MVU is optimal, unless an unbiased ...
1
vote
1answer
20 views

How to fit a superimposed distribution (\eg a Gaussian distribution + a Uniform distribution)

Suppose we have a set of independent observations of a random variable X, which is a Superimposition of two mutual independent random variables (i.e. X = Y + Z), Y follows a uniform distribution, ...
2
votes
0answers
49 views

Model selection with different fixed effects and different corARMA structures

I analyzed the effect of temperature (4 different areas) on laying date: LDT ~ Aa3+Bb+Cc+Dd. Because of autocorrelation in residuals I used ...
3
votes
1answer
62 views

The form of the Log-Likelihood Function in Mixed Linear Models

Let us assume the following mixed effects model: $y = X\beta+Zu+e$ where $y$ is a vector of n observable random variables, $\beta$ is a vector of $p$ fixed effects, $X$ and $Z$ are known matrices, ...
1
vote
2answers
47 views

Maximum Likelihood Estimator (MLE) for $2 \theta^2 x^{-3}$

I'm having a bit of trouble solving this. $$ f(x_i; \theta) = 2 \theta^2 x_i^{-3}, 0 \le \theta \le x_i \lt \infty $$ I start by finding $f(\textbf{x}; \theta)$: $$ f(\textbf{x}; \theta) = \prod{f(...
0
votes
0answers
23 views

Likelihood ratio test, Wald test and LM test for variance of a normal distribution

Let y1, y2....yt follow a N(0,sigma^2) distribution. [Note that the mean is zero and you know that it is zero]. Derive the LR, LM and Wald test of hypothesis sigma^2 = 1. I have got the MLE, the ...
0
votes
0answers
39 views

Find the distribution function $F$ for $min_{1 \le i \le n}{X_i}$ [duplicate]

Given a random sample $X_1, X_2, ..., X_n$ where each $X_i$ has pdf: $$ f(x; \theta) = 3 \theta^3 x^{-4} $$ and $0 \lt \theta \le x \le \infty$. Show that the distribution function $F$ for $min_{1 \...
0
votes
2answers
34 views

What does $\sum_{\{\textbf{x}:T(\textbf{x})=t\}} f(\textbf{x}; \theta)$ mean?

This is in the following context: $$ q(t;\theta) = P(T=t;\theta) = \sum_{\{\textbf{x}:T(\textbf{x})=t\}} f(\textbf{x}; \theta) $$ Where $T=T(\textbf{X})$ is a statistic, $q(t; \theta)$ is the pmf of ...
0
votes
2answers
45 views

Maximum likelihood in Naive Bayes classifier

With regards to the Naive Bayes classificator, I have read the following in Wikipedia and wanted to know why it is like that: "In many practical applications, parameter estimation for naive Bayes ...
1
vote
0answers
28 views

Trouble with MLE [closed]

I have a random sample $X_1, X_2, ..., X_n$ with $X_i$ having a pdf $$ f(x;\theta) = 2\theta^2x^{-2} $$ I'd like to find the MLE of $\theta$. First, because this is a random sample, all $X_i$ are ...
2
votes
1answer
43 views

Two approaches for finding a MLE in a binomial setting

I'm learning towards an exam in mathematical statistics and I came across the following question. I was wondering if the second approach of solving the question is legitimate. If both are correct, is ...
1
vote
1answer
27 views

Recursive Bayes Learning

I'm trying to work through an example from Richard Dudas Pattern Classification on Recursive Bayes Learning. My main question is why do we choose the $max[D^n] $ in: $$max[D^n] \le \theta \le 10 $$ ...
3
votes
0answers
34 views

Does any `R` package offer `gnorm`, `hnorm`, and similar? What about other languages?

R typically offers functions prefixed by d, p, q, and r ...
1
vote
0answers
10 views

Should lower and upper bounds for a distribution count as parameters in AIC model selection

Suppose we want a random variable $X$ to be constrained and thereby to lie within specified bounds other than the natural bounds of the underlying distribution. This should be understood such that if ...
2
votes
1answer
43 views

Independent and Identically distributed assumption in Maximum likelihood estimation

I was reading about Maximum likelihood estimation from various sources on the internet and I noticed that MLE makes an assumption about the data known as IID but I didn't completely understand why is ...
2
votes
1answer
70 views

Interpreting matrix notation to run MLE in R

I am trying to re-create some indicators from the World Bank, using the methodology described in this paper, and I need to do maximum likelihood estimation, preferably using R. The aim is to get an ...
13
votes
1answer
734 views

Why does MLE make sense, given the probability of an individual sample is 0?

This is kind of an odd thought I had while reviewing some old statistics and for some reason I can't seem to think of the answer. A continuous PDF tells us the density of observing values in any ...
0
votes
1answer
48 views

Likelihood function for linear regression

For linear regression, the likelihood function can be found with: However if your data points are multi-dimensional such that x,...
0
votes
0answers
23 views

Additive Gaussian Processes with Penalized Likelihood

I have a problem with many - say $D$ - input variables, $\mathbf x=(x_1,\dotsc,x_D)^\top$. I have have dataset $\mathcal D$ of $n$ input/outputs, with $n<D$. Only $\delta<<D$ should suffice ...