a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

1
vote
0answers
23 views

Difficulty in taking the derivative in order to derive the MLE : Unknown parameter vanishes?

The probability that there are $k$ observations within distance $t$ of $x$ can be written as : $$\mathbb{P}[N(t,k) = k] ={n-1\choose k}[f(x)H_t]^k[1-f(x)H_t]^{n-k-1}$$ The pdf of the distance from ...
3
votes
0answers
10 views

Inferences about a distribution given running maximum values

Here is a question inspired by this question from StackOverflow. Suppose you have observations of a variable which is measured once a minute, but the values are only recorded if they are greater than ...
0
votes
1answer
15 views

Holdout set for image task

I need to validate whether one or two templates/shapes are present in an image. Fitting two templates has a better maximum likelihood then fitting one template which is a clear symptom of overfitting. ...
2
votes
0answers
17 views

Likelihood maximization: MCEM algorithm versus MCMC algorithm

Hello Everyone this is my first question. I am a particle physicist and I am doing some empirical studiues on parameters estimation using different methods (this might give me some handle to study on ...
0
votes
0answers
4 views

F-test for nested models fitted over two curves with shared parameters

I am currently doing a numerical minimization routine to simultaneously fit two curves (with shared parameters) to two datasets. I've managed to show that, assuming the likelihood of the combined ...
0
votes
0answers
22 views

Maximum likelihood method

I try to calibration the parameters $\theta$ of my probabilistic model from available (limited) information at some point $\mathbf{x}_i, i=(1,...,n)$ on a 3D space defined by $[0, M]^3 \subset ...
1
vote
0answers
16 views

Existence and Uniqueness of an Estimator

The object to be observed consists of B cubes $(b_{1},\ldots,b_{B})$. The detector consists of $D$ parts namely $(d_{1},\ldots,d_{D})$. Let $p(b_{i},d_{j})$ denote the probability of detecting a ...
0
votes
1answer
64 views

Logistic regression with +1/-1 labels

I am trying to implement logistic regression where the label space is {-1,+1} instead of the usual {0,1}. I know that I can model this as a 0-1 problem but nevertheless I wanted to see if I can derive ...
2
votes
0answers
36 views

Bayesian inference when the data are distorted in an unknown manner

Say I make observations of a spatial distribution on a 3D grid. Due to unknown combination of errors, the data on the grid is non-uniformly blurred, and so we can't consider each grid point to be ...
0
votes
0answers
13 views

Idea on how to estimate a cost function

I shall appreciate ideas in solving the following problem : I need to find a maximum likelihood estimate for the distance function given in paper ...
2
votes
2answers
91 views
+50

Determine Maximum Likelihood Estimate (MLE) of loglogistic distribution

I am given two data sets containing dates and losses (in some currency). I have to determine the maximum likelihood estimates of the parameters of loglogistic distribution. I googled and found a ...
5
votes
0answers
34 views

Does using bootstrapped samples improve parameter estimates for a fitted distribution?

The R package retimes has a function for fitting an ex-Gaussian distribution to a set of observations. The method involves taking multiple bootstrapped samples of the observations, and fitting the ...
1
vote
1answer
19 views

Find the sampling distribution of the MLE of the uniform distribution [duplicate]

The MLE is $ \theta = max [x1,...,xn] $ And $ P(max [Xi] < t) = P(Xi < t)^n = P(t/\theta) $ But the question asks me to show that $ P(max[Xi]< t) = (min[\theta, t]/ \theta)^n * I[t>0] $ ...
-1
votes
1answer
24 views

Estimator $\gamma = \sum a_i\times x_i$ , where $X_i \sim \exp(t_i \theta )$ Show $\gamma$ is unbiased if $\sum a_i/t_i = 1$

I'm getting really confused with the estimators in this question! $X_i \sim \exp(t_i \theta x)$ where $t_i$ are positive constants. The MLE for $\theta = \frac n{\sum t_i x_i}$ And $\phi = ...
5
votes
3answers
86 views

Computing the Variance of an MLE

Suppose we have i.i.d. $n$ observations $(X_1,X_2,...X_n)$ from a population with density $$f_\theta(x)=\begin{cases}\theta x^{\theta-1}&\text{ if }0\leq x\leq ...
2
votes
1answer
41 views

numerical difference between sum of squared residuals and likelihood

I previously asked a question that got labelled as duplicated because I did not explain it correctly. I should not have used the regression model as an example because I can see how, by using that as ...
1
vote
0answers
31 views

Theorem A3 in Asymptotic properties of MLE for the i.n.i.d. case

Can someone explain to me why in the theorem below (case in $R^1$) $$\lim \inf I(A\cap B(u))|X_k(u)| \leq I(A\cap B)|X_k|$$ Full text of proof: ...
1
vote
0answers
22 views

A simple approach to maximum likelihood estimation for a model with no closed-form solution

I would like to estimate the best fitting parameters of a parametric model, $f(\theta)$, that does not have a closed-form solution. There are $n$ i.i.d. environmental observations and the aim is to ...
2
votes
1answer
46 views

ML estimate of exponential distribution (with censored data)

In Survival Analysis, you assume the survival time of a r.v. $X_i$ to be exponentially distributed. Considering now that I have $x_1,\dots,x_n$ "outcomes" of i.i.d r.v.'s $X_i$. Only some proportion ...
2
votes
2answers
64 views

Which one is better maximum likelihood or marginal likelihood and why?

While performing regression if we go by the definition from: What is the difference between a partial likelihood, profile likelihood and marginal likelihood? that, Maximum Likelihood Find β and θ ...
0
votes
0answers
44 views

Why is the most probable assignment for all variables in MRFs called MAP assignment?

I am new to graphical model, especially Markov Random Fields. I have a question about MAP assignment. Let say we have the graph structure and all the potential functions. MAP estimation is finding ...
2
votes
1answer
41 views

MLE for the .95 percentile of the normal distribution

The question is: let $X_1, ..., X_n \sim N(\mu, \sigma^2)$. Let $\tau$ be the .95 percentile, i.e. $P(X<\tau)$ = .95. What is the MLE of $\tau$?_ What I have tried: $P(X<\tau) = P(Z < ...
3
votes
1answer
26 views

Deriving K-means algorithm as a limit of Expectation Maximization for Gaussian Mixtures

Christopher Bishop defines the expected value of the complete-data log likelihood function (i.e. assuming that we are given both the observable data X as well as the latent data Z) as follows: $$ ...
1
vote
1answer
49 views

intercept bias in logistic case-control regression: which is the reason?

I don't understand the reason why if I use case-control sampling in a logistic regression then the intercept is biased. The book Agresti 2007 (An introduction to categorical data analysis) says: ...
3
votes
1answer
62 views

How to calculate the likelihood function

Lifetime of 3 electronic components are $X_{1} = 3, X_{2} = 1.5,$ and $X_{3} = 2.1$. THe random variables had been modeled as a random sample of size 3 from the exponential distribution with parameter ...
0
votes
0answers
26 views

Marginal distribution MLE or MCMC

I'm a bit confused about how to maximise the following likelihood: $\mathcal{L}(k, \lambda, p) \sim \mathrm{Binomial}(n, k, p)\mathrm{Poisson}(\lambda, n)$ i.e. my probability is relatd to a number ...
2
votes
1answer
100 views

Maximum likelihood estimate for uniform distribution

I stumbled to understand how to compute the MLE when talking about uniform random variable (and more generally continuous ones). The problem : Lets say we have 2 samples following the uniform ...
1
vote
0answers
7 views

Distribution of weighted data

I want to fit a distribution to weighted data, and I have some issues, starting with the parameter estimation. How can I perform the MLE with the WEIGHTED data? The histogram of the weighted data fits ...
0
votes
0answers
8 views

using EM for parameter estimation when there is no missing value or hidden variable

I am trying to estimate the parameters (conditional probabilities) for a Bayesian Network (each node corresponds to a discrete variable). There is no missing values in my data and there is no hidden ...
1
vote
2answers
55 views

Intuition for the “information matrix equality” result?

I am trying to understand the intuition behind the "information matrix equality" condition in the Maximum Likelihood context (perhaps this is the only context?): $$ -E[H(\theta)] = E[s(\theta) ...
3
votes
0answers
56 views

Deriving the log-likelihood with heteroskedastic errors in linear IV model (with interesting applications once done)

I am trying to derive the concentrated log-likelihood within a limited information maximum likelihood context. The linear model is a compacted instrumental variable regression model and I am ...
0
votes
1answer
38 views

Probit assumptions of unbiasednes

Can someone tell me what are the assumptions of unbiasednes for simple probit model like this $ Prob(y=1|x) = G^{-1}(\beta_0 + x\beta) $ I know that dependent variable models are estimated by MLE so ...
3
votes
1answer
49 views

Maximum Likelihood Estimation of Dirichlet Mean

Consider the problem of computing a Maximum-Likelihood estimate of the parameters to a finite Dirichlet distribution, given a set of multinomial observations (probability vectors) assumed to have been ...
0
votes
0answers
29 views

Calculating Log-Likelihood from Simulated Distribution

I want to perform some sort of model evaluation of a multivariate distribution with the property that it is difficult/impossible to calculate the likelihood (of the whole model, you can do it for ...
3
votes
2answers
82 views

Self-study: Finding the maximum likelihood estimates of the parameters of a density function

Consider a random sample $x_1,x_2,...,x_n$ from a newly-generated distribution, whose probability density function is given below \begin{equation} f(x|\alpha,\beta,\sigma)=\frac{1}{\Gamma \left( ...
0
votes
1answer
30 views

Finding maximum likelihood estimates of parameters of multiple normal populations

I've just started studying maximum likelihood and likelihood ratio tests. I've calculated the maximum likelihood of a normal population with unknown mean and variance. However, I've been given this ...
1
vote
1answer
59 views

Different quantiles of a fitted GPD in different R packages?

I am performing an extreme value analysis for meteorological data, to be precise for precipitation data available in mm/d. I am using a threshold excess approach for estimating the parameters of a ...
0
votes
0answers
22 views

Unsuccessful ML estimation of a modified Weibull model's parameters

I'm running this code in R to find the ML estimators of a modified Weibull model. ...
2
votes
1answer
14 views

Verify accuracy of asymptotic variance of estimator

The asymptotic variance of a maximum likelihood estimator can be obtained from the inverse of the Hessian of the log-likelihood function at the MLE, and the variance of derived quantities can be ...
0
votes
1answer
56 views

Gradient of Log-Likelihood

Considering the following functions I'm having a tough time finding the appropriate gradient function for the log-likelihood as defined below: $a_k(x)=\sum_{i=1}^D w_{ki}\cdot x_i$ $P(y_k|x) = ...
0
votes
0answers
8 views

max margin vs max posterior/likelihood advatages

I am working on some parameter learning approaches for image classification. What is the differences between the following two for image classification? max margin methods maximum likelihood/ ...
1
vote
0answers
29 views

LS vs MLE for Gaussian Conditional Random Field estimation

Is there such a thing as Least Squares estimation for the conditional mean and covariance of a conditional gaussian random field? I'm looking at this paper by Wytock and Kolter 2013, in which they ...
3
votes
2answers
47 views

Finding sampling distribution of normal MLE and likelihood

I'm reviewing old exams in preparation for a statistics final, and I'm stuck on a particular question: Suppose that you have n independent random variables $Y_i$, with each distributed normal with ...
1
vote
1answer
40 views

Probability of a model given an image

I would like to write the likelihood function for an image with respect to theoretically predicted values. Assuming uniform Gaussian noise, the pixels are statistically independent, and we can write a ...
4
votes
1answer
184 views

computing the posterior of two Gaussian probability distributions

I am a bit confused how to solve a Bayesian statistics problem. I have a parameter $\epsilon^s$ which is defined as following: $$\epsilon^s=\frac{\epsilon-g(\pi,z)}{1-g^*(\pi,z)\epsilon}$$ where ...
2
votes
1answer
37 views

In Max. Likelihood the expected score is zero for the true values. Is it also true for any other values?

The usual proof of the Expected score in ML expected score being zero goes 'similar' to this: $f(z;\theta)$ is the density function, for data $z$, and parameter vector $\theta$, so $\int ...
2
votes
0answers
56 views

Guidelines to estimate using MLE from this definition of error function

Consider a stable causal, single-input/single output, linear time-invariant, discrete-time system. The noisy output is $y_n = \sum_{i=0}^{p-1} c_i d_{n-i} + w_{n}$ where $c_i$ is the real-valued ...
2
votes
1answer
37 views

Maximum likelihood estimation from 2 exponentially distributed sample

$X_1,X_2,...,X_n$ and $Y_1,Y_2,...,Y_n$ are independent samples from the exponential distributions with parameters $\lambda$ and $\frac{1}{\lambda}$. What is the MLE for $\lambda$? I used the ...
5
votes
2answers
86 views

Confidence intervals for maximum likelihood estimator with constraints

Let us suppose I have a maximum likelihood estimator for a multivariate parameter $\vec{\theta}$. The parameter is subject to the following constraints: $\theta_i \in [0,1]$ $\sum_i \theta_i = 1$ ...
1
vote
1answer
33 views

Maximum Likelihood Estimation and Standard Errors

Suppose, I have the following model: $$ Y = X^T\beta + u_t $$ where $u_t$ ~ GARCH(1, 1) with Gaussian mixture as error distribution (or even something more weird, like normal-inverse-gaussian and ...