a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

0
votes
1answer
11 views

How to estimate which distribution a new observation belongs to, when the distributions are given by a set of obervations

The question is that I have several probability distributions. For each distribution $P_i$, I don't know what it is. Instead, I have a set of observations $\{x_i^j\}_{j=1}^{n_i}$ drawn from each of ...
0
votes
0answers
10 views

Back end processing of Pattern Matrix during EFA. [on hold]

I want to know that how pattern matrix (using maximum likelihood and varimax rotation) is calculated. What is the underlying formula or pattern that displays such results? Please guide.
1
vote
2answers
37 views

Maximum likelihood estimation for Gaussian mixture

When doing maximum likelihood (ML) inference on a Gaussian mixture model (GMM), Bishop notes in PRML that if there is more than one mixture component in the GMM and the mean of one Gaussian collapses ...
1
vote
1answer
28 views

derivative of loss function

If we have a paralyzed loss function of the form of: \begin{align} L(\beta)& =\frac{1}{2}(y-X\beta)^T(y-X\beta)+ \lambda \beta^T f(\beta) \end{align} where $X_{n\times m}$ and $\beta_{m \times ...
3
votes
1answer
27 views

Is there a method to form a CI based on an MLE which is not the mean

I know how to find confidence intervals for parameters based on sample means. I want to know whether there exists a method for finding confidence intervals based on MLE's which are not sample means. ...
3
votes
1answer
46 views

Why should one use EM vs. say, Gradient Descent with MLE?

Mathematically, it's often seen that expressions and algorithms for Expectation Maximization (EM) are often simpler for mixed models, yet it seems that almost everything (if not everything) that can ...
1
vote
0answers
19 views

MLE of CTMC parameters

Let the data set be $$D = \{(s_0, t_0), (s_1, t_1), ..., (s_{N-1}, t_{N-1})\}$$ where $N=|D|$. Each $s_i$ is a state from the state space $S$ and during the time $[t_i,t_{i+1}]$ the chain is in state ...
0
votes
0answers
42 views

How to use variation in MLEs to construct a new PDF?

My simulation has 8 uncertainties and 1 output quantity of interest: Satellite mass. Each uncertainty is characterized by a Gaussian distribution. For a large number $N$; I feed these $8 \times N$ ...
0
votes
1answer
37 views

A maxdet problem with three variables - maximum likelihood estimation in Mathlab/Mata/R

Problem Given $n$ known vectors $y_j = (y_1, y_2, ..., y_{K_j})', \forall j=1,n$ and a constant $K$; determine three vectors ($\alpha, \beta, \theta^2)$ that maximise: $$ \text{T} (\alpha, \beta, ...
6
votes
2answers
250 views

Finding the maximum point of probability density function

This might be a stupid question,but i'm curious about why we always find mle using the first (partial) derivative without checking the end points or singular point or the second (partial) derivative? ...
0
votes
0answers
12 views

Find a likelihood function from varying data for Bayes Theorem

I'm not sure how to model a likelihood function for the following problem: Assume I've got a sensor producing the following raw values (normalized to an interval $\pm$1): In reality there are more ...
1
vote
0answers
40 views

Histogram binning approach to find the pdf and log-likelihood expression?

There is a paper : Presles, B., Debayle, J., Cameirao, A., Fevotte, G. and Pinoli, J.~C., Volume estimation of 3D particles with known convex shapes from its projected areas, IEEE International ...
1
vote
0answers
14 views

The score of a dynamic model is a martingale difference sequence

I am going to write down some parts of Dynamic models for volatility and heavy tails by Andrew Harvey (2008) with my comments in bold and then ask for an alternative explanation of the final part. ...
2
votes
1answer
43 views

A simple question about MAP and MLE

I recently got this simple question from a friend. But I am quite confused about it. Suppose we toss a coin $N$ times, and got heads $m$ times. Assume the binomial distribution with $p$ which is the ...
1
vote
0answers
15 views

Relation between bootstrap mean and parameter value estimated via maximum likelihood

I have an observed data set $O$ and a synthetic model $S(\theta)$ which attempts to describe it. By fixing $\theta$ to different values I can generate $M$ synthetic realizations of the model: $$S_k ...
0
votes
1answer
46 views

Finding the Maximum Likelihood Estimate in R [closed]

How do I estimate the parameters through MLE for a Likelihood function like. Note that the function has 2 multiply signs and there is a integration term in the second one.
2
votes
1answer
38 views

Likelihood ratio for normal distribution with known variance

Let $X_1,...,X_n$ random sample of $X$~$N(\mu,\sigma^2)$ with known $\sigma^2$.Take $a=.05$ find the expression for power function of the likelihood-ratio test $$H_0:\mu\leq 0\space vs\space ...
2
votes
1answer
46 views

Likelihood-ratio test

I was studying on this subject and I got some questions. Lets take the test $$H_0:\theta\in\Theta_0 \space vs\space H_1:\theta\in\Theta_0^c$$ where $\Theta$ is the parametric space and ...
0
votes
0answers
10 views

multiple imputation - likelihood base

in nonignorable mechanism, selection model or pattern-mixture model is a multiple imputation method or a likelihood-base method? I am confused, i know in MI, missing data were filled in and then We ...
5
votes
1answer
61 views

Do IID experiments maximize Fisher information?

Let $x$ and $y$ be random variables whose probabilities depend on an unknown parameter $\theta$. I am specifically interested in the case that both $x$ and $y$ are Bernoulli, but the question below ...
1
vote
1answer
41 views

Maximum Likelihood, normal distribution

Let $Z$~$N(\mu,\sigma^2+1)$, find the maximum likelihood estimator for $\mu$ and $\sigma^2$. I did but I want to check that this right actually ...
2
votes
1answer
61 views

Modelling time varying volatility when GARCH(1,1) coefficients sum to value greater one

First of all I have to admit that I am not a time-series expert so any help is highly appreciated. I have a financial time-series (a fixed income total return index measured on a weekly basis) for ...
1
vote
1answer
43 views

Estimating ARCH model using ML or OLS

ARCH(p) models are defined as: $σ^2 = a_0 + ∑a_ie^2_{t-i}+e_t$, $i>0$ Now, as with any VMA model, estimating this model using OLS/ML is impossible, because the error term is not observable. But ...
0
votes
0answers
8 views

Closed form solution for logistic scale parameter?

Several distributions have closed forms for the maximum likelihood estimate their scale parameter. For example, the maximum likelihood estimate for the standard deviation is the square root of the ...
1
vote
0answers
23 views

Maximum likelihood estimator for minimum from a vector of random variables

Let $X_i, i =1,\ldots,N$ be vectors of random varaibles. Each of them has $m$ components representing dimension, $X_{ij}, j=1,2,\ldots,m$. Specific values $x_{ij}$ are observations or data. From, ...
3
votes
1answer
84 views

Likelihood-based hypothesis testing

$N_A$ and $N_B$ are variables of the counts of the number of events 'A' and events 'B' respectively. Those variables follow Poisson distributions with parameters $\lambda_A$ and $\lambda_B$. In ...
8
votes
1answer
125 views

Why exactly is the observed Fisher information used?

In the standard maximum likelihood setting (iid sample $Y_{1}, \ldots, Y_{n}$ from some distribution with density $f_{y}(y|\theta_{0}$)) and in case of a correctly specified model the Fisher ...
0
votes
0answers
19 views

Can we interpret the size of log likelihood value?

Does low or high likelihood value after an optimisation process indicate towards something? Can we interpret it?
0
votes
0answers
13 views

Fitting Data with Ties Using unique() Walk-Around and ks.test

I'm looking for a distribution that best fits my data. Looking at the frequency plot, seems like the distribution could follow a Gamma or Weibull distribution. With that in mind, I use the ...
5
votes
1answer
84 views

Is the MAP estimate always identical to the ML estimate when a diffuse prior is used?

Let's say I have two normally distributed variables: say height and body mass. I want to estimate the Pearson's correlation coefficient between them. I have a model that estimates the mean and SD of ...
0
votes
0answers
36 views

Multinomial distribution pdf and fisher information matrix

The question is from Robert Hogg etal's Introduction to Mathematical Statistics 6th Edition Page 347 Example 6.4.5. It says: Consider a random trial which can result in one, and only one, of k ...
0
votes
0answers
41 views

Singular Hessian/Observed information Matrix at optimal solution

I am trying to estimate the standard errors of an maximum likelihood estimate (multidimensional) in R'sfunction optim. I want to ...
0
votes
0answers
12 views

Is a skill score based on mean squared error appropriate for evaluating regression models based on maximum likelihood estimation?

I would like to evaluate a generalized linear model assuming a gamma distribution of the target variable and with one to several predictors by means of cross-validation. With the observational series ...
1
vote
0answers
37 views

How can we get “estimated standard deviation” from MLE?

I'm looking for MLE as in the following picture. I have some questions: How can we get "the estimated standard deviation"? Is "the estimated standard deviation" included in the output of the ...
9
votes
1answer
91 views

Deriving likelihood function for IV-probit

So I have a binary model where $y_1^*$ is the latent unobserved variable and $y_1 \in \{0,1\}$ the observed. $y_2$ determines $y_1$ and $z_2$ is thus my instrument. So in short the model is. ...
1
vote
1answer
58 views

Testing a power-law hypothesis from the averaged distribution

A way to test the existence of power-law in the distribution is given in the following paper: http://arxiv.org/abs/0706.1062 The gist of the whole procedure is as follows: Let $\textbf{X}$ be the ...
0
votes
0answers
11 views

how to write these two inequality? (Asymptotic properties of m estimator)

$T_n^*:=sup\{t|\sum ψ(x_i;t)\gt 0\}$ $T_n^{**}:=inf\{t|\sum ψ(x_i;t)\lt 0\}$ As it's seen in the above figure, $-\infty \lt T_n^{*} \le T_n^{**} \lt +\infty$ Then, how to show that the followings ...
2
votes
1answer
39 views

Is maximum likelihood a form of data substitution? Or not?

I’m using maximum likelihood with missing data. In this case of missing data, is maximum likelihood a form of data substitution? I’m significantly more familiar with multiple imputation which I ...
2
votes
0answers
21 views

Papers suggesting ML and Bootstrapping can be used together?

Does anybody have any good research papers for me to read on whether using bootstrapping and maximum likelihood estimation together is a good idea, particularly when ML is being used with a relatively ...
1
vote
0answers
38 views

Profile likelihood

I am considering a normal distribution with mean $\beta_1 + \beta_2\exp(-\phi x)$ and variance $\sigma^2$, i.e. $y \sim N(\beta_1 + \beta_2\exp(-\phi x), \sigma^2) $. My aim is to calculate the ...
1
vote
1answer
56 views

Can we express logistic loss minimization as a maximum likelihood problem?

I have a simple question about the equivalence of loss minimization and likelihood maximization for logistic regression. Say are given some data $(x_i,y_i) \text{ for } ~i = 1,\ldots,N$ where $x_i ...
0
votes
0answers
24 views

explain the interpretation of the following proposition and how to prove it?

Please explain the interpretation of the following proposition and how to prove it? Proposition: Assume that $\exists t_0 $ s.t. $\lambda(t)\gt 0 $ for $t \lt t_0$ and s.t. $\lambda(t)\lt 0 $ ...
1
vote
1answer
87 views

Proof of a step of a lemma on the asymptotics of maximum likelihood where a Taylor expansion is used

I am trying to understand a proof of quite a long theorem that I report completely for the sake of completeness. This is From Jensen and Rahbek Asymptotic Inference for Nonstationary GARCH (2004). My ...
2
votes
1answer
41 views

Censored Binomial model - log likelihood

I have a dataset with multiple samples of batches of observations (e.g. one batch of 20 obs., one batch of 50 obs,, etc.). There is a probability that the batches have contaminated observations, with ...
2
votes
0answers
51 views

Show that MLE of $\lambda = \frac{n-T_n}{S_n+cT_n}$

$X_i$ are i.i.d exponential, mean $\lambda^{-1}$ for $1 \leq i \leq n$ and, the values are measured such that $X_i = c$ if $X_i \geq c$ and $X_i$ otherwise. Show that MLE of $\lambda = ...
0
votes
0answers
57 views

Confidence interval of the mean for a beta distribution when alpha and beta are estimated [duplicate]

I have elementary knowledge in statistics. I'm trying to estimate the confidence interval for mean of a beta distribution as specified in this article using log likelihood estimation given alpha, beta ...
3
votes
2answers
60 views

Bernoulli maximum likelihood

Suppose $X_1, X_2, \dots, X_n$ are iid Bernoulli(p) random variables. How do you find the restricted maximum likelihood for p where $0<p<0.5$? My work so far: Write out the likelihood: ...
3
votes
0answers
32 views

Understanding variance estimation by restricted maximum likelihood (REML)

I'm reading Doug Bates' theory paper on R's lme4 package to better understand the nitty-gritty of mixed models, and came across an intriguing result that I'd like to understand better, about using ...
0
votes
0answers
35 views

Estimation with ML or Bayesian

A marketing department is supposed to find the market share of their product. To answer this question, a survey among 720 representative people is conducted, 696 of which complete the poll with ...
4
votes
1answer
43 views

How to find maximum likelihood of multiple exponential distributions with different parameter values

Let's say that I have a bunch of independent samples, $X_1, X_2, \dots, X_n$ and that they all follow Exponential($\theta_i$) distributions. (So they all have pdf $f(x_i)=\theta_i\exp(-\theta_iy_i)$.) ...