a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

0
votes
0answers
21 views

Quasi-Newton Accelerator (QN1) for EM Algorithm

I am trying to implement what is called a "very simple to implement" accelerator for the EM algorithm. Specifically I am talking about the QN1 algorithm, described here, and am using a multivariate ...
0
votes
0answers
15 views

Method of moments and MLE [on hold]

Consider an i.i.d. sample of random variables with density function $f (x|σ) = \frac{1}{2σ} \exp(−\frac{|x|}{σ})$ a. Find the method of moments estimate of $σ$. b. Find the maximum likelihood ...
0
votes
0answers
15 views

How to find sampling distribution of MLE [on hold]

Suppose that certain electronic components have lifetimes that are exponentially distributed: $f(t|τ) = \frac{1}{τ} \exp(−\frac{t}{τ}), t ≥ 0$. Five new components are put on test, the first one fails ...
0
votes
0answers
24 views

Assumptions and terminology for dynamic regression with endogenous offset ($y_t=y_{t-1}+\beta X_{t-1}+\epsilon_t$)

I'm dealing with a fairly simple time series regression model with the following basic form: $y_t=y_{t-1}+\beta X_{t-1}+\epsilon_t$ I'm assuming that observations of $y$ are known without error. $X$ ...
1
vote
2answers
32 views

How to use an optimization solver to get t-stats and p-values for the estimates?

I calculate a data log likelihood (evaluated at a set of parameters to be estimated), and my task is to find the set of parameters that maximize my log likelihood. My problem is: thought there are a ...
0
votes
0answers
8 views

choice of maximum likelihood over expectation maximisation

Given a probability distribution two common statistical measures are the expectation value and the maximum likelihood (equivalent to mean and mode?). My question is, given a probability distribution ...
1
vote
1answer
85 views

Why do you have to use MLE instead of OLS in time series data?

I know it has something to do with the errors being correlated with the variable, but I'm not sure exactly what that means. Could someone please give me a quick simple explanation about why you must ...
2
votes
0answers
38 views

Sandwich Estimator in Maximum Likelihood Estimation of Logit

I am estimating a discrete choice model using mixed logit using Halton Draws. So everything is basically numerically done. The code is written in MATLAB. I am using MATLAB's fminunc to do ...
0
votes
0answers
18 views

Full Information Maximum Likelihood, Imputation and Classification

I need to do a classification of a dataset, I have some missing data and I would like to try some "missing data techniques" to achieve the best accuracy. I already tried multiple imputation and ...
2
votes
0answers
22 views

EM algorithm: With prior vs. not prior

I have a working EM algorithm without prior. I am asking for some advice on how to add prior on latent variables. Define: $t_i \in \{ +1, -1 \} $: variables of interest to be predicted $p_j \in ...
0
votes
2answers
48 views

Estimate Number of People

In stats we just finished learning about the theory behind MLEs, so I presume this question has something to do with them ... but there is so little information I have no idea where to begin. You ...
-1
votes
1answer
23 views

Does the order of variables in a Markov Regime Switching model matter?

since Ive received feedback that my previous question was not well-recieved Ill just have to give it another shot. I am estimating Markov Regime Switching Models, and I am getting different results ...
3
votes
1answer
56 views

Stuck on a maximum likelihood estimation

I am given a random variable with the following density function : $e^{\beta-y}$. I need to find the ML estimator of $\beta$. To find ML estimators, I usually apply the following steps (as explained ...
0
votes
0answers
45 views

Estimator of The Mean of the Ratio of Uniformly Distributed Variables

Given two random variables, $ X \sim U \left[ {\mu}_{x} - \frac{{l}_{x}}{2} > 0, {\mu}_{x} + \frac{{l}_{x}}{2} \right] $ and $ Y \sim U \left[ {\mu}_{y} - \frac{{l}_{y}}{2} > 0, {\mu}_{y} + ...
5
votes
1answer
110 views

What will be the estimator for these parameters

Question: $y_0 = z^d$ is computed from the sum of some recordings by a sensor. Let, there be $k$ sensor nodes. This parameter is calculated by each sensor node and then transmitted to the base ...
6
votes
2answers
224 views

Can we use MLE to estimate Neural Network weights?

I just started to study about stats and models stuff. Currently, my understanding is that we use MLE to estimate the best parameter(s) for a model. However, when I try to understand how the neural ...
0
votes
0answers
26 views

minimization of weighted frobenius norm for pca

So my problem is i like to derive pca solution as the maximum likelihood estimate for the true data.So basically i am assuming that my measured data has two component one is low rank component and ...
1
vote
1answer
33 views

How to find the pdf and likelihood function for bernoulli gaussian model: Why it is not mixture of Gaussians

$z(k) = h^T \mathbf{y(k)} + n(k)$ is an FIR model where $\mathbf{y(k)} = [y(k),\ldots,y(k - p+1)]$ is the 0/1 input. Based on the assumption that $0/1$ follows a Bernoulli distribution. $y(k)$ is a ...
1
vote
1answer
65 views

Advantages of taking the logarithm to minimize the likelihood

In regression/classification problem, we are often interested in minimizing a cost function with respect to the parameters of the model. In many cases, the cost function is the negative likelihood. To ...
3
votes
2answers
40 views

Variance of the maximum likelihood estimator of Rayleigh Distribution

I want to calculate the variance of the maximum likelihood estimator of a Rayleigh distribution using $N$ observations. The density probability function of this distribution is : $$ f(\sigma,y_i) = ...
1
vote
1answer
48 views

Transforming continuous variable to ordinal for estimation with ordered logit

I currently have a continuous variable. However, I would like to transform it into 5 intervals using cutpoints of my choosing to carry out an ordered logit estimation. That is: Will this affect ...
1
vote
0answers
31 views

How do you do EM algorithm for a factored model for a recommender system?

Let $X$ be a $n \times d$ matrix with users as rows and movies as columns. Each user is a single row $x^{(u)} \in \mathbb{R}^d$ (i.e. for user u there are at most d ratings for the d movies). Also ...
1
vote
0answers
16 views

The most general definition of the Likelihood function for continuous data (including truncation and censoring)

How would you rigorously define the likelihood function for censored/truncated observations? Even in most lifetime/reliability literature (where these types of observations are frequently encountered) ...
0
votes
2answers
133 views

Find the mle of $\theta$

This is from Robert Hogg's Introduction to Mathematical Statistics 6th Edition Exercise 6.1.13. The question is: Let $X_{1},X_{2},...,X_{n} $ be a random sample from a distribution on $\mathbb{R}$ ...
0
votes
0answers
9 views

Significance of m parameter in m-estimate

To assign a probability to events that have not occurred yet(for a fixed set of events), one of the simplest methods is to use the m-estimator, which is defined as the following: $$Pr(A) = ...
0
votes
0answers
27 views

How can I show that $E[(\hat\theta -\theta)^2]<Var(\bar X)=\dfrac{1}{n}$? [duplicate]

Suppose $X_1, X_2, \dots, X_n$ are i.i.d $N(\theta, 1),\theta_0 \lt\theta$ , Find the MLE of $\theta$ and show that it is better than the sample mean $\bar X$ in the sense of having smaller mean ...
1
vote
1answer
47 views

Maximum Likelihood Estimate (MLE) equivalent to finding $\hat y$ in linear regression with i.i.d. Gaussian noise distribution

In an assignment I need to show that for linear regression, with the noise i.i.d. Gaussian distributed $\epsilon_i \sim N(0,\sigma^2)$, that finding the Maximum Likelihood Estimate (MLE) is equivalent ...
1
vote
0answers
11 views

How to derive type-2 maximum likelihood method for RVM regression

According to PRML book(7.85,7.86,exercise 7.12), the marginal likelihood for RVM regression is $$ \ln p(y|X,\alpha,\beta)=−1/2\{N\ln2π + \ln|C| + y^TC^{−1}y\} $$ $$ A=diag\{\alpha_1,..,\alpha_D\} $$ ...
4
votes
0answers
72 views

Are all models useless? Is any exact model possible — or useful?

This question has been festering in my mind for over a month. The February 2015 issue of Amstat News contains an article by Berkeley Professor Mark van der Laan that scolds people for using inexact ...
2
votes
1answer
55 views

Maximum likelihood of constrained distribution

A random variable $X$ is represented by the following CDF: $F(x)=(1+\frac{1}{x^2})^{-\beta}$ , $x\in(0, \infty), \beta>0$ Question: How do you get the MLE of $P(X>1)$ for the distribution? I ...
1
vote
1answer
55 views

Choosing reasonable parameters for a negative binomial distribution

My data is a list of observations and a count for each observation. The data is overdispersed, the mean is ~1,200 and the variance is ~18,000,000. I want to use a negative binomial model to assign ...
2
votes
0answers
26 views

Does maximum likelihood minimize a kind of generalized “0-1 loss”?

A very good point was raised here about how the optimal betting strategy under 0-1 loss was to bet on the mode, while under MSE loss the optimal strategy was to bet on the mean. Maximum likelihood ...
1
vote
1answer
75 views

Likelihood ratio tests using ML vs. REML

I am using Mixed effects models (nlme package in R) to choose the model with the best random and fixed effects. I am following the procedure of Zurr et al. (2009) ...
5
votes
1answer
215 views

Degeneracy paradox

Say I have a highly biased coin that lands heads with $p_h=0.01$ and tails with $p_t=0.99$, and I flip it $98$ times. The probability of zero heads is ${p_t}^{98} \approx 0.373$. The probability of ...
0
votes
0answers
66 views

Maximum likelihood method vs. least squares method

What is the main difference between maximum likelihood estimation (MLE) vs. least squares estimaton (LSE) ? Why can't we use MLE for predicting $y$ values in linear regression and vice versa? Any ...
2
votes
1answer
53 views

Motivation for gradient descent method over canonical method (for OLS/MLE) for simple linear regression?

I am beginner in machine learning and I am currently trying to find the motivation for gradient descent method. I am confused why we want to employ gradient descent method for linear regression? I see ...
5
votes
0answers
63 views

Bayesian estimates for Deming regression coinciding with least-squares estimates

Consider the following Deming model with independent replicates : $$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, ...
0
votes
0answers
22 views

Box-Cox transformation for the ordered outcome model

I wonder if there is someone out there who had the following problem. Namely, I am trying to fit an ordered logit model (-ologit-) in Stata but before that I would ...
0
votes
0answers
14 views

Information matrix at Wald statistic

AFAIK, the Wald statistic of ML estimator uses the limit normality of MLE, and it looks as: assume to test $H_0 : \theta = \theta_0$ $T = (\hat{\theta} - \theta_0)' (I(\theta_0))^{-1} (\hat{\theta} ...
1
vote
0answers
53 views

Fitting GLM with Quasi-Newton method

I'm trying to code my own quasi-Newton algorithm for fitting GLMs in R. My results do not match up with glm and I've been over my code many, many times so I'm ...
5
votes
1answer
56 views

Finding quality of fit for two discrete variables with low statistics

I have data from an experiment which I am trying to explain using a model. I do not have an analytic formula for the prediction of the model but instead I got its prediction through a simulation. The ...
0
votes
1answer
16 views

log multivariate normal differentiation (MLE)

I've come across a lot of explanations of how to differentiate the multivariate normal, but they all appear to skip the step that I'm stuck on. Here's what I've got so far. By logging and removing ...
4
votes
1answer
114 views

Convergence from the EM Algorithm with bivariate mixture distribution

I have a mixture model which I want to find the maximum likelihood estimator of given a set of data $x$ and a set of partially observed data $z$. I have implemented both the E-step (calculating the ...
3
votes
1answer
51 views

Maximizing combined log likelihood from different dataset using r

I have observed data $y$. And I have a function that gives me estimated $\hat{y} = f(x,\hat{P})$ where P is the parameter I want to estimate. I was able to optim() command in R to get maximized ...
2
votes
1answer
23 views

On FIML assumptions

In Hayashi's Econometrics, page 529, he states one of the assumptions we need for the FIML estimator. My doubt is in the third line of point 1). He says that the vector ...
0
votes
1answer
43 views

Weibull distribution

I need to find a distribution that fail regularity condition.Maybe weibull distribution can be but I did not find why, please help me.
0
votes
0answers
22 views

Practical application of Cramer-Rao lower bound to calculate the variance of estimator

I would like to use the Cramer-Rao lower bound to help me estimate the variance of my maximum likelihood estimator, across a range of different samples of data. My question is, how do I do this ...
3
votes
2answers
117 views

What to do when your likelihood function has a double product with small values near zero - log transform doesn't work?

I currently have a likelihood function defined as the following: $$ L=\prod_{i=1}^{N}\left[\prod_{s=1}^{S_i}L_{is}(y\space|\space \rho_A)\times\phi + \prod_{s=1}^{S_i}L_{is}(y\space|\space ...
0
votes
0answers
65 views

How can the R-matrix in a mixed model be estimated?

In Henderson's Mixed Model equation: $y = X\beta + Zv + \epsilon$ where the joint variance of v and the error term is: $Var\begin{bmatrix} v \\ \epsilon \end{bmatrix} = \begin{bmatrix} G & ...
0
votes
0answers
13 views

How to estimate Mean and Variance of a Gaussian dataset of 20 numbers using ML,MAP and Bayesian Inference?

I have generated 20 random numbers from a Gaussian distribution with mean 5, and standard deviation 1. I have a question that has asked me to estimate the mean and variance using the above methods. ...