a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

1
vote
0answers
10 views

log multivariate normal differentiation with VAR process

I am trying to estimate a regime switching model with an autoregressive component using the EM algorithm. The process itself can be presented this way: $$ r_{t}= A_{n \times (n+1)} \boldsymbol ...
0
votes
0answers
15 views

Maximum likelihood estimation beta-binomial distribution with R

I have seen that there is a possibility to estimate the parameters of a beta binomial distribution with the ML method by the use of the function mle2 . But I'm having some problems maybe because of ...
1
vote
1answer
29 views

Restricted Maximum Likelihood

Why don't we use restricted maximum likelihood to estimate parameters in non-mixed models?
2
votes
1answer
31 views

Efficiency becomes 0 when sample size becomes big?

I am trying to solve Robert Hogg's mathematical anaysis 6th exercise 6.2.11. The problem says. Let $\bar{X}$ be the mean of a random sample of size n from a $N(\theta,\sigma^2)$ distribution, ...
0
votes
1answer
22 views

Log Likelihoods of Exponential Families

How can one derive the log-likelihood of the saturated model of an exponential family in general? Differentiating the log likelihood w.r.t $\theta$ gives $y_i=\hat{\mu_i}$ but I don't think replacing ...
2
votes
1answer
33 views

Prove the loglikelihood is strictly concave for ABO allele frequency blood type data

I am working through the problems in Kenn Lange's book Numerical Analysis for Statisticians. I am going to try and do all of the problems in the book, though none of them are specifically assigned for ...
0
votes
0answers
39 views

Statistics Questions on Roulette Game [on hold]

I have a few statistics questions and wanted to see if someone could give the right reasoning. Let's say there is a roulette game and a roulette wheel with equal no. Of red and black slots. The ...
1
vote
2answers
28 views

Maximum likelihood estimation of p in a Binomial sample

Assuming I need to find the ML estimator for p, p being the chance of success in a Binomial experiment $Bin(N,p)$, I would expect my density function to be: $$ f(y) = {{N}\choose{y}} p^y(1-p)^{N-y} ...
0
votes
0answers
26 views

MLE of Poisson Regression in R

Given the following Poisson Regression Model: $$ Y_1,...Y_n \sim^{indep} Poi (\lambda_i)\\ log(\lambda_i)=\beta_0+\beta_1 x_i + \beta_2 x_i^2\\ \text{for }i = 1,...n $$ I am trying to find the log ...
1
vote
0answers
23 views

Which Hessian to use to compute standard errors

Let that I have a data vector $\textbf{x} = (x_1,x_2,x_3....x_n)$ Say these are realizations of IID random variables having a common density $f_\theta$ Likelihood computed using $i^th$ observation ...
1
vote
0answers
56 views

Delta Method to Compute Standard Errors of Transformed Variables

I am estimating a a finite mixture model to identify proportions of four behavioral types using an experimental dataset. This dataset has data for 500 individuals and each individual has 30 tasks. In ...
5
votes
1answer
69 views

Estimation with MLE and returning the score/gradient (QMLE)

I am estimating a simple AR(1) process by the ML approach. I also wish to compute the Quasi MLE standard errors, which is given by the sandwich form of the Hessian and the Score (see for example the ...
0
votes
0answers
29 views

Quasi-Newton Accelerator (QN1) for EM Algorithm

I am trying to implement what is called a "very simple to implement" accelerator for the EM algorithm. Specifically I am talking about the QN1 algorithm, described here, and am using a multivariate ...
1
vote
0answers
153 views

Likelihood and sufficient statistics

a)Find the maximum likelihood estimador for $a$ in the density $f(x;a)=\frac{2}{a^2}(a-x)I_{(0,a)}(x)$. b)Is it a sufficient statistics? I did $$\prod f(x;a)=\prod_{i=1}^2 ...
2
votes
1answer
74 views

Assumptions and terminology for dynamic regression with endogenous offset ($y_t=y_{t-1}+\beta X_{t-1}+\epsilon_t$)

I'm dealing with a fairly simple time series regression model with the following basic form: $y_t=y_{t-1}+\beta X_{t-1}+\epsilon_t$ I'm assuming that observations of $y$ are known without error. $X$ ...
1
vote
2answers
45 views

How to use an optimization solver to get t-stats and p-values for the estimates?

I calculate a data log likelihood (evaluated at a set of parameters to be estimated), and my task is to find the set of parameters that maximize my log likelihood. My problem is: thought there are a ...
0
votes
0answers
10 views

choice of maximum likelihood over expectation maximisation

Given a probability distribution two common statistical measures are the expectation value and the maximum likelihood (equivalent to mean and mode?). My question is, given a probability distribution ...
1
vote
1answer
93 views

Why do you have to use MLE instead of OLS in time series data?

I know it has something to do with the errors being correlated with the variable, but I'm not sure exactly what that means. Could someone please give me a quick simple explanation about why you must ...
2
votes
0answers
63 views

Sandwich Estimator in Maximum Likelihood Estimation of Logit

I am estimating a discrete choice model using mixed logit using Halton Draws. So everything is basically numerically done. The code is written in MATLAB. I am using MATLAB's fminunc to do ...
0
votes
0answers
18 views

Full Information Maximum Likelihood, Imputation and Classification

I need to do a classification of a dataset, I have some missing data and I would like to try some "missing data techniques" to achieve the best accuracy. I already tried multiple imputation and ...
2
votes
0answers
23 views

EM algorithm: With prior vs. not prior

I have a working EM algorithm without prior. I am asking for some advice on how to add prior on latent variables. Define: $t_i \in \{ +1, -1 \} $: variables of interest to be predicted $p_j \in ...
0
votes
2answers
49 views

Estimate Number of People

In stats we just finished learning about the theory behind MLEs, so I presume this question has something to do with them ... but there is so little information I have no idea where to begin. You ...
-1
votes
1answer
23 views

Does the order of variables in a Markov Regime Switching model matter?

since Ive received feedback that my previous question was not well-recieved Ill just have to give it another shot. I am estimating Markov Regime Switching Models, and I am getting different results ...
3
votes
1answer
59 views

Stuck on a maximum likelihood estimation

I am given a random variable with the following density function : $e^{\beta-y}$. I need to find the ML estimator of $\beta$. To find ML estimators, I usually apply the following steps (as explained ...
0
votes
0answers
45 views

Estimator of The Mean of the Ratio of Uniformly Distributed Variables

Given two random variables, $ X \sim U \left[ {\mu}_{x} - \frac{{l}_{x}}{2} > 0, {\mu}_{x} + \frac{{l}_{x}}{2} \right] $ and $ Y \sim U \left[ {\mu}_{y} - \frac{{l}_{y}}{2} > 0, {\mu}_{y} + ...
5
votes
1answer
113 views

What will be the estimator for these parameters

Question: $y_0 = z^d$ is computed from the sum of some recordings by a sensor. Let, there be $k$ sensor nodes. This parameter is calculated by each sensor node and then transmitted to the base ...
6
votes
2answers
231 views

Can we use MLE to estimate Neural Network weights?

I just started to study about stats and models stuff. Currently, my understanding is that we use MLE to estimate the best parameter(s) for a model. However, when I try to understand how the neural ...
0
votes
0answers
28 views

minimization of weighted frobenius norm for pca

So my problem is i like to derive pca solution as the maximum likelihood estimate for the true data.So basically i am assuming that my measured data has two component one is low rank component and ...
1
vote
1answer
34 views

How to find the pdf and likelihood function for bernoulli gaussian model: Why it is not mixture of Gaussians

$z(k) = h^T \mathbf{y(k)} + n(k)$ is an FIR model where $\mathbf{y(k)} = [y(k),\ldots,y(k - p+1)]$ is the 0/1 input. Based on the assumption that $0/1$ follows a Bernoulli distribution. $y(k)$ is a ...
1
vote
1answer
68 views

Advantages of taking the logarithm to minimize the likelihood

In regression/classification problem, we are often interested in minimizing a cost function with respect to the parameters of the model. In many cases, the cost function is the negative likelihood. To ...
3
votes
2answers
48 views

Variance of the maximum likelihood estimator of Rayleigh Distribution

I want to calculate the variance of the maximum likelihood estimator of a Rayleigh distribution using $N$ observations. The density probability function of this distribution is : $$ f(\sigma,y_i) = ...
1
vote
1answer
52 views

Transforming continuous variable to ordinal for estimation with ordered logit

I currently have a continuous variable. However, I would like to transform it into 5 intervals using cutpoints of my choosing to carry out an ordered logit estimation. That is: Will this affect ...
1
vote
0answers
32 views

How do you do EM algorithm for a factored model for a recommender system?

Let $X$ be a $n \times d$ matrix with users as rows and movies as columns. Each user is a single row $x^{(u)} \in \mathbb{R}^d$ (i.e. for user u there are at most d ratings for the d movies). Also ...
1
vote
0answers
17 views

The most general definition of the Likelihood function for continuous data (including truncation and censoring)

How would you rigorously define the likelihood function for censored/truncated observations? Even in most lifetime/reliability literature (where these types of observations are frequently encountered) ...
0
votes
2answers
144 views

Find the mle of $\theta$

This is from Robert Hogg's Introduction to Mathematical Statistics 6th Edition Exercise 6.1.13. The question is: Let $X_{1},X_{2},...,X_{n} $ be a random sample from a distribution on $\mathbb{R}$ ...
0
votes
0answers
9 views

Significance of m parameter in m-estimate

To assign a probability to events that have not occurred yet(for a fixed set of events), one of the simplest methods is to use the m-estimator, which is defined as the following: $$Pr(A) = ...
0
votes
0answers
27 views

How can I show that $E[(\hat\theta -\theta)^2]<Var(\bar X)=\dfrac{1}{n}$? [duplicate]

Suppose $X_1, X_2, \dots, X_n$ are i.i.d $N(\theta, 1),\theta_0 \lt\theta$ , Find the MLE of $\theta$ and show that it is better than the sample mean $\bar X$ in the sense of having smaller mean ...
1
vote
1answer
50 views

Maximum Likelihood Estimate (MLE) equivalent to finding $\hat y$ in linear regression with i.i.d. Gaussian noise distribution

In an assignment I need to show that for linear regression, with the noise i.i.d. Gaussian distributed $\epsilon_i \sim N(0,\sigma^2)$, that finding the Maximum Likelihood Estimate (MLE) is equivalent ...
1
vote
0answers
13 views

How to derive type-2 maximum likelihood method for RVM regression

According to PRML book(7.85,7.86,exercise 7.12), the marginal likelihood for RVM regression is $$ \ln p(y|X,\alpha,\beta)=−1/2\{N\ln2π + \ln|C| + y^TC^{−1}y\} $$ $$ A=diag\{\alpha_1,..,\alpha_D\} $$ ...
4
votes
0answers
74 views

Are all models useless? Is any exact model possible — or useful?

This question has been festering in my mind for over a month. The February 2015 issue of Amstat News contains an article by Berkeley Professor Mark van der Laan that scolds people for using inexact ...
2
votes
1answer
56 views

Maximum likelihood of constrained distribution

A random variable $X$ is represented by the following CDF: $F(x)=(1+\frac{1}{x^2})^{-\beta}$ , $x\in(0, \infty), \beta>0$ Question: How do you get the MLE of $P(X>1)$ for the distribution? I ...
1
vote
1answer
57 views

Choosing reasonable parameters for a negative binomial distribution

My data is a list of observations and a count for each observation. The data is overdispersed, the mean is ~1,200 and the variance is ~18,000,000. I want to use a negative binomial model to assign ...
2
votes
0answers
26 views

Does maximum likelihood minimize a kind of generalized “0-1 loss”?

A very good point was raised here about how the optimal betting strategy under 0-1 loss was to bet on the mode, while under MSE loss the optimal strategy was to bet on the mean. Maximum likelihood ...
1
vote
1answer
88 views

Likelihood ratio tests using ML vs. REML

I am using Mixed effects models (nlme package in R) to choose the model with the best random and fixed effects. I am following the procedure of Zurr et al. (2009) ...
5
votes
1answer
215 views

Degeneracy paradox

Say I have a highly biased coin that lands heads with $p_h=0.01$ and tails with $p_t=0.99$, and I flip it $98$ times. The probability of zero heads is ${p_t}^{98} \approx 0.373$. The probability of ...
0
votes
0answers
70 views

Maximum likelihood method vs. least squares method

What is the main difference between maximum likelihood estimation (MLE) vs. least squares estimaton (LSE) ? Why can't we use MLE for predicting $y$ values in linear regression and vice versa? Any ...
2
votes
1answer
57 views

Motivation for gradient descent method over canonical method (for OLS/MLE) for simple linear regression?

I am beginner in machine learning and I am currently trying to find the motivation for gradient descent method. I am confused why we want to employ gradient descent method for linear regression? I see ...
5
votes
0answers
63 views

Bayesian estimates for Deming regression coinciding with least-squares estimates

Consider the following Deming model with independent replicates : $$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, ...
0
votes
0answers
23 views

Box-Cox transformation for the ordered outcome model

I wonder if there is someone out there who had the following problem. Namely, I am trying to fit an ordered logit model (-ologit-) in Stata but before that I would ...
0
votes
0answers
14 views

Information matrix at Wald statistic

AFAIK, the Wald statistic of ML estimator uses the limit normality of MLE, and it looks as: assume to test $H_0 : \theta = \theta_0$ $T = (\hat{\theta} - \theta_0)' (I(\theta_0))^{-1} (\hat{\theta} ...