Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
3,216
questions
1
vote
1
answer
244
views
Maximum likelihood estimation when parameters are functions of another data series
We have two time series: $X_t$ and $R_t$, and a model saying that $R_{t+1} = (\mu(X_t) - \frac{1}{2}\sigma^2(X_t))\Delta T + \sigma(X_t) \sqrt{\Delta T} \epsilon_t$,
where $\Delta T$ is given constant ...
- 745
4
votes
2
answers
832
views
Fitting to data with a Bernoulli (I think) distribution
I have a series of data to which I want to fit my model. The model predicts the probability of success at a given value of x. I have a single data point at a number of points in this space. As I have ...
- 1,161
3
votes
1
answer
2k
views
First order conditions for maximum likelihood estimator
Given a distribution $f(x_i,y_i;\alpha,\beta)$ and joint probability function $F(x_1,...,x_N,y_1,...,y_N;\alpha, \beta)$
The first order derivatives are $\frac{\partial F}{\partial \alpha}$ and $\...
- 153
9
votes
1
answer
982
views
Distribution of reciprocal of regression coefficient
Suppose that we have a linear model $y_i = \beta_0 + \beta_1 x_i + \epsilon_i$ that meets all the standard regression (Gauss-Markov) assumptions. We are interested in $\theta = 1/\beta_1$.
Question ...
- 13.6k
4
votes
1
answer
3k
views
Cramer-Rao lower bound questions
I've been reviewing questions from a statistics exam of the last year. There is a question with the probability density function below
$$\displaystyle f(x,\theta) = \frac 1{2\theta^3}x^2e^{-\frac x\...
- 41
3
votes
1
answer
114
views
Adjusting existing algorithm - likelihood for presence-only data
Logistic regression fits a model that predicts a binary variable whilst performing a logit transformation of the linear combination (LC) of predictors: 1/1 + exp(-LC).
I have a working machine ...
- 1,927
5
votes
1
answer
2k
views
Connection between bootstrap (both parametric and non-parametric) and maximum likelihood [duplicate]
Can anyone show me what the connection between bootstrap methods and maximum likelihood is?
I was told that as the number of bootstraps goes to infinity, the statistics like confidence intervals, ...
- 741
3
votes
1
answer
1k
views
Maximum Likelihood Estimation question - minimum log likelihood
I know the formula for the likelihood of some parameters given the data. The result has to be maximised and I can avoid multiplication using the log. How can I make this a minimisation problem (i.e. ...
- 1,927
9
votes
2
answers
683
views
A good book with equal stress on theory and math
I have had enough courses on statistics during my school years and at the university. I have a fair understanding of the concepts, such as, CI, p-values, interpreting statistical significance, ...
7
votes
2
answers
6k
views
Which link function for a regression when Y is continuous between 0 and 1?
I've always used logistic regression when Y was categorical data 0 or 1.
Now I have this dependent variable that is really a ratio/probability. That means it can be any number between 0 and 1.
I ...
- 1,388
3
votes
1
answer
881
views
Is there a known MLE for the numerator df of a sample of F statistics?
Suppose you have observations $f_1, f_2, \ldots, f_n$ which are drawn i.i.d. from a central F distribution, with unknown numerator degrees of freedom, $n_1$ and known denominator degrees of freedom, $...
- 14.1k
27
votes
5
answers
13k
views
Is there always a maximizer for any MLE problem?
I wonder if there is always a maximizer for any maximum (log-)likelihood estimation problem? In other words, is there some distribution and some of its parameters, for which the MLE problem does not ...
- 18.5k
7
votes
1
answer
13k
views
Finding expectation of reciprocal of sample mean
Consider the following distribution belonging to the exponential family.
$p_{\theta}(x) = \theta e^{-\theta x} $
The MLE estimating the $\theta$ parameter is
$\newcommand{\thetaMLE}{\hat{\theta}_{\...
- 551
3
votes
1
answer
177
views
Reason for taking the mean of experimental observations
Whenever we do some experiment in Physics, e.g. measure Planck's constant, we take the mean of several experimental observations. This seems to be the most obvious way to report a result.
Is there a ...
- 551
17
votes
1
answer
5k
views
Properties of logistic regressions
We're working with some logistic regressions and we have realized that the average estimated probability always equals the proportion of ones in the sample; that is, the average of fitted values ...
- 171
15
votes
1
answer
10k
views
Calculating likelihood from RMSE
I have a model for predicting a trajectory (x as a function of time) with several parameters. At the moment, I calculate the root mean square error (RMSE) between the predicted trajectory and the ...
- 153
4
votes
1
answer
923
views
What makes the formula for fitting logistic regression models in Hastie et al "maximum likelihood"?
I am learning logistic regression from The elements of statistical learning: data mining, inference, and prediction, by Trevor Hastie, Robert Tibshirani, Jerome H. Friedman.
Suppose $G$ is a random ...
- 18.5k
2
votes
0
answers
89
views
Paired multiarm bandit
I have a set of independent experiments with different distributions and I'm trying to determine which has the highest mean payoff. I would like to treat this as a multi-arm bandit problem, but the ...
- 21
3
votes
1
answer
306
views
-1 df for a saturated model with ML estimator?
I estimate a saturated / just-identified model (specifically: an APIM model, actor-partner-interdepence model) with the R package lavaan.
This model is saturated ...
- 4,550
6
votes
1
answer
2k
views
Maximum likelihood estimation procedures for state-space linear models
State-space models are represented by a state equation and an observation equation (or system of equations to be more precise). These equations are parametarized by components including a transition ...
- 3,051
2
votes
2
answers
484
views
Predicting a maximum value with little data
My problem is i'm trying to figure out how many servers might be required to handle a theoretical maximal load of data requests. To do that I need to know what the maximum number of requests in a ...
- 123
3
votes
3
answers
468
views
Confidence measure for classification result from a MAP estimator
I'm using a maximum a posteriori probability (MAP) estimator in a classification problem. After estimating all the a posteriori probability, the standard way is to simply take the class associated ...
- 131
13
votes
2
answers
1k
views
Hessian of profile likelihood used for standard error estimation
This question is motivated by this one. I looked up two sources and this is what I found.
A. van der Vaart, Assymptotic Statistics:
It is rarely possible to compute a profile likelihood ...
- 34.1k
1
vote
3
answers
237
views
What to call this intermediate step in my maximum a posteriori calculation?
I have calculated a posterior
$p(P_1, P_2 | D)$
where $P_1$ has a few and $P_2$ has many dimensions. In the process of calculating the maximum a posteriori estimate for $(P_1,P_2)$ given certain ...
- 125
9
votes
2
answers
9k
views
How do you calculate standard errors for a transformation of the MLE?
I need to make inference about a positive parameter $p$. To acomodate the positiveness I reparametrized $p=\exp(q)$. Using MLE routine I computed point estimate and s.e for $q$. The invariance ...
- 91
3
votes
0
answers
185
views
Quantifying the loss of efficiency incurred by a pseudo-likelihood estimator relative to the true maximum likelihood estimator
I'm working on a couple of problems where I use pseudo-likelihood (a.k.a. composite likelihood) as an estimation method since full maximum likelihood is not feasible.
To give some background without ...
- 42.5k
56
votes
2
answers
6k
views
Intuition behind why Stein's paradox only applies in dimensions $\ge 3$
Stein's Example shows that the maximum likelihood estimate of $n$ normally distributed variables with means $\mu_1,\ldots,\mu_n$ and variances $1$ is inadmissible (under a square loss function) iff $n\...
- 1,564
4
votes
1
answer
2k
views
Lognormal distribution using binned or grouped data
I understand the Max likelihood estimators for mu and sigma for the lognormal distribution when data are actual values. However I need to understand how these formulas are modified when data are ...
- 41
2
votes
2
answers
2k
views
ARMA model coefficient standard errors
I'm writing Python code to use the Kalman State-space approach to estimate ARMA model coefficients using MLE however, I'm not too clear on how to derive the coefficient estimates standard errors from ...
- 21
8
votes
1
answer
8k
views
Fitting a generalized least squares model with correlated data; use ML or REML?
Reading the Linear Mixed Model (LMM) literature I am aware that fitting a model using REML provides better estimates of variance parameters than fitting via ML. However, we should not compare nested ...
- 42.6k
5
votes
1
answer
135
views
Estimation by future likelihood maximization
Background
Conventional approaches to fitting a priori models to observed data seek to find those model parameters that maximize the likelihood of the data. For more complicated models, this ...
- 13k
2
votes
0
answers
271
views
Is there a covariance MLE which takes into account independence relationships?
In the extreme case where all of the components of an $M$-variate observation are pairwise independent from each other, a multivariate normal distribution can be decomposed into the product of $M$ ...
- 211
5
votes
2
answers
1k
views
Which measure of model fit to report when performing likelihood based regression: AIC, BIC, Pseudo R-square?
I'd like to hear your opinions on the following:
What parameters would you report when estimating different likelihood based regression? AIC, BIC, Pseudo $R^2$?
What is the standard to report?
It ...
- 5,765
7
votes
2
answers
3k
views
Does high log-likelihood imply high R^2
Does a high LL value imply that the model has a high $R^2$? I'm a very beginner to statistics so please excuse my naivete.
27
votes
4
answers
7k
views
How to ensure properties of covariance matrix when fitting multivariate normal model using maximum likelihood?
Suppose I have the following model
$$y_i=f(x_i,\theta)+\varepsilon_i$$
where $y_i\in \mathbb{R}^K$ , $x_i$ is a vector of explanatory variables, $\theta$ is the parameters of non-linear function $...
- 34.1k
6
votes
2
answers
3k
views
Two poisson random variables and likelihood ratio test
I have two cases:
Two random poisson variables $X_1 \sim \text{Pois}(\lambda_1)$, $X_2 \sim \text{Pois}(\lambda_2)$, and testing:
Null Hypothesis: $\lambda_1 = \lambda_2$
Alternate hyp: $\lambda_1 \...
- 165
8
votes
1
answer
635
views
Multinomial choice with binary observations
Is there a standard name for a multinomial choice model where the observations are in the form of binary questions such as "do you prefer A to B" and "do you prefer B to D"? This seems like a common ...
- 1,612
7
votes
3
answers
4k
views
Maximum likelihood estimation of dlmModReg
I'm studying R package dlm. So far it seems very powerful and flexible package, with nice programming interfaces and good documentation.
I've been able to successfully use dlmMLE and dlmModARMA to ...
- 171
8
votes
2
answers
10k
views
Estimating Lambda for Box Cox transformation for ANOVA
Assumptions:
In an ANOVA where the normality assumptions are violated, the Box-Cox transformation can be applied to the response variable. The lambda can be ...
- 417
22
votes
2
answers
3k
views
What are the disadvantages of the profile likelihood?
Consider a vector of parameters $(\theta_1, \theta_2)$, with $\theta_1$ the parameter of interest, and $\theta_2$ a nuisance parameter.
If $L(\theta_1, \theta_2 ; x)$ is the likelihood constructed ...
- 20.8k
33
votes
2
answers
23k
views
When should I *not* use R's nlm function for MLE?
I've run across a couple guides suggesting that I use R's nlm for maximum likelihood estimation. But none of them (including R's documentation) gives much theoretical guidance for when to use or not ...
- 1,360
0
votes
2
answers
2k
views
Simulation of maximum likelihood ratio test to test two poisson random variables
I have two random poisson variables $x_1$ and $x_2$ with value 10 and 25 respectively. I am interested to use likelihood ratio test to test the null hypothesis: $\lambda_1=\lambda_2$, versus alernate ...
- 165
2
votes
2
answers
2k
views
How difficult is it to train a gaussian mixture model compared to other models?
I have finally been able to wrap my head around the mechanics of how to initialize and train a multivariate Gaussian mixture model using expectation maximization algorithm. So I wonder how difficult ...
- 251
0
votes
0
answers
2k
views
How to convert log likelihoods into scores in Naive Bayes?
I am currently implementing a text classification program with Naive Bayes. I produce two multinominal models in my training function: p(w|nonSPAM) and p(w|SPAM)) as well as a prior probability P(S).
...
user3924
4
votes
0
answers
124
views
Factor models with small noises
The standard factor model formulation is
$y=W x+\epsilon$
where $x \sim \mathcal{N}(0, I)$, $\epsilon \sim\mathcal{N}(0, \Sigma)$. $W$ and $\Sigma$ are typically estimated from MLE. The solution can ...
- 5,440
18
votes
5
answers
4k
views
Can the empirical Hessian of an M-estimator be indefinite?
Jeffrey Wooldridge in his Econometric Analysis of Cross Section and Panel Data (page 357) says that the empirical Hessian "is not guaranteed to be positive definite, or even positive semidefinite, for ...
5
votes
2
answers
245
views
Determinant perturbation approximation
The following problem comes from a max likelihood calculation for gaussian families, but is of independent interest.
Is it possible to find a closed-form approximation for small values of $x$ for
$\...
- 5,440
12
votes
2
answers
7k
views
How do I know which method of parameter estimation to choose?
There are quite a few methods for parameter estimation out there. MLE, UMVUE, MoM, decision-theoretic, and others all seem like they have a fairly logical case for why they are useful for parameter ...
- 1,965
2
votes
1
answer
1k
views
Likelihood at MLE and transformations, the multivariate normal case
Given a univariate sample $\vec X = X_1, ..., X_n$ with standard deviation 1 and a strictly monotone transformation $t: R \to R$ with the property that the standard deviation of $t(\vec X)$ is also 1 (...
- 420
-1
votes
2
answers
2k
views
Maximum likelihood and sufficient statistics
fT(t;B,C) = exp(-t/C)-exp(-t/B) / C-B where our mean is C+B and t>0.
so far i have found my log likelihood functions and differentiated them as follows:
dl/dB = sum[t*exp(t/C) / (B^2(exp(t/c)-exp(t/...
- 107