Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
3,104
questions
0
votes
0
answers
34
views
Estimating moments of censored data with multiple bounds
Suppose I draw $n$ samples of some random variable $X$. I repeat this process $k$ times so that I end up with $k \times n$ observations.
Each time I draw a random sample, my data is censored, meaning ...
- 495
1
vote
0
answers
30
views
Provide an example of a dataset where maximum likelihood is inapplicable as third moments and fourth moments "assumptions" do not apply
An additional complication arises with estimation, since maximum likelihood estimation may not be feasible without making unrealistically strong ?????"assumptions"????? about third‐ and ...
- 792
0
votes
1
answer
37
views
Finding Maximum Likelihood Estimator (MLE) when x depends on \theta [duplicate]
I am having trouble understanding how to calculate the Maximum Likelihood Estimator when x depends on $\theta$.
For example, to find the MLE of $$f(x) =\frac{2x}{\theta^2} \mbox{ where } 0\le x \leq \...
0
votes
0
answers
10
views
How to find the limiting distribution? [closed]
I have two problems in which the limiting distribution needs to be found. Both require a different method to solve and I do not know how to determine the right method. The first problem is the "...
- 131
3
votes
2
answers
54
views
Formal Definition of Identification
This definition of identification (the bracketed part) is confusing to me because (based on my obvious misunderstanding) it fails for probit:
Probit with 2 covariates: $f=\Theta(X_1\theta_1+X_2\...
- 143
4
votes
2
answers
105
views
Maximum likelihood estimator of $p$ for the binomial (truncated) distribution
This question is from the book "Introduction To The Theory Of Statistics, Mood Alexander", Chapter 7, Problem 15. In genetic investigations one frequently samples from a binomial ...
- 43
1
vote
2
answers
31
views
Normalising likelihood for BIC/AIC calculation
I am running some model inference using AIC and BIC. My problem is that when I go and calculate the (maximum) loglikelihoods of my models, they are usually really high (range between 4700 and 1400 ...
- 13
3
votes
1
answer
49
views
Does a misspecified model always have lower likelihood value than the correct model?
Suppose the true dgp is
$$
x_i \sim d_1(\theta_1), \quad i=1,\ldots,N
$$
where $d_1$ is some probability distribution with parameter(s) $\theta_1$,
but I wrongly assume
$$
x_i \sim d_2(\theta_2).
$$
...
- 812
0
votes
0
answers
11
views
How do I perform maximum likelihood estimation for one variable?
This is a simpleton's question, I appreciate. But I'm having a little confusion with discussion of dependent/independent variables, and many more things I thought I'd imagine.
I have a N samples of a ...
- 71
0
votes
0
answers
9
views
Should I used a mixed effects model to predict for a new factor level and gradually learn its coefficients with new data?
I am trying to create a predictive model for a set of industrial dryers. The dryers gradually dry product over time. A manual sample and lab test is required to produce a water measurement.
Each ...
- 356
0
votes
0
answers
22
views
Finding max value of the columns of joint probability table (from data)
I am sampling binary vectors from two correlated sources with unknown correlation and distribution (source $A$ and source $B$). For each sampling time I have a pair of binary vectors $\mathbf{v}_{A,t},...
- 21
2
votes
0
answers
34
views
Likelihood of a random vector with each component following a different distribution
How do you write down the likelihood for random vectors when each component follows a different distribution with a dependence structure?
For example, Suppose there are n-random vectors, mutually ...
1
vote
0
answers
38
views
Is there anyway to calculate the integral of a trace? [closed]
I would like to calculate the integral of a scalar function as follows:
$$f(x)=\mathrm{tr}((\mathbf{A}x+\mathbf{B})^{-1}\mathbf{B}),$$
where $\mathbf{A}$ and $\mathbf{B}$ are two $n\times n$ positive-...
- 11
1
vote
0
answers
7
views
How to estimate the parameters of a Burr XII distribution using MLE
I have a dataset and I am wondering if would be a reasonable fit for a number of distribution types.
I was looking to fit the Burr XII distribution in Python initially (using scipy library) and then ...
- 472
3
votes
3
answers
187
views
Linear probability model with crossentropy (log) loss
For better or for worse, some people shoehorn binary $y$ variables into an ordinary least squares linear regression.
$$
\mathbb E[Y\vert X]=\hat y=X\beta
$$
If we encode the $y_i$ as either $0$ or $1$,...
- 35.8k
3
votes
1
answer
65
views
How do Measure "Robustness" in Statistics?
I am an MBA Student taking courses in Statistics.
Our prof was comparing two different methods of estimating the parameters for a regression model: General Method of Moments (GMM) and Maximum ...
1
vote
0
answers
81
views
Are "Moments" More Robust Then "MLE"?
I am an MBA Student taking courses in Statistics.
We are learning about different ways to estimate the parameters (i.e. coefficients) of a Regression Model. Our professor indicated that there are two ...
0
votes
0
answers
12
views
Constructing vine copulas from separately estimated bivariate copulas?
I recently came across copulas. I then wondered if you could construct complex multivariate copulas from simpler bivariate copulas... and I discovered vine copulas. One thing that is not clear to me ...
- 65
1
vote
0
answers
21
views
Can the parameters be identified separately in this model?
Suppose we have a model
$y_i=\epsilon_i^1+\epsilon_i^2$
$\epsilon_i^1\sim N(0,\sigma_1^2)$
$\epsilon_i^2\sim N(0,\sigma_2^2)$
$\epsilon_i^1\perp\epsilon_i^2$
Assume $y_i$ is i.i.d. I have derived the ...
2
votes
2
answers
86
views
Relationship Between "Profile Likelihood" and "EM Algorithm"?
I was reading Rao (2017) (Ch3) on profile likelihood. An example is provided which shows how the parameters of a Weibull Distribution can be estimated using the "profile likelihood approach"...
- 6,078
0
votes
0
answers
33
views
General rule when max likelihood is suboptimal
Inspired by the question regarding Bessel's correction, I wonder whether there is a general rule regarding applicability of maximum likelihood for parameter estimation.
My guess is that the parameters ...
- 6,860
2
votes
1
answer
56
views
Understanding the application of MLE in Naive Bayes
I was looking at the Naive Bayes classifier models (Binomial, Multinomial and Gaussian) and trying to understand the theory behind them a bit better, but am unsure if I understand the MLE approach ...
- 173
1
vote
2
answers
249
views
Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS)
I have the following regression
$$y = X\beta +u$$
where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
- 976
0
votes
2
answers
52
views
MLE to address multicollinearity in linear regression
OLS estimation assumes that the explanatory variables are independent in the linear regression model. There isn't such assumption when using the MLE estimation. So, my question is, can we use MLE to ...
- 11
2
votes
1
answer
38
views
Maximum likelihood for mixture of Bernoullis with known mixture proportions
Given the hierarchical model
$$
\begin{align}
k & \sim \text{Categorical}(\pi_1, \dots, \pi_K) \\
X \mid k & \sim \text{Bernoulli}(\theta_{k})
\end{align}
$$
and an i.i.d. sample $X_1, \dots, ...
- 3,257
1
vote
0
answers
10
views
Minimisation of KL divergence vs minimisation of empirical processes indexed by a metric space in MLE
I am trying to relate the interpretations of MLE as (1)minimisation of KL-divergence and (2)minimisation of empirical processes indexed by a metric space.
Questions:
Is it always true that maximum ...
- 121
1
vote
0
answers
13
views
What is the MLE for regression machine learing models? [duplicate]
From my understanding, in linear regression maximizing the log-likelihood function is equivalent to maximizing the negative MSE. But what about other common regression machine learning models ...
- 11
1
vote
0
answers
42
views
Is there a general method to estimate the parameter of a Poisson distribution numerically using MLE in R? [duplicate]
I am trying to solve this problem:
Assuming for the phishing values to come from a Poisson distribution of
unknown lambda parameter, use the sample values to numerically estimate the parameter with
...
- 23
1
vote
1
answer
43
views
Maximum likelihood of Normal density under selection
Consider the density function given by
$$
\left[\dfrac{\gamma_{\leq0} \mathbb{1}(t \leq 0) + \gamma_{>0} \mathbb{1}(t > 0)}{\gamma_{\leq0}\Phi\left(- \mu / \sigma\right) + \gamma_{>0}\Phi\...
- 63
0
votes
0
answers
6
views
training a Bayesian neural network with the weighted log-likelihood in the ELBO
I have a dataset (x,y, $\Delta$y), where $\Delta$y represents the uncertainty over the measures. I am using a bayesian neural network for a regression task over this dataset and
I would like to ask if ...
- 207
3
votes
1
answer
74
views
Is the MLE variance estimator for the normal distribution asymptotically normal?
In textbooks, it is mentioned that the maximum likelihood estimators are asymptotically normal. I am having trouble with understanding how this can be true for the estimator of variance (for example, ...
- 33
0
votes
0
answers
17
views
Is gradient descent for non-parametric maximum likelihood estimation? [duplicate]
In my reading of maximum likelihood estimation, they go through samples with KNOWN distributions (e.g. binomial, poisson, etc.). I wonder how can I connect to my knowledge of machine learning.
In ...
- 145
0
votes
0
answers
14
views
What sample should be full information maximum likelihood (FIML) be applied to: FIML for longitudinal missing data in an ongoing study?
Context: I have longitudinal (epidemiological) data and plan to run latent growth curve models with four time points (baseline, one-year, two-year, and three-year follow-ups). There is missingness in ...
- 13
0
votes
0
answers
15
views
Periodic oscillations or discontinuities in likelihood function / loglikelihood function sampling over 2D grid of data
Is it expected behaviour that the loglikelihood function may have periodic oscillations within an envelope, or even periodic discontinuities for data sampled over a 2D grid?
For example, generate mock ...
- 1
0
votes
0
answers
25
views
Relation between likelihood function and conditional probability [duplicate]
I can't quite understand why the LHS equals to the RHS in the highlighted part in the following? A=theta, the model parameter. This is part of the deduction of loss function for logistic regression. ...
- 145
2
votes
1
answer
42
views
For the logit model, how is the maximum likelihood and minimum distance estimator related?
Suppose I have data $\{y_i,x_i\}_{i=1}^N$, where $x_i\in\{s_1,...,s_K\}$ and follows a discrete uniform distribution. For each realized $x_i$, $y_i$ is generated by the logit model, i.e., $Pr(Y_i=1|...
- 1,918
0
votes
0
answers
24
views
Summation signs in maximum likelihood (ridge regression example)
I can't find how to deal with summation signs ($\sum$) when performing maximum likelihood estimation. I've encountered it in ridge regression:
$\frac{1}{n} \sum_{i=1}^n (y_i-\theta^Tx_i)^2 + \lambda\...
- 93
1
vote
1
answer
55
views
Random-intercept by ID with no repeated measures: is maximum likelihood estimation dropping cases?
I have two-time point data, where I want to measure "change" in a dependent variable (si_ch_r) predicted by change in three independent variables and ...
1
vote
0
answers
16
views
Rescaling logistic regression coefficients such that variance remains constant
I'm reading "A Modern Maximum-Likelihood Theory for High-dimensional Logistic Regression" by Pragya Sur, and trying to recreate Figure 2, for my own edification. The covariates, $X$, are i.i....
- 11
0
votes
0
answers
31
views
Show that the posterior mean can be written as a weighted average of the prior mean and the MLE
From the geometric distribution pmf, $f(x_i |\theta) = (1-\theta)^{x_i -1} \theta; x_i = 1, 2, \cdots$, I have obtained the 1-parameter exponential family as $$exp \left\{\log \frac{\theta}{1-\theta} +...
user365863
1
vote
0
answers
32
views
Logistic regression using MLE on unlabeled data
I'm coming with a problem in terms of performing MLE on a dataset which is unlabeled.
What I'm doing is I have some variables (generated from some distributions) and few coefficients (predefined) ...
0
votes
0
answers
26
views
Minimum Eigenvalue of Hessian matrix is zero
I am estimating a Latent Class Model with a large number of parameters; after the statistical software ends the optimization routine, it displays a message that says:
Minimum absolute eigenvalue of ...
- 239
1
vote
1
answer
48
views
Log Likelihood for a Gaussian process regression model
According to Bishop, the author from "Statistical Pattern Recognition", we can optimize the hyperparameters of a Gaussian process by maximizing the likelihood function
$$p(\textbf{t}|\theta),...
- 73
0
votes
1
answer
33
views
Is the MLE estimate of the mean of Gamma distribution same as the average?
Consider the Gamma distribution:
$\Gamma(\alpha,\beta) = \frac{\beta^\alpha x^{\alpha-1} e^{-\beta x}}{\Gamma(\alpha)}$
According to many sources (e.g. wikipedia), the mean is given by $\mu=\alpha/\...
- 285
0
votes
1
answer
55
views
Why does my Poisson GLM with identity link converge, but a Negative Binomial GLM with identity link NOT converge?
I am trying to fit a GLM to count data, with a single feature (also a count), and intercept term. Both the feature and response are nonnegative (but may be 0). I need to use identity link because I ...
0
votes
0
answers
17
views
Is a conditional logistic regression on a 1:M case-control equivalent to a unconditional model with "centered" data?
I've been studying conditional logistic regression using this book, and it states that for 1 case- M control studies the maximum likelihood estimation of the conditional logistic regression can be ...
- 437
1
vote
1
answer
58
views
Comparing GAM models with/without fixed effect interactions using REML versus ML
I have several GAM models fit with package mgcv that share the same smooths and random effects groups. I would like to compare support for whether interactions ...
- 89
0
votes
1
answer
44
views
Can there be any situations where MLE performs better than MPS in terms of MSE or Bias?
Cheng and Amin (1983) proposed the maximum product of spacing estimation method as an alternative to maximum likelihood estimation. They stated that MPS behaves better in small sample cases than MLE ...
- 93
1
vote
1
answer
36
views
Deriving the expression for $p(\mathcal{K})$ where $\mathcal{K} = \{(\mathbf{s}^k,\mathbf{d}^k), k = 1,..., K\}$
This is a follow up from this question.
Consider a model of diseases and symptoms. $s_i\in\{0,1\}$ is a binary random variable indicating whether the patient is showing the $i$-th symptom and $d_j\in ...
- 71
0
votes
0
answers
7
views
Technical efficiency measurement using Tobit model
I want to measure the technical efficiency of layer poultry farmers producers using the Tobit model. but I collected the data and enter the excel sheet to import it into STATA 13. Now I am confusing ...
