Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

Filter by
Sorted by
Tagged with
0 votes
0 answers
34 views

Estimating moments of censored data with multiple bounds

Suppose I draw $n$ samples of some random variable $X$. I repeat this process $k$ times so that I end up with $k \times n$ observations. Each time I draw a random sample, my data is censored, meaning ...
  • 495
1 vote
0 answers
30 views

Provide an example of a dataset where maximum likelihood is inapplicable as third moments and fourth moments "assumptions" do not apply

An additional complication arises with estimation, since maximum likelihood estimation may not be feasible without making unrealistically strong ?????"assumptions"????? about third‐ and ...
0 votes
1 answer
37 views

Finding Maximum Likelihood Estimator (MLE) when x depends on \theta [duplicate]

I am having trouble understanding how to calculate the Maximum Likelihood Estimator when x depends on $\theta$. For example, to find the MLE of $$f(x) =\frac{2x}{\theta^2} \mbox{ where } 0\le x \leq \...
0 votes
0 answers
10 views

How to find the limiting distribution? [closed]

I have two problems in which the limiting distribution needs to be found. Both require a different method to solve and I do not know how to determine the right method. The first problem is the "...
  • 131
3 votes
2 answers
54 views

Formal Definition of Identification

This definition of identification (the bracketed part) is confusing to me because (based on my obvious misunderstanding) it fails for probit: Probit with 2 covariates: $f=\Theta(X_1\theta_1+X_2\...
4 votes
2 answers
105 views

Maximum likelihood estimator of $p$ for the binomial (truncated) distribution

This question is from the book "Introduction To The Theory Of Statistics, Mood Alexander", Chapter 7, Problem 15. In genetic investigations one frequently samples from a binomial ...
  • 43
1 vote
2 answers
31 views

Normalising likelihood for BIC/AIC calculation

I am running some model inference using AIC and BIC. My problem is that when I go and calculate the (maximum) loglikelihoods of my models, they are usually really high (range between 4700 and 1400 ...
  • 13
3 votes
1 answer
49 views

Does a misspecified model always have lower likelihood value than the correct model?

Suppose the true dgp is $$ x_i \sim d_1(\theta_1), \quad i=1,\ldots,N $$ where $d_1$ is some probability distribution with parameter(s) $\theta_1$, but I wrongly assume $$ x_i \sim d_2(\theta_2). $$ ...
  • 812
0 votes
0 answers
11 views

How do I perform maximum likelihood estimation for one variable?

This is a simpleton's question, I appreciate. But I'm having a little confusion with discussion of dependent/independent variables, and many more things I thought I'd imagine. I have a N samples of a ...
  • 71
0 votes
0 answers
9 views

Should I used a mixed effects model to predict for a new factor level and gradually learn its coefficients with new data?

I am trying to create a predictive model for a set of industrial dryers. The dryers gradually dry product over time. A manual sample and lab test is required to produce a water measurement. Each ...
  • 356
0 votes
0 answers
22 views

Finding max value of the columns of joint probability table (from data)

I am sampling binary vectors from two correlated sources with unknown correlation and distribution (source $A$ and source $B$). For each sampling time I have a pair of binary vectors $\mathbf{v}_{A,t},...
2 votes
0 answers
34 views

Likelihood of a random vector with each component following a different distribution

How do you write down the likelihood for random vectors when each component follows a different distribution with a dependence structure? For example, Suppose there are n-random vectors, mutually ...
1 vote
0 answers
38 views

Is there anyway to calculate the integral of a trace? [closed]

I would like to calculate the integral of a scalar function as follows: $$f(x)=\mathrm{tr}((\mathbf{A}x+\mathbf{B})^{-1}\mathbf{B}),$$ where $\mathbf{A}$ and $\mathbf{B}$ are two $n\times n$ positive-...
  • 11
1 vote
0 answers
7 views

How to estimate the parameters of a Burr XII distribution using MLE

I have a dataset and I am wondering if would be a reasonable fit for a number of distribution types. I was looking to fit the Burr XII distribution in Python initially (using scipy library) and then ...
3 votes
3 answers
187 views

Linear probability model with crossentropy (log) loss

For better or for worse, some people shoehorn binary $y$ variables into an ordinary least squares linear regression. $$ \mathbb E[Y\vert X]=\hat y=X\beta $$ If we encode the $y_i$ as either $0$ or $1$,...
  • 35.8k
3 votes
1 answer
65 views

How do Measure "Robustness" in Statistics?

I am an MBA Student taking courses in Statistics. Our prof was comparing two different methods of estimating the parameters for a regression model: General Method of Moments (GMM) and Maximum ...
1 vote
0 answers
81 views

Are "Moments" More Robust Then "MLE"?

I am an MBA Student taking courses in Statistics. We are learning about different ways to estimate the parameters (i.e. coefficients) of a Regression Model. Our professor indicated that there are two ...
0 votes
0 answers
12 views

Constructing vine copulas from separately estimated bivariate copulas?

I recently came across copulas. I then wondered if you could construct complex multivariate copulas from simpler bivariate copulas... and I discovered vine copulas. One thing that is not clear to me ...
  • 65
1 vote
0 answers
21 views

Can the parameters be identified separately in this model?

Suppose we have a model $y_i=\epsilon_i^1+\epsilon_i^2$ $\epsilon_i^1\sim N(0,\sigma_1^2)$ $\epsilon_i^2\sim N(0,\sigma_2^2)$ $\epsilon_i^1\perp\epsilon_i^2$ Assume $y_i$ is i.i.d. I have derived the ...
2 votes
2 answers
86 views

Relationship Between "Profile Likelihood" and "EM Algorithm"?

I was reading Rao (2017) (Ch3) on profile likelihood. An example is provided which shows how the parameters of a Weibull Distribution can be estimated using the "profile likelihood approach"...
  • 6,078
0 votes
0 answers
33 views

General rule when max likelihood is suboptimal

Inspired by the question regarding Bessel's correction, I wonder whether there is a general rule regarding applicability of maximum likelihood for parameter estimation. My guess is that the parameters ...
  • 6,860
2 votes
1 answer
56 views

Understanding the application of MLE in Naive Bayes

I was looking at the Naive Bayes classifier models (Binomial, Multinomial and Gaussian) and trying to understand the theory behind them a bit better, but am unsure if I understand the MLE approach ...
1 vote
2 answers
249 views

Application of Maximum Likelihood estimation (MLE) to the step of Feasible Generalized Least Square (FGLS)

I have the following regression $$y = X\beta +u$$ where $y$ and $u$ are $(n\times 1)$ and $X$ is a fixed $(n \times k)$ matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of ...
  • 976
0 votes
2 answers
52 views

MLE to address multicollinearity in linear regression

OLS estimation assumes that the explanatory variables are independent in the linear regression model. There isn't such assumption when using the MLE estimation. So, my question is, can we use MLE to ...
  • 11
2 votes
1 answer
38 views

Maximum likelihood for mixture of Bernoullis with known mixture proportions

Given the hierarchical model $$ \begin{align} k & \sim \text{Categorical}(\pi_1, \dots, \pi_K) \\ X \mid k & \sim \text{Bernoulli}(\theta_{k}) \end{align} $$ and an i.i.d. sample $X_1, \dots, ...
  • 3,257
1 vote
0 answers
10 views

Minimisation of KL divergence vs minimisation of empirical processes indexed by a metric space in MLE

I am trying to relate the interpretations of MLE as (1)minimisation of KL-divergence and (2)minimisation of empirical processes indexed by a metric space. Questions: Is it always true that maximum ...
  • 121
1 vote
0 answers
13 views

What is the MLE for regression machine learing models? [duplicate]

From my understanding, in linear regression maximizing the log-likelihood function is equivalent to maximizing the negative MSE. But what about other common regression machine learning models ...
  • 11
1 vote
0 answers
42 views

Is there a general method to estimate the parameter of a Poisson distribution numerically using MLE in R? [duplicate]

I am trying to solve this problem: Assuming for the phishing values to come from a Poisson distribution of unknown lambda parameter, use the sample values to numerically estimate the parameter with ...
1 vote
1 answer
43 views

Maximum likelihood of Normal density under selection

Consider the density function given by $$ \left[\dfrac{\gamma_{\leq0} \mathbb{1}(t \leq 0) + \gamma_{>0} \mathbb{1}(t > 0)}{\gamma_{\leq0}\Phi\left(- \mu / \sigma\right) + \gamma_{>0}\Phi\...
0 votes
0 answers
6 views

training a Bayesian neural network with the weighted log-likelihood in the ELBO

I have a dataset (x,y, $\Delta$y), where $\Delta$y represents the uncertainty over the measures. I am using a bayesian neural network for a regression task over this dataset and I would like to ask if ...
  • 207
3 votes
1 answer
74 views

Is the MLE variance estimator for the normal distribution asymptotically normal?

In textbooks, it is mentioned that the maximum likelihood estimators are asymptotically normal. I am having trouble with understanding how this can be true for the estimator of variance (for example, ...
  • 33
0 votes
0 answers
17 views

Is gradient descent for non-parametric maximum likelihood estimation? [duplicate]

In my reading of maximum likelihood estimation, they go through samples with KNOWN distributions (e.g. binomial, poisson, etc.). I wonder how can I connect to my knowledge of machine learning. In ...
  • 145
0 votes
0 answers
14 views

What sample should be full information maximum likelihood (FIML) be applied to: FIML for longitudinal missing data in an ongoing study?

Context: I have longitudinal (epidemiological) data and plan to run latent growth curve models with four time points (baseline, one-year, two-year, and three-year follow-ups). There is missingness in ...
  • 13
0 votes
0 answers
15 views

Periodic oscillations or discontinuities in likelihood function / loglikelihood function sampling over 2D grid of data

Is it expected behaviour that the loglikelihood function may have periodic oscillations within an envelope, or even periodic discontinuities for data sampled over a 2D grid? For example, generate mock ...
  • 1
0 votes
0 answers
25 views

Relation between likelihood function and conditional probability [duplicate]

I can't quite understand why the LHS equals to the RHS in the highlighted part in the following? A=theta, the model parameter. This is part of the deduction of loss function for logistic regression. ...
  • 145
2 votes
1 answer
42 views

For the logit model, how is the maximum likelihood and minimum distance estimator related?

Suppose I have data $\{y_i,x_i\}_{i=1}^N$, where $x_i\in\{s_1,...,s_K\}$ and follows a discrete uniform distribution. For each realized $x_i$, $y_i$ is generated by the logit model, i.e., $Pr(Y_i=1|...
  • 1,918
0 votes
0 answers
24 views

Summation signs in maximum likelihood (ridge regression example)

I can't find how to deal with summation signs ($\sum$) when performing maximum likelihood estimation. I've encountered it in ridge regression: $\frac{1}{n} \sum_{i=1}^n (y_i-\theta^Tx_i)^2 + \lambda\...
  • 93
1 vote
1 answer
55 views

Random-intercept by ID with no repeated measures: is maximum likelihood estimation dropping cases?

I have two-time point data, where I want to measure "change" in a dependent variable (si_ch_r) predicted by change in three independent variables and ...
1 vote
0 answers
16 views

Rescaling logistic regression coefficients such that variance remains constant

I'm reading "A Modern Maximum-Likelihood Theory for High-dimensional Logistic Regression" by Pragya Sur, and trying to recreate Figure 2, for my own edification. The covariates, $X$, are i.i....
0 votes
0 answers
31 views

Show that the posterior mean can be written as a weighted average of the prior mean and the MLE

From the geometric distribution pmf, $f(x_i |\theta) = (1-\theta)^{x_i -1} \theta; x_i = 1, 2, \cdots$, I have obtained the 1-parameter exponential family as $$exp \left\{\log \frac{\theta}{1-\theta} +...
user avatar
1 vote
0 answers
32 views

Logistic regression using MLE on unlabeled data

I'm coming with a problem in terms of performing MLE on a dataset which is unlabeled. What I'm doing is I have some variables (generated from some distributions) and few coefficients (predefined) ...
0 votes
0 answers
26 views

Minimum Eigenvalue of Hessian matrix is zero

I am estimating a Latent Class Model with a large number of parameters; after the statistical software ends the optimization routine, it displays a message that says: Minimum absolute eigenvalue of ...
  • 239
1 vote
1 answer
48 views

Log Likelihood for a Gaussian process regression model

According to Bishop, the author from "Statistical Pattern Recognition", we can optimize the hyperparameters of a Gaussian process by maximizing the likelihood function $$p(\textbf{t}|\theta),...
  • 73
0 votes
1 answer
33 views

Is the MLE estimate of the mean of Gamma distribution same as the average?

Consider the Gamma distribution: $\Gamma(\alpha,\beta) = \frac{\beta^\alpha x^{\alpha-1} e^{-\beta x}}{\Gamma(\alpha)}$ According to many sources (e.g. wikipedia), the mean is given by $\mu=\alpha/\...
  • 285
0 votes
1 answer
55 views

Why does my Poisson GLM with identity link converge, but a Negative Binomial GLM with identity link NOT converge?

I am trying to fit a GLM to count data, with a single feature (also a count), and intercept term. Both the feature and response are nonnegative (but may be 0). I need to use identity link because I ...
0 votes
0 answers
17 views

Is a conditional logistic regression on a 1:M case-control equivalent to a unconditional model with "centered" data?

I've been studying conditional logistic regression using this book, and it states that for 1 case- M control studies the maximum likelihood estimation of the conditional logistic regression can be ...
  • 437
1 vote
1 answer
58 views

Comparing GAM models with/without fixed effect interactions using REML versus ML

I have several GAM models fit with package mgcv that share the same smooths and random effects groups. I would like to compare support for whether interactions ...
  • 89
0 votes
1 answer
44 views

Can there be any situations where MLE performs better than MPS in terms of MSE or Bias?

Cheng and Amin (1983) proposed the maximum product of spacing estimation method as an alternative to maximum likelihood estimation. They stated that MPS behaves better in small sample cases than MLE ...
  • 93
1 vote
1 answer
36 views

Deriving the expression for $p(\mathcal{K})$ where $\mathcal{K} = \{(\mathbf{s}^k,\mathbf{d}^k), k = 1,..., K\}$

This is a follow up from this question. Consider a model of diseases and symptoms. $s_i\in\{0,1\}$ is a binary random variable indicating whether the patient is showing the $i$-th symptom and $d_j\in ...
  • 71
0 votes
0 answers
7 views

Technical efficiency measurement using Tobit model

I want to measure the technical efficiency of layer poultry farmers producers using the Tobit model. but I collected the data and enter the excel sheet to import it into STATA 13. Now I am confusing ...

1
2 3 4 5
63