Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
2,437
questions
1
vote
1answer
29 views
If $\epsilon_i \sim \mathcal{N}(0, \sigma^2)$, why does this also imply $x_i|\beta \sim \mathcal{N}(0, \sigma^2)$
I have seen this stated in multiple sources, where if the errors in a linear model ($y_i = \beta x_i + \epsilon_i$) follow $\epsilon_i \sim \mathcal{N}(0, \sigma^2)$, then $x_i|\beta \sim \mathcal{N}(...
2
votes
2answers
34 views
What estimation method establishes sample mean and sample variance as estimators of mean and variance?
Sample mean and sample variance can be derived as MLE estimators for the mean and variance of a normal distribution.
For a distribution in general, what kind of estimation method leads to sample ...
2
votes
1answer
42 views
How to estimate maximum liklihood of a custom log likelihood function?
I am not very familiar with maximum likelihood estimation.
But I would like to test the null hypothesis $\mu = 0, \sigma = 1, \rho = 0$ by estimating the following model: $$z_t - \mu = \rho(z_{t-1} ...
0
votes
0answers
29 views
How to fit a probability distribution in R or Python using maximum likelihood? [closed]
This is different from fitting to sample data. Here, I have a vector X = (x1,x2,x3,.....xN) of size N and associated probability vector P = (p1,p2,p3,...pN). I need to find the best distribution that ...
0
votes
0answers
9 views
Methods to optimize hyperparameters of an ARD kernel for Gaussian Process Regression
I am using Gaussian Process regression to build a model from my feature set, which consists of 40 parameters and ~250 samples in my training set. I've chosen an RBF kernel with ARD (different length ...
1
vote
1answer
24 views
Exhaustive list of techniques used to estimate population mean and variance?
In beginning stats, we were told that:
$\bar{x}$ is an unbiased estimate of $\mu$
$\frac{1}{n - 1}\sum(x - \bar{x})^2$ is an unbiased estimate of $\sigma^2$
As I am reading more, I have learned that ...
1
vote
0answers
19 views
Does Maximum Likelihood Estimation solve n < p problem in regression?
If we use Maximum Likelihood Estimation to estimate regression parameters (B and sigma), and if we have less observations (n) than predictors (p), can we bypass dimension reduction ? My understanding ...
4
votes
2answers
41 views
Any theoretical basis for estimating parameter using $P(\theta | D)$ instead of MLE?
To my understanding, when trying to estimate the value of a parameter $\theta$ of a model (e.g. $mu$ of a Normal distribution) given some data $D$ , we can find the MLE which is $\hat{\theta} = ...
0
votes
0answers
9 views
+50
Hypotheses for likelihood-ratio confidence intervals
I'm working on a physics research project, and in the process I've managed to both firm up my understanding of statistical significance (I hope) and confuse myself somewhat. The question I'm looking ...
-1
votes
0answers
13 views
Find the maximum likelihood estimate Ī²1,e in R from a linear model [closed]
Given a data set of heights and weights, we have a linear model. Find the maximum likelihood estimate Ī²1,e.
I have R in version 3.2.1 and I cannot find a way to use mle. Please help me!
I don't even ...
1
vote
0answers
14 views
Hierarchical Poisson regression with a normal group level
I have a group of items. I model each with the Poisson regression (counts), but one of the regression coefficients is modelled on a group level using the normal distribution.
Assuming there are $m$ ...
1
vote
0answers
19 views
Likelihood for dependent data conditioned on explanatory variables
Let $Y_t$ be a $k\times 1$ vector of dependent variables explained by
\begin{equation}
Y_t=\beta'X_{t-1}+\varepsilon_t,\quad t=1,\cdots,T
\end{equation}
where $X_{t-1}$ is a $k\times 1$ vector of ...
0
votes
0answers
12 views
likelihood interval
When we do maximum likelihood estimation of a (let's say scalar) parameter $\theta$, we get a point estimate wherever the likelihood function is maximized. However, if we want an interval estimate, ...
1
vote
2answers
53 views
Do neural networks make assumptions about data and when to use standardization?
I was reading about when to use standardization vs normalization and what I could understand was that standardization should be used when the model in use makes some assumptions about the data (I don'...
0
votes
0answers
15 views
LSE and MLE of regression coefficients when residuals are skewed normal [closed]
How can I calculate LSE and MLE of regression coefficients when residuals are skewed normal. also how to implement the alpha values given in the picture.
Thanks
2
votes
0answers
26 views
Cumulative Effect
Let $X_t$ be a causal AR($p$) model. If we have $n$ observations of this time series and fit an AR($m$) model with $m\gt p$ to the data; that is
$$X_t = \phi_1 X_{t-1} + \cdots + \phi_m X_{t-m} + W_t,...
1
vote
0answers
15 views
Nonstatespace Form ARMA Estimation Procedure
Consider the following observed stochastic process $\{y_0,y_2,..,y_T\}$ governed by the equation,
$$y_t= \phi_1 y_{t-1} +\theta_1 \epsilon_{t-1} +\tilde{ \epsilon}_t$$
Where $\epsilon_t \sim \...
2
votes
0answers
17 views
Strong consistency in quantum estimation problem
I'm reading the paper: Strong consistency and asymptotic efficiency for adaptive quantum estimation problems by Akio Fujiwara.
In this paper, describes the next adaptive scheme of estimation:
"...
2
votes
0answers
17 views
Why does model selection (AIC and LOO) outcomes differ between ML and bayesian approaches
I am interested in understanding whether my continuous data (dput code at bottom for reproducibility) are fit better by a linear model (Gaussian distribution) or a gamma distributed model.
I ...
0
votes
0answers
8 views
difference between naive maximum likelihood implementation and OLS
I am exploring MLE and implemented a very naive version for censored data. In the simplest case (without censoring) there should be no difference to OLS in theory. Because I was lazy, I thought it ...
4
votes
0answers
30 views
Predictive model (binary) doesn't seem to fit my own data
I have tried to create a predictive model based on the probit model (common in my field). The model is given as:
$$\operatorname{Prob} = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{t}\exp\left(-\frac{x^2}{2}...
0
votes
0answers
5 views
Variance estimation window selection by MLE in R
I would like to estimate the optimal rollback window for estimating the covariance matrix of a time series.
The idea is to create a likelihood function that measures how likely a posterior ...
0
votes
1answer
17 views
Different results for model comparison using ANOVA function for REML and ML
I got this interesting results when I use anova test to compare two nested models. I fitted two nested mixed effects models with ...
1
vote
0answers
37 views
What is the dependent variable in an Item Count Technique Regression (List experiment)
I try to understand the logic behind an Item Count Technique Regression (Imai 2011; Blair & Imai 2012). I am especially interested in the ML estimator for the multivariate regression analysis (...
0
votes
0answers
23 views
MLE, standarderrors from Hessian versus OLS standarderrors
I am minimizing a negative log likelihood for OLS using multiple predictors (the X matrix is generated using a D-optimal design).The standard errors should be the square root of the diagional elements ...
0
votes
1answer
17 views
Optimizing PDF using CDF deltas
In the paper introducing PixelCNN++ they optimize the PDF of a logistic function by minimizing the difference between the CDF around the sample points (x Ā± epsilon). I am trying to understand when it ...
1
vote
0answers
23 views
MLR Test Solution clarification/expansion
Suppose that $X$ is a continuous random variable with p.d.f $f(x) =
\theta x^{\theta-1}I\{x ā (0; 1)\}$. Derive the generalized likelihood ratio test for testing $H_0 : \theta = \theta_0$
versus $...
1
vote
1answer
22 views
pdf of a maximum order statistic with a uniform distribution
I have the following problem,
$X_1,X_2,...,X_n$~$U[\theta,2\theta]$ I am tasked with finding the pdf of the mle. First off, I know that $f(x)=\frac{1}{\theta}$ just by definition and after some ...
4
votes
3answers
144 views
Derivation in the MLE calculation [duplicate]
In the context of the likelihood ratio test, I was told to use the following formula (1):
$$
\sum^n_{i=1}(X_i-\mu_0)^2=\sum^n_{i=1}(X_i-\bar{X})^2 + n(\bar{X}-{\mu}_0)^2 \qquad(1)
$$
in order to ...
1
vote
1answer
47 views
Censored likelihood function in R returning -Inf or NaN
I have an original likelihood for the distribution of network sizes, $y$, conditioned on the number of combined networks, $n$, under negative binomial parameters mean $R$ and dispersion $k$:
$L(R,k|...
0
votes
1answer
45 views
MLE for parameter n in Binomial [closed]
How is the MLE for n calculated if p and k are given? I've been stuck on this forever and cannot seem to even find a similar example of calculating MLE of n.
1
vote
0answers
15 views
fitting a normal curve to xy coordinates with MLE2 in R
I am trying to fit a normal curve to a series of x,y coordinates found in an R dataframe. My goal is to find the best-fitting normal curve an record the mean and sd. I am trying to replicate the ...
1
vote
0answers
21 views
How do you know if a likelihood function is increasing or decreasing?
I have the following function $f(x|\alpha,\beta)=\frac{\alpha}{\beta^\alpha} x^{\alpha-1}, 0\le x \le \beta$ and taking its likelihood I have the following result,
$$lik(\alpha,\beta|x)=\prod_{i=1}^n \...
0
votes
0answers
14 views
Likelihood equations returning different values in R
I made mechanistic adjustments to a negative binomial likelihood equation to account for the distribution of whole network sizes, Y generated from branching ...
0
votes
0answers
23 views
All else equal, should an MLE estimation have a lower standard error than OLS?
If I have a model with Y$\in${0,1}, and am estimating y= $\beta$x+$\eta$
My understanding is if I use a probit model say, I am imposing structure on the DGP by assuming Y|x=$\eta$ is distributed ...
1
vote
1answer
41 views
Establishing connection between ERM (Empirical Risk Minimization) and MLE
My question is specifically about section 2.3.5 "Connection to maximum likelihood estimation" from Nielsen (2016) where the connection of Empirical Risk Minimization (ERM) to Maximum likelihood ...
1
vote
0answers
82 views
Hierarchical models: what are the disadvantages of jointly optimizing first- and second-level parameters
Consider a hierarchical model. For each observatory unit $i$, we observe a vector of Data $D_i$, and we define some probability (density or mass) function $P(D_i|\theta_i)$, which is governed by a (...
0
votes
1answer
59 views
Logistic Regression with continuous exogenous variable
So I'm trying to understand how a logistic regression can work with a continuous exogenous variable.
I'm trying out this code with Python and statsmodels
...
1
vote
1answer
31 views
Find the MLE of $\hat{\gamma}$ of $\gamma$ based on $X_1, … , X_n$
new user here self-studying some mathematical statistics.
I have a problem that has been tripping me up for a few days now. The problem is as follows:
For $1 \leq i \leq n$ and letting $X_1, ... , ...
0
votes
1answer
8 views
Two-step maximum likelihood inference
Suppose we have an latent r.v. $Z$ (not observed) and an observed r.v. $X$, where $X$ depends on $Z$ via some conditional distribution $p(x|z)$. Given $x$, we will try to infer $z$.
Standard maximum ...
0
votes
0answers
20 views
MLEs multivariate normal distribution estimation
Iām a beginner in this field, I hope the problem will be clearā¦ .
Under some regularity assumption the MLE estimators of unknown parameters are unbiased and their distributions is a multivariate ...
0
votes
0answers
7 views
Exact logit but for multinomial or conditional logit?
Logit uses MLE, which exhibits bias in small samples.
As a result there is a procedure "Exact Logit" that I believe uses a method different than MLE to account for this.
Is there something similar ...
0
votes
0answers
11 views
Integrating Nuisance Parameters out of Product of Multivariate PDFs
Suppose we wish to model two variables $x$ and $y$, as having an underlying linear relation with added errors. That is, with data $(x,y)_i: i = 1,...,n$, we model:
$$
\begin{pmatrix} x_i \\ y_i \end{...
0
votes
1answer
21 views
Estimating Mean from Sum of Data vs. Estimating from Complete Set
I am playing around with a set of count data $\{k_j\}$, $j=1,...,N$ where all data points are i.i.d. Poisson distributed with (unknown) mean $\mu$. I have tried two ways of estimating $\mu$ from the ...
0
votes
1answer
25 views
Maximum Likelihood Estimation (log-likelihood) Mistake search [closed]
I wanted to ask if my way was right because the end function looks quite complicated:
iid X1, ... , Xn with f(x) = 1/2Ļ * exp(ā|x|/Ļ), xāR, Ļ > 0 with Ļ being our searched parameter.
My try:
L(Ļ)
&...
0
votes
0answers
46 views
Maximum likelihood estimate of Īø
Here's the question im trying to solve.
Correct me if im wrong please. S(360) means the deaths that occured in the time period of 0 to 360 hours. Does S7 means that there will be seven deaths during ...
0
votes
1answer
20 views
MSE Bias Variance tradeoff in estimating the variance of noise for MLE linear regression
I am just grasping the bias variance trade-off as it is explained by the MSE heuristic. We have that if $y = f(x) + \epsilon$ for $\epsilon \sim N(0,\sigma^2)$ we can show that
\begin{aligned}
MSE(y, ...
1
vote
1answer
30 views
Maximum likelihood inference by estimating the parameters of the probability distribution
I'm wondering if the following two formulations of maximum likelihood inference yield the same result.
Let $Z$ be a 0-or-1 latent random variable and $X$ a random variable that depends on $Z$ ...
2
votes
2answers
81 views
How do you perform a two-dimensional grid-search for the MLE in R?
I'm currently working on parameter estimation for a mixture of two normal distributions. I have two parameters I am trying to estimate using the maximum likelihood method: the probability and the ...
3
votes
2answers
56 views
Estimating parameters using a mixture of normal distributions
I'm currently working on a statistics assignment using R. I am having a bit of trouble with questions 1 and 2.
For 1, my code is:
...
