a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
0
votes
0answers
5 views
Effect of prior on the posterior in terms of regularization perspective
I have some difficulty understanding how the posterior will be affected if we use a Laplacian prior instead of a Gaussian prior from regularization perspective.
The Gaussian form is encoded in $L_2$ ...
0
votes
0answers
6 views
What are level-2 covariance parameters within Iterative Generalised Least Squares Models to estimate multilevel models?
I am referring to Goldstein & Rasbash (1992): Efficient computational procedures for the estimation of parameters in multilevel models based on iterative generalised least squares. Computational ...
2
votes
1answer
26 views
Is it ever convenient to maximize different functions of the likelihood than the logarithm?
We all know that it's often much more convenient to maximize the log-likelihood rather than the likelihood to get a parameter estimate, since it amounts to the same thing by the fact that the ...
0
votes
0answers
10 views
Choose between ratio of estimators or estimator from the ratio of data
I have to estimate a function which is, say, the proportion of people under 20 in each point of a given territory. Let's call that $h(x,y)$ for $(x,y)$ in my territory.
To get that, I have, for every ...
0
votes
0answers
15 views
Multi parameters - Metropolis Hastings Algorithm - Concrete example
have a little project for which I have to estimate parameters on a PSF (Point Spread Function = response of the system to a dirac, i.e a star in my case).
Thanks to the Metropolis-Hastings algorithm, ...
3
votes
0answers
47 views
Maximum likelihood: Why is the number of non-zero eigenvalues equal to $x^T \hat{\Sigma}^{-1} x$
I've been reading this code (based on this R package) and I found that the number of non-zero eigenvalues of the estimated covariance is roughly equal to $x_i^T \hat{\Sigma}^{-1} x_i$. I want to know ...
1
vote
0answers
42 views
Confusion about the use of the MLE & the posterior in parameter estimation for logistic regression
In classification one usually computes
$$
C = \operatorname*{argmax}_k p(C=k\mid X)
$$
where $p(C=k\mid X)$ is the posterior distribution.
In a simple logistic regression setting with $C \in \{0, 1\}...
0
votes
0answers
4 views
Duration Analysis with Clumping at Infinity?
I am currently trying to build a model to analyze how price setting today affects how long it takes for a customer to return. My first cut was to fit a Weibull regression where the log of the scale ...
0
votes
0answers
14 views
Maximum likelihood estimation of parameters in a DLM [on hold]
I have two time series yt and xt that are linked and the relation can be written in a state-space form as in the attached screenshot.
I have no idea on how to write the program on R since no similar ...
1
vote
0answers
16 views
Bias and over-fitting in Maximum Likelihood estimation
In his book, "Pattern recognition and Machine learning", Bishop talks about the influence of the bias and overfitting in the MLE framework. Here is a quote from p.28, just before he has shown that the ...
0
votes
0answers
13 views
Determine Threshold for different detection test
I am looking at a couple of question based on the image below:
The question then requires I select all thresholds (vertical lines) which could be used for Maximum Likelihood Testing, Maximum A ...
-1
votes
0answers
23 views
MLE of lamda in exponential distribution [closed]
For an exponential distribution, given data T = 1,
Based on the definition of MLE, which values of lamda, lamda = 1 or lamda = 2 is more appropriate to explain the data?
If I use pdf of exponential, ...
1
vote
0answers
36 views
Compound poisson distribution
I'm trying to compute a maximum likelihood of compound Poisson gamma distribution in R. The distribution is defined by $ \sum_{j=1}^{N} Y_j $ where $Y_n$ is i.i.d sequence independent $\operatorname{...
2
votes
3answers
78 views
Maximum likelihood estimators and overfitting
In his book, Bishop claims that overfitting is caused by an unfortunate property of the Maximum likelihood estimator. I dont really understand how the MLE relates to overfitting.
To me, roughly, ...
0
votes
0answers
23 views
Marginal Log Likelihood of Count Data to Simplify MLE
My question concerns the calculation of a marginal likelihood given some priors for my underlying exponential mixture distribution.
Background
My data is the (orderd) set of integers $\{N_\ell\}$. ...
0
votes
0answers
14 views
Asymptotic Variance
The following is a sample size of 10 drawn from a Poisson distribution with rate lambda:
0 1 2 1 3 2 1 2 1 2
a. What is the maximum likelihood estimator of lambda?
b.Find the asymptotic variance ...
0
votes
1answer
25 views
What Is Meant by “Maximising” Posterior Probability?
My textbook says the following:
The optimal coding decision (optimal in the sense of having the smallest probability of being wrong) is to find which value of $\mathbf{s}$ is most probable, ...
4
votes
1answer
231 views
If both prior and likelihood are Gaussian what can we say about the posterior? [on hold]
If X is a random variable that has Gaussian prior and Gaussian likelihood. What can be inferred about the posterior?
As posterior is proporional to prior*likelihood which are Gaussians, the posterior ...
1
vote
0answers
47 views
How to derive correlation using regression without empirical proof?
I just finished learning MLE, Regression, Covariance and now in to Correlation.I want to transform logically from Regression to Correlation using Covariance.
Regression:
A simple regression model ...
0
votes
0answers
25 views
What is the Cox Partial Likelihood the partial likelihood of?
In linear regression, the likelihood works as follows: Suppose that $Y \mid X, \beta \sim \mathcal{N}(\beta^T X, 1)$. Then the likelihood of $\beta$ given a datum $(x, y)$ is $L(\beta; x, y) = \...
3
votes
1answer
43 views
Finding MLE of $p$ where $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$
Let $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$
be independent Bernoulli random variables where $p\in[0,1/3]$. Derive
the MLE of $p$.
We have that
$$L(p\mid \vec{x})=p^{x_1}(1-...
5
votes
0answers
67 views
Finding MLE and MSE of $\theta$ where $f_X(x\mid\theta)=\theta x^{−2} I_{x\geq\theta}(x)$
Consider i.i.d random variables $X_1$, $X_2$, . . . , $X_n$ having pdf
$$f_X(x\mid\theta) = \begin{cases} \theta x^{−2} & x\geq\theta \\ 0 &
x\lt\theta \end{cases}$$
where $\theta \...
0
votes
0answers
15 views
LRS for poisson process
How to use likelihood ratio statistics test for Poisson process with different intensity function model?
Model 1: $ \lambda(t)=\lambda$
Model 2: $ \lambda(t)=\exp( \beta_1 + \beta_2 t)$
I think the ...
1
vote
1answer
34 views
How is to maximize a function $f(x,y)$ for values of $y$?
Maximum Likelihood Method:
Likelihood function asks what value of parameter, $\theta$, makes the data set most probable.
Let the distribution is
$$f(x;p)=\binom{3}{x}p^x(1-p)^{n-x},\quad x=0,1,2,3.$...
0
votes
0answers
29 views
Optimize log probability instead of probability inside an integral
I've the following doubt.
Let's say that I want to find a parameter $\rho$ that maximizes the following function (similar to a likelihood function):
$$\int \lbrack\prod Poissionpdf(rho,z) * Normalpdf(...
1
vote
0answers
23 views
Finding standard errors of maximum likelihood estimates
Suppose we use Maximum Likelihood estimation to estimate certain parameters in a model. Furthermore, suppose that the log likelihood function can not be solved analytically and thus must be optimised ...
0
votes
0answers
22 views
Esimating tree creation model parameters using MLE
I'm working on a project, trying to make sense of the creation of trees (representing Reddit posts, for that matter). The tree creation process can be seen as a sequential process, where in each time ...
0
votes
0answers
37 views
Can we unify generalized linear models and ordinary least squares by switching between two metric spaces
Lots of smart people out there. Maybe someone has seen this concept. In linear regression using ordinary least squares (OLS) we simply project the response Y onto the range of the design matrix X. ...
0
votes
0answers
12 views
Using MLE to estimate parameters of jump-diffusion model
Given a dataset, I want to estimate the parameters of a jump diffusion model using MLE.
The model is as follows
$$X_t = \mu t+\sigma B_t +\sum_{i=1}^{N_t}Y_i$$ Here $B_t$ is a one-dimensional Brownian ...
1
vote
1answer
44 views
Maximum likelihood estimator for a mixture of 2 distributions
Let $X_1, ..., X_n$ be iid with one of two PDFs.
If $\theta = 0$, then
$f(x; \theta) = 1, \ 0 < x < 1$.
if $\theta = 1$, then
$f(x; \theta) = \frac{1}{2\sqrt{x}}, \ 0 < x < 1$.
What ...
1
vote
1answer
43 views
Confidence interval for the 95th percentile of the normal distribution
Let $X_1, .., X_n \sim Normal(\mu, \sigma^2)$.
Let $\tau$ be the 95th percentile of this distribution. Thus,
$P(X_i < \tau) = 0.95$.
What is the $1 - \alpha$ confidence interval for $\tau$?
I ...
1
vote
0answers
29 views
Estimate unknown sum of iid random variables
Let $X_1, X_2, \dots$ be a sequence of independent and identically distributed discrete random variables with common mass function $f_X(x)$ defined for when $x \in \{0,1,\dots,N\}$ and $N$ is known.
...
2
votes
1answer
28 views
Why is the quadratic approximation to the relative likelihood positive?
We can approximate the log likelihood at the real parameter value $l(\theta)$ with the MLE estimate $l(\hat\theta)$ using second order Taylor polynomials, like so:
$$l(\theta) - l(\hat\theta) \approx ...
0
votes
1answer
42 views
Differences between exponential distribution with very different n sizes
I need to test for the differences between three groups of observations (grouped along $x$-axis), which appear to follow an exponential distribution along the $y$-axis dimension (see example fig.). ...
0
votes
0answers
16 views
Calculating log-likelihood or BIC of KNN and random forest/boosting models
I am trying to find the BIC of the KNN, random forest and boosting models (for regression, not classification) to use in a combined model that uses Bayesian model averaging to predict a target ...
4
votes
2answers
294 views
Estimating the MLE where the parameter is also the constraint
Independent random variables $X_1,X_2,\ldots,X_n \sim f_X$ are modeled with a common density
$$f_X(x) = \frac{\alpha(x/\beta)^{\alpha-1}}{\beta} \quad \quad \quad \text{for all } 0 \le x \le \beta.$$
...
0
votes
0answers
12 views
Can negative of empirical second derivative of the log likelihood with respect to the parameters not be semi-positive definite?
This is the empirical Fischer Information. Also consider the outer product with itself of the first derivative of the log likelihood with respect to the parameters. This will always be semi-negative ...
1
vote
0answers
24 views
Classification Using The Hidden Markov Model
I am having difficulty understanding certain concepts regarding the classification using HMM. There are numerous post here and in the internet about that, but they never get to detail or they all ...
0
votes
0answers
9 views
Maximum likelihood with order statistics in R [duplicate]
I'm testing the Maximum Likelihood method, when only the maximum value of a sample is provided. I'm assuming the sample is from a Gaussian Distribution.
First I generate 10.000 random number with ...
0
votes
1answer
69 views
Maximum likelihood estimator for Bernoulli parameter based on standard normal
$X_i \sim Normal(\psi,1), \ \ i = 1, ..., n$
$Y_i = 1$ if $X_i \ge 0.$
$Y_i = 0$ if $X_i < 0.$
Let $\theta = P(Y_i = 1)$.
What is the MLE of $\theta$?
I know how to find the MLE of a Bernoulli ...
0
votes
0answers
17 views
How to prove that the solution obtained from MAP estimate is unique?
The data-generating function is in the form y = w.x + ε where ε is simply Gaussian additive noise model.
I have derived a MAP estimate of the weight vector (w) using posterior/prior distribution. ...
1
vote
0answers
44 views
Using properties of score function to show unbias in function and compute variance
I would like to compute the fisher indicator for this function:
$$f(y;u)=\frac{1}{\sqrt{2\pi y^3}} e^{\frac{-(y-u)^2}{2u^2y}}$$
With $y>0$. I have computed the log likelihood function and the score ...
0
votes
2answers
42 views
Maximum likelihood estimator based on 1 datum for non-canonical discrete distribution
One observation is taken from a discrete distribution with a parameter $\theta$. There are 3 possible values of $\theta$: 1, 2, and 3. The PMF is given below.
What is the MLE of $\theta$?
I ...
1
vote
0answers
21 views
Name of estimates in the Cox model?
The estimates of the regression parameters in the Cox model are based on the partial likelihood :
https://en.wikipedia.org/wiki/Proportional_hazards_model
Thus, strictly speaking, they are not ...
0
votes
0answers
25 views
MLE with and without censoring asymptotically the same?
Scenario I. Suppose that $X_i \sim F(\mid \theta)$ are idd random variables, $i=1,...$ and $\theta\subset\Theta$ are the parameters of the model. Let $\hat{\theta}_n$ be the MLE of $\theta$.
...
0
votes
0answers
23 views
MLE for Mixture Model [duplicate]
$$P(W) = \sum_{k=1}^K \pi_k P(W|\mu_k)$$
$$P(W|\mu_k) = \prod_{i=1}^M(\mu_k(i))^{W(i)}$$
Here, it is a mixture model and $W$ is a one-hot encoded vector over a dictionary of size $M$ ie $\sum_{i=1}^{...
0
votes
1answer
29 views
D-dimensional Gaussian posterior distribution
A $D$-dimensional Gaussian random variable $𝑥$ with distribution $N(x| μ, Σ)$ in which the covariance $Σ$ is known and for which we wish to infer the mean $μ$ from a set of observations $X = {𝑥_1, . ...
0
votes
0answers
28 views
Finding MSE when parameter space is restricted.
Let $X_1,.. X_n$~ Exp ($\lambda$) , where $\lambda \ge 2$ . I need to find the Mean squared error (MSE) of the maximum likelihood estimator (MLE) of $\lambda$.
The MLE of this is $ \hat{\lambda_{...
5
votes
3answers
93 views
Likelihood function when $X\sim U(0,\theta)$
Let $X_1, ..., X_n$ be $i.i.d$ random variables, uniformly distributed over $(0,\theta)$. Derive the likelihood function given the sample $x_1, ..., x_n$.
Answer
The likelihood function is:
\begin{...
5
votes
3answers
188 views
Consistent unbiased estimator for the location parameter of Cauchy (theta, 1)
Given Cauchy distribution with pdf $p(x) = \frac{1}{\pi ((x - \theta)^2 + 1)}$
how can I find a consistent unbiased estimator for $\theta$?
My reasoning so far
Tried MLE, but there seems to be no ...
