Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

0
votes
0answers
5 views

Effect of prior on the posterior in terms of regularization perspective

I have some difficulty understanding how the posterior will be affected if we use a Laplacian prior instead of a Gaussian prior from regularization perspective. The Gaussian form is encoded in $L_2$ ...
0
votes
0answers
6 views

What are level-2 covariance parameters within Iterative Generalised Least Squares Models to estimate multilevel models?

I am referring to Goldstein & Rasbash (1992): Efficient computational procedures for the estimation of parameters in multilevel models based on iterative generalised least squares. Computational ...
2
votes
1answer
26 views

Is it ever convenient to maximize different functions of the likelihood than the logarithm?

We all know that it's often much more convenient to maximize the log-likelihood rather than the likelihood to get a parameter estimate, since it amounts to the same thing by the fact that the ...
0
votes
0answers
10 views

Choose between ratio of estimators or estimator from the ratio of data

I have to estimate a function which is, say, the proportion of people under 20 in each point of a given territory. Let's call that $h(x,y)$ for $(x,y)$ in my territory. To get that, I have, for every ...
0
votes
0answers
15 views

Multi parameters - Metropolis Hastings Algorithm - Concrete example

have a little project for which I have to estimate parameters on a PSF (Point Spread Function = response of the system to a dirac, i.e a star in my case). Thanks to the Metropolis-Hastings algorithm, ...
3
votes
0answers
47 views

Maximum likelihood: Why is the number of non-zero eigenvalues equal to $x^T \hat{\Sigma}^{-1} x$

I've been reading this code (based on this R package) and I found that the number of non-zero eigenvalues of the estimated covariance is roughly equal to $x_i^T \hat{\Sigma}^{-1} x_i$. I want to know ...
1
vote
0answers
42 views

Confusion about the use of the MLE & the posterior in parameter estimation for logistic regression

In classification one usually computes $$ C = \operatorname*{argmax}_k p(C=k\mid X) $$ where $p(C=k\mid X)$ is the posterior distribution. In a simple logistic regression setting with $C \in \{0, 1\}...
0
votes
0answers
4 views

Duration Analysis with Clumping at Infinity?

I am currently trying to build a model to analyze how price setting today affects how long it takes for a customer to return. My first cut was to fit a Weibull regression where the log of the scale ...
0
votes
0answers
14 views

Maximum likelihood estimation of parameters in a DLM [on hold]

I have two time series yt and xt that are linked and the relation can be written in a state-space form as in the attached screenshot. I have no idea on how to write the program on R since no similar ...
1
vote
0answers
16 views

Bias and over-fitting in Maximum Likelihood estimation

In his book, "Pattern recognition and Machine learning", Bishop talks about the influence of the bias and overfitting in the MLE framework. Here is a quote from p.28, just before he has shown that the ...
0
votes
0answers
13 views

Determine Threshold for different detection test

I am looking at a couple of question based on the image below: The question then requires I select all thresholds (vertical lines) which could be used for Maximum Likelihood Testing, Maximum A ...
-1
votes
0answers
23 views

MLE of lamda in exponential distribution [closed]

For an exponential distribution, given data T = 1, Based on the definition of MLE, which values of lamda, lamda = 1 or lamda = 2 is more appropriate to explain the data? If I use pdf of exponential, ...
1
vote
0answers
36 views

Compound poisson distribution

I'm trying to compute a maximum likelihood of compound Poisson gamma distribution in R. The distribution is defined by $ \sum_{j=1}^{N} Y_j $ where $Y_n$ is i.i.d sequence independent $\operatorname{...
2
votes
3answers
78 views

Maximum likelihood estimators and overfitting

In his book, Bishop claims that overfitting is caused by an unfortunate property of the Maximum likelihood estimator. I dont really understand how the MLE relates to overfitting. To me, roughly, ...
0
votes
0answers
23 views

Marginal Log Likelihood of Count Data to Simplify MLE

My question concerns the calculation of a marginal likelihood given some priors for my underlying exponential mixture distribution. Background My data is the (orderd) set of integers $\{N_\ell\}$. ...
0
votes
0answers
14 views

Asymptotic Variance

The following is a sample size of 10 drawn from a Poisson distribution with rate lambda: 0 1 2 1 3 2 1 2 1 2 a. What is the maximum likelihood estimator of lambda? b.Find the asymptotic variance ...
0
votes
1answer
25 views

What Is Meant by “Maximising” Posterior Probability?

My textbook says the following: The optimal coding decision (optimal in the sense of having the smallest probability of being wrong) is to find which value of $\mathbf{s}$ is most probable, ...
4
votes
1answer
231 views

If both prior and likelihood are Gaussian what can we say about the posterior? [on hold]

If X is a random variable that has Gaussian prior and Gaussian likelihood. What can be inferred about the posterior? As posterior is proporional to prior*likelihood which are Gaussians, the posterior ...
1
vote
0answers
47 views

How to derive correlation using regression without empirical proof?

I just finished learning MLE, Regression, Covariance and now in to Correlation.I want to transform logically from Regression to Correlation using Covariance. Regression: A simple regression model ...
0
votes
0answers
25 views

What is the Cox Partial Likelihood the partial likelihood of?

In linear regression, the likelihood works as follows: Suppose that $Y \mid X, \beta \sim \mathcal{N}(\beta^T X, 1)$. Then the likelihood of $\beta$ given a datum $(x, y)$ is $L(\beta; x, y) = \...
3
votes
1answer
43 views

Finding MLE of $p$ where $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$

Let $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$ be independent Bernoulli random variables where $p\in[0,1/3]$. Derive the MLE of $p$. We have that $$L(p\mid \vec{x})=p^{x_1}(1-...
5
votes
0answers
67 views

Finding MLE and MSE of $\theta$ where $f_X(x\mid\theta)=\theta x^{−2} I_{x\geq\theta}(x)$

Consider i.i.d random variables $X_1$, $X_2$, . . . , $X_n$ having pdf $$f_X(x\mid\theta) = \begin{cases} \theta x^{−2} & x\geq\theta \\ 0 & x\lt\theta \end{cases}$$ where $\theta \...
0
votes
0answers
15 views

LRS for poisson process

How to use likelihood ratio statistics test for Poisson process with different intensity function model? Model 1: $ \lambda(t)=\lambda$ Model 2: $ \lambda(t)=\exp( \beta_1 + \beta_2 t)$ I think the ...
1
vote
1answer
34 views

How is to maximize a function $f(x,y)$ for values of $y$?

Maximum Likelihood Method: Likelihood function asks what value of parameter, $\theta$, makes the data set most probable. Let the distribution is $$f(x;p)=\binom{3}{x}p^x(1-p)^{n-x},\quad x=0,1,2,3.$...
0
votes
0answers
29 views

Optimize log probability instead of probability inside an integral

I've the following doubt. Let's say that I want to find a parameter $\rho$ that maximizes the following function (similar to a likelihood function): $$\int \lbrack\prod Poissionpdf(rho,z) * Normalpdf(...
1
vote
0answers
23 views

Finding standard errors of maximum likelihood estimates

Suppose we use Maximum Likelihood estimation to estimate certain parameters in a model. Furthermore, suppose that the log likelihood function can not be solved analytically and thus must be optimised ...
0
votes
0answers
22 views

Esimating tree creation model parameters using MLE

I'm working on a project, trying to make sense of the creation of trees (representing Reddit posts, for that matter). The tree creation process can be seen as a sequential process, where in each time ...
0
votes
0answers
37 views

Can we unify generalized linear models and ordinary least squares by switching between two metric spaces

Lots of smart people out there. Maybe someone has seen this concept. In linear regression using ordinary least squares (OLS) we simply project the response Y onto the range of the design matrix X. ...
0
votes
0answers
12 views

Using MLE to estimate parameters of jump-diffusion model

Given a dataset, I want to estimate the parameters of a jump diffusion model using MLE. The model is as follows $$X_t = \mu t+\sigma B_t +\sum_{i=1}^{N_t}Y_i$$ Here $B_t$ is a one-dimensional Brownian ...
1
vote
1answer
44 views

Maximum likelihood estimator for a mixture of 2 distributions

Let $X_1, ..., X_n$ be iid with one of two PDFs. If $\theta = 0$, then $f(x; \theta) = 1, \ 0 < x < 1$. if $\theta = 1$, then $f(x; \theta) = \frac{1}{2\sqrt{x}}, \ 0 < x < 1$. What ...
1
vote
1answer
43 views

Confidence interval for the 95th percentile of the normal distribution

Let $X_1, .., X_n \sim Normal(\mu, \sigma^2)$. Let $\tau$ be the 95th percentile of this distribution. Thus, $P(X_i < \tau) = 0.95$. What is the $1 - \alpha$ confidence interval for $\tau$? I ...
1
vote
0answers
29 views

Estimate unknown sum of iid random variables

Let $X_1, X_2, \dots$ be a sequence of independent and identically distributed discrete random variables with common mass function $f_X(x)$ defined for when $x \in \{0,1,\dots,N\}$ and $N$ is known. ...
2
votes
1answer
28 views

Why is the quadratic approximation to the relative likelihood positive?

We can approximate the log likelihood at the real parameter value $l(\theta)$ with the MLE estimate $l(\hat\theta)$ using second order Taylor polynomials, like so: $$l(\theta) - l(\hat\theta) \approx ...
0
votes
1answer
42 views

Differences between exponential distribution with very different n sizes

I need to test for the differences between three groups of observations (grouped along $x$-axis), which appear to follow an exponential distribution along the $y$-axis dimension (see example fig.). ...
0
votes
0answers
16 views

Calculating log-likelihood or BIC of KNN and random forest/boosting models

I am trying to find the BIC of the KNN, random forest and boosting models (for regression, not classification) to use in a combined model that uses Bayesian model averaging to predict a target ...
4
votes
2answers
294 views

Estimating the MLE where the parameter is also the constraint

Independent random variables $X_1,X_2,\ldots,X_n \sim f_X$ are modeled with a common density $$f_X(x) = \frac{\alpha(x/\beta)^{\alpha-1}}{\beta} \quad \quad \quad \text{for all } 0 \le x \le \beta.$$ ...
0
votes
0answers
12 views

Can negative of empirical second derivative of the log likelihood with respect to the parameters not be semi-positive definite?

This is the empirical Fischer Information. Also consider the outer product with itself of the first derivative of the log likelihood with respect to the parameters. This will always be semi-negative ...
1
vote
0answers
24 views

Classification Using The Hidden Markov Model

I am having difficulty understanding certain concepts regarding the classification using HMM. There are numerous post here and in the internet about that, but they never get to detail or they all ...
0
votes
0answers
9 views

Maximum likelihood with order statistics in R [duplicate]

I'm testing the Maximum Likelihood method, when only the maximum value of a sample is provided. I'm assuming the sample is from a Gaussian Distribution. First I generate 10.000 random number with ...
0
votes
1answer
69 views

Maximum likelihood estimator for Bernoulli parameter based on standard normal

$X_i \sim Normal(\psi,1), \ \ i = 1, ..., n$ $Y_i = 1$ if $X_i \ge 0.$ $Y_i = 0$ if $X_i < 0.$ Let $\theta = P(Y_i = 1)$. What is the MLE of $\theta$? I know how to find the MLE of a Bernoulli ...
0
votes
0answers
17 views

How to prove that the solution obtained from MAP estimate is unique?

The data-generating function is in the form y = w.x + ε where ε is simply Gaussian additive noise model. I have derived a MAP estimate of the weight vector (w) using posterior/prior distribution. ...
1
vote
0answers
44 views

Using properties of score function to show unbias in function and compute variance

I would like to compute the fisher indicator for this function: $$f(y;u)=\frac{1}{\sqrt{2\pi y^3}} e^{\frac{-(y-u)^2}{2u^2y}}$$ With $y>0$. I have computed the log likelihood function and the score ...
0
votes
2answers
42 views

Maximum likelihood estimator based on 1 datum for non-canonical discrete distribution

One observation is taken from a discrete distribution with a parameter $\theta$. There are 3 possible values of $\theta$: 1, 2, and 3. The PMF is given below. What is the MLE of $\theta$? I ...
1
vote
0answers
21 views

Name of estimates in the Cox model?

The estimates of the regression parameters in the Cox model are based on the partial likelihood : https://en.wikipedia.org/wiki/Proportional_hazards_model Thus, strictly speaking, they are not ...
0
votes
0answers
25 views

MLE with and without censoring asymptotically the same?

Scenario I. Suppose that $X_i \sim F(\mid \theta)$ are idd random variables, $i=1,...$ and $\theta\subset\Theta$ are the parameters of the model. Let $\hat{\theta}_n$ be the MLE of $\theta$. ...
0
votes
0answers
23 views

MLE for Mixture Model [duplicate]

$$P(W) = \sum_{k=1}^K \pi_k P(W|\mu_k)$$ $$P(W|\mu_k) = \prod_{i=1}^M(\mu_k(i))^{W(i)}$$ Here, it is a mixture model and $W$ is a one-hot encoded vector over a dictionary of size $M$ ie $\sum_{i=1}^{...
0
votes
1answer
29 views

D-dimensional Gaussian posterior distribution

A $D$-dimensional Gaussian random variable $𝑥$ with distribution $N(x| μ, Σ)$ in which the covariance $Σ$ is known and for which we wish to infer the mean $μ$ from a set of observations $X = {𝑥_1, . ...
0
votes
0answers
28 views

Finding MSE when parameter space is restricted.

Let $X_1,.. X_n$~ Exp ($\lambda$) , where $\lambda \ge 2$ . I need to find the Mean squared error (MSE) of the maximum likelihood estimator (MLE) of $\lambda$. The MLE of this is $ \hat{\lambda_{...
5
votes
3answers
93 views

Likelihood function when $X\sim U(0,\theta)$

Let $X_1, ..., X_n$ be $i.i.d$ random variables, uniformly distributed over $(0,\theta)$. Derive the likelihood function given the sample $x_1, ..., x_n$. Answer The likelihood function is: \begin{...
5
votes
3answers
188 views

Consistent unbiased estimator for the location parameter of Cauchy (theta, 1)

Given Cauchy distribution with pdf $p(x) = \frac{1}{\pi ((x - \theta)^2 + 1)}$ how can I find a consistent unbiased estimator for $\theta$? My reasoning so far Tried MLE, but there seems to be no ...