All Questions
Tagged with density-function maximum-likelihood
36 questions
1
vote
0
answers
32
views
Likelihood function is a product of PDFs [duplicate]
I am learning about the likelihood function given iid random variables $X_i$ and realizations $x_i$: $\mathcal{L}(\theta | x) = \prod_{i=1}^n \mathbb{P}(X_i = x_i)$. One thing I am confused about is ...
- 193
0
votes
1
answer
74
views
Maximum likelihood estimate for mixture with components using both cartesian and polar coordinates
I have a set of points (x,y) that were generated from a mixture of two components: one component uses Cartesian coordinates, and the other polar coordinates.
For example, with probability $\gamma$ I ...
- 15
4
votes
2
answers
223
views
Seemingly contradiction: probability density function and maximum likelihood calculations for continuous random variable
Claim 1: For continuous random variable, $P(X=x)=0$, where $x$ is a particular number.
Claim 2: When we use maximum likelihood estimation, we plug-in mean, standard deviation and data point $x$ into ...
- 365
1
vote
1
answer
321
views
Maximum likelihood estimate for mixture of different distributions
I'd like to estimate the parameters of a mixture model using MLE. The density is:
$$
f(x,y) = \mathcal{N}(x, y; \boldsymbol{\mu}, \Sigma) \cdot \alpha + \mathcal{N}(x; \mu, \sigma^2) \cdot \mathcal{U}...
- 15
3
votes
0
answers
42
views
MLE is undefined for "densityless" distribution like Cantor distribution
I thought of a situation where we are given a random variable $X$ that has a Cantor "rescaled" distribution. That means that for a parameter $p>0$, $X$ has CDF $F_X(x)=C(\frac xp)$. This ...
- 435
3
votes
1
answer
125
views
The probability/cumulative density function for inequality of two random variables
I have two random variables X and Y which came from different inverse gaussian (IG) ...
- 718
0
votes
1
answer
235
views
Find the MLE density function of uniform [-\theta,\theta] [duplicate]
For $X_1,\dots,X_n$, i.i.d $X_n \sim \mathrm{unif}[-\theta,\theta]$, the ML: $\hat\theta_{MLE}=\mathrm{max}\{-X_{(1)},X_{(n)}\}$. Find the density function. Hint: For $x_1,\dots,x_n$ : $\textrm{max}\{-...
1
vote
1
answer
90
views
How does one reduce the Maximum Likelihood rule to a pairwise comparison of the probability densities?
I am studying an article, where the authors have discussed something about "the ML decision rule reducing to a pairwise comparison of the conditional PDFs" because the means and variances ...
- 79
0
votes
1
answer
373
views
Using PDF values for Likelihood [duplicate]
Given that PDF value $𝑓_𝑋(𝑥)$ for a particular $𝑥=𝑥_1$ does not have any probabilistic meaning (by definition $𝑝(𝑥=𝑥_1)=0$). We still see the use of $𝑓_𝑋(𝑥_1)$ as its likelihood.
My ...
- 143
0
votes
0
answers
419
views
Likelihood values from Sigmoid [duplicate]
Repost of Mathemetics StackExchange question.
There are multiple doubts of mine associated around this theme:
In MLE, we try to find the PDF parameters ($\theta$) which maximise the likelihood of the ...
- 143
1
vote
0
answers
60
views
exponential parameter estimtion from the smallest k-th order statistics
Assume $X_1, X_2, X_3,\ldots,X_n$ are i.i.d. samples from Exp($\lambda$). Assume that the integer $k<n$, is it possible to find a an unbiased estimator for $\lambda$ from the k-th smallest ordered ...
1
vote
1
answer
206
views
How to fit a unnormalized parametric distribution with MLE?
I'm somewhat familiar with parametric estimation using MLE in the context of fitting the parameters of a distribution given a sample. Is there a way of generalizing this approach to unnormalized ...
- 73
0
votes
1
answer
97
views
Simple notation question: pdf for mle of uniform?
I have simple notation question related to pdf for mle of uniform $U(0,\theta)$.
Given following pdf $f(\hat{\theta}_{MLE}) = \frac{n \cdot \hat{\theta}_{MLE}^{(n-1)}}{\theta^n} $ , I'm confused ...
- 251
1
vote
1
answer
526
views
Probability Density Function and Maximum Likelihood Estimation for Multinomial Logistic Regression and GMM
I have some confusion about a few very basic concepts and terminology.
Let's assume we have two models for classification, a multinomial logistic regression (MLR) model and a GMM classifier. I'm not ...
- 503
0
votes
0
answers
456
views
How does multiplying densities together work?
This might be a pretty simple question but I am reading about MLE and notice that we multiply densities just like we do if they were probabilities of independent events.
How does this intuitively or ...
- 3,263
14
votes
1
answer
1k
views
Why does MLE make sense, given the probability of an individual sample is 0?
This is kind of an odd thought I had while reviewing some old statistics and for some reason I can't seem to think of the answer.
A continuous PDF tells us the density of observing values in any ...
- 243
11
votes
1
answer
2k
views
Finding MLE and MSE of $\theta$ where $f_X(x\mid\theta)=\theta x^{−2} I_{x\geq\theta}(x)$
Consider i.i.d random variables $X_1$, $X_2$, . . . , $X_n$ having pdf
$$f_X(x\mid\theta) = \begin{cases} \theta x^{−2} & x\geq\theta \\ 0 &
x\lt\theta \end{cases}$$
where $\theta \...
- 1,792
-1
votes
1
answer
4k
views
PDF of the maximum likelihood estimator of a uniform distribution
Suppose $ \{X_1, \dots , X_n \}$ is a random sample from:
$$
f_X(x; \theta) = \frac{1}{\theta} \text{, for } 0 \leq x \leq \theta
$$
The Likelihood function is easy to calculate:
$$ L_Y(\theta; y)...
- 107
12
votes
2
answers
3k
views
Why maximum likelihood estimation use the product of pdfs rather than cdfs
I'm learning logistic regression and got confused when I saw the equation of the textbook. I knew that for a continuous distribution, to calculate the probability, the pdf $f(x)$ is meaningless. ...
- 536
6
votes
1
answer
1k
views
Why do we need density in estimation and cumulative distribution in transformation?
In statistic, when we want to find the estimation of the model parameters, do we only work with the density function?
Why do we need to work with density function instead of Cumulative distribution ...
- 425
4
votes
0
answers
88
views
Automatic fitting of normalization constant as a parameter in noise contrastive estimation
In the paper on Noise Contrastive Estimation, the authors define a parameterized density function $p_m^0\left(x;\alpha\right)$ to estimate the unnormalized PDF of the data, and then further define a ...
- 1,481
4
votes
2
answers
2k
views
How to estimate normalization constant during optimization of complex parameterized PDF using MLE?
If I have some data in $\mathbb{R}^N$ space, with $N$ beiling large and I want to estimate the density function of this data, $\mathbf{P}_{data}(x)$, using the fantastic property of neural networks ...
- 1,481
1
vote
1
answer
2k
views
Negative loss while training Gaussian Mixture Density Networks
In classification problems, the usual negative log-likelihood loss function
$L(\theta)=\sum_{i=1}^N -\log(p(y_i|x_i,\theta))$
is always non-negative, since the $y_i$'s are discrete random variables ...
- 707
0
votes
0
answers
32
views
Estimator pdf = likelihood function? [duplicate]
In the context of a discussion of consistency of an estimator $\hat{\theta}_n$ (where n is the sample size), what is the meaning of its pdf? Is it the same as its likelihood function?
- 101
4
votes
1
answer
2k
views
How to estimate parameters of a distribution with left-truncated and right-censored data?
I have been trying to find the best way of estimating parameters for a known pdf from a data-set that is left-truncated and right-censored.
More precisely, I have lifetimes for a system where there ...
- 101
8
votes
2
answers
922
views
In MLE for continuous rv, why is it ok to evaluate a pdf at a point?
In MLE for continuous case, my course notes define the likelihood function to be:
$$
L(\theta) = L(\theta;y) = \prod_{i=1}^n f(y_i;\theta)
$$
Where $f$ is the joint pdf of $y_i$ given $\theta$.
I ...
- 733
1
vote
1
answer
485
views
Parameter estimation problem: maximum likelihood [duplicate]
Suppose I have some observations $x_{1}, x_{2}, \dots, x_{n}$. I also have a probability density function with one unknown parameter $\theta$. I would like to find such $\theta$, which would give the ...
- 21
4
votes
0
answers
297
views
Maximum likelihood estimation involving both probabilities and probability densities
Note: based on suggestions in the comments, I have rewritten this question. Please refer to the history for the original version.
In general my question regards how to compute likelihoods in mixed ...
- 519
2
votes
1
answer
264
views
Unable to calculate the density function for AR
The model is an AR(p) process excited by a white Gaussian noise $\epsilon_t$,
\begin{align}
Y_t = &c+ \phi_1Y_{t-1} + \phi_2 Y_{t-2}+ \ldots+ \phi_p Y_{t-p} + \epsilon_t\\
\epsilon_t = &\...
- 1,485
6
votes
1
answer
168
views
Assuming a probability density for MLE to do model selection
Motivation: I am trying to use Akaike Information Criterion to assess model ranking and over-fitting risk for a set of nonlinear models. I am an electrical engineer with no formal statistical training ...
- 659
3
votes
1
answer
373
views
maximum-likelihood of a sequence of events described by a Bernoulli distribution
I am having quite some troubles with the following homework:
In a city it's measured for the whole year whether it rained or not.
A distribution $\textrm{Bernoulli}(r_t|\rho)$ characterizes the ...
- 938
3
votes
1
answer
413
views
Scale parameter MLE scheme known but how to find according distribution PDF?
For known location, we can find the scale parameter of a normal distribution by calculating the sum of squared differences to the location, then dividing by n-1 and taking the square root. This is the ...
- 445
2
votes
2
answers
319
views
Generating functional-form PDF from Max Likelihood Estimation
For the purpose of this question, please consider me a stats newbie.
I'm working on a (very fun!) research project which involves estimating a pdf of "personal values" -- i.e. how much a certain ...
- 21
2
votes
0
answers
184
views
Posterior distribution
Suppose $X1,..,X4$ be iid from pdf $f(x|\theta)=\frac{1}{\theta}$ ,for $0<x<\theta$.
The prior distribution is
$\pi(\theta)=\frac{2}{\theta^3}$ , for $\theta>1$
I have to obtain:
a)...
- 851
2
votes
1
answer
1k
views
Weibull MLE: what is the method/algorithm used to perform the optimization?
Regarding fitting a Weibull two parameters probability density function (pdf), with the fitdistr from MASS package, in R:
1 - What is the iteration algorithm/...
- 3,080
3
votes
2
answers
408
views
Does the EM algorithm for mixtures still address the missing data issue?
There is a PDF $p(D| \theta)=p(X,Z| \theta)$ with observed values $X$ but also some missing or incomplete values $Z$ (for eg. resulting from censoring).
The expectation-maximization (EM) algorithm is ...
- 515