Tagged Questions

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

3
votes
0answers
28 views

Explain log likelihood behaviour

(This question is related to a previous one I made, here) I have a set of 2D observations (measured data) of sample size $N$: $$O = \{(x_1, y_1), (x_2, y_2), ..., (x_N, y_N)\}$$ I also have a model ...
1
vote
0answers
14 views

Lemma 3 of the note on the consistencty of the maximum likelihood estimate by A.Wald

Can someone explain to me how Lemma 3 in this article follows from (13) and (14) and the fact that the functions are decreasing? Below is my transcription of the Lemma. Let $X_1, X_2,...$ - i.i.d. ...
1
vote
0answers
35 views

Deriving the maximum likelihood for the parameters in linear regression

Notation: $\textbf{w}$ is an M-dimensional vector of parameters (including the bias parameter), $\textbf{x}_n$ is an M-dimensional vector of the features of each training example, $\textbf{t}$ is an ...
1
vote
0answers
8 views

Joint significance testing w/ large samples

long time listener, first time caller. Unfortunately I can't show code as the computer the analysis is done on has rather tight security. I need to test for joint significance in a logit model that ...
2
votes
1answer
59 views

Maximum likelihood estimator for minimum of exponential distributions

I am stuck on how to solve this problem. So, we have two sequences of random variables, $X_i$ and $Y_i$ for $i=1,...,n$. Now, $X$ and $Y$ are independent exponential distributions with parameters ...
0
votes
1answer
13 views

MLE for two-parameter exponential distribution

I have to find the parameters of a two-parameter exponential distribution using the MLE. But imposing FOC I do not find enough conditions to found both the paramenters. Any hint? Thanks!
2
votes
2answers
104 views

Maximum Likelihood Estimation

If $V_1, V_2,\ldots V_{n_1}$ and $W_1, W_2,\ldots,W_{n_2}$ are independent random samples of size $n_1$ and $n_2$ from normal populations with the means $\mu_1$, $\mu_2$ and the common variance ...
4
votes
3answers
141 views

Inconsistency between R and SAS for MLE on Weibull

I have the following data Y which I want to get an MLE estimate for the parameters using a Weibull distribution in R. 1468, 1872, 475, 1372, 3830, 1849, 978, ...
1
vote
1answer
39 views

Unable to calculate the density function for AR

The model is an AR(p) process excited by a white Gaussian noise $\epsilon_t$, \begin{align} Y_t = &c+ \phi_1Y_{t-1} + \phi_2 Y_{t-2}+ \ldots+ \phi_p Y_{t-p} + \epsilon_t\\ \epsilon_t = ...
1
vote
0answers
43 views

How to Construct a Maximum Likelihood Estimator for Parameter from a Beta Distribution?

I have a question from my Stats Homework that is worded as follows: Based on a random sample size n from the Beta(a,2), construct the MLE of a I really don't ...
0
votes
0answers
12 views

Exponential distribution of interval censored data MLE using optim on R error

I have a set of data which is interval censored and I wish to calculate its maximum likelihood error using optim. This is my code: ...
1
vote
1answer
60 views

Likelihood ratio test for comparing two exponential distributions

I am trying to use a likelihood ratio test to compare the parameters of two exponential distributions. by this thread Likelihood Ratio for two-sample Exponential distribution I found that I can use ...
1
vote
1answer
25 views

Is there any method to quantify parameter estimation uncertainty of method of moments fitting technique?

If I want to fit a distribution (let's say we can be certain about the type) to observations using maximum-likelihood method, I have many options to express the parameter estimation uncertainty due to ...
0
votes
0answers
23 views

Tuning a Maximum Liklehood estimation

This is a follow up question to this question on Cross Validated to which I unfortunately didn't get any replies. The context stayed the same but the method I want to use is completely different so I ...
5
votes
3answers
34 views
5
votes
1answer
154 views

Estimating parameters for a binomial

First of all I'd like to precise that I'm not an expert of the subject. Suppose to have two random variables $X$ and $Y$ that are binomial, respectively $X\sim B(n_1,p)$ and $Y\sim B(n_2,p),$ note ...
0
votes
0answers
27 views

K-means and maximum likelihood!

Is there any relation between k-means and the maximum-likelihood estimate in unsupervised learning? Any references would be appreciates! Thank you!
1
vote
0answers
23 views

Likelihood function for MANOVA

I'm trying to get a handle on MANOVA and Repeated Measures ANOVA. My confusion is around what the likelihood function looks like. Here's how I would formulate the likelihood function Notation: The ...
0
votes
1answer
63 views

Maximum Likelihood Estimator for Exponential Smoothing

I'm not a statistician, so I would love an easy to understand answer. Is there a maximum likelihood estimator that can be stated as an explicit function of the observed data for the models enumerated ...
0
votes
1answer
29 views

Point estimation MLE and MME

Consider the family of probability mass functions given by f(x;k) = 3(4^(k-x)) x = k + 1, k + 2,.... and indexed by parameter k E Z. For a random sample of size n, derive with justification: a) ...
2
votes
1answer
51 views

Conceptual question on estimation : How to calculate the variance of estimation error

EDIT/ UPDATE: I have understood CRLB & why we need it. But my problem is something else. In book ...
0
votes
0answers
33 views

Maximum Likelihood for online decision making

Sorry, this might be a trivial question but I have to ask to be sure. I have a logistic regression model that does a good job of predicting churn. This has been cross validated, etc. Now, imagine ...
2
votes
0answers
60 views

Expectation-Maximization with dependent latent variables

Deriving the equations for a Expectation Maximization over my model, I end up with a posterior for the latent variables (E-step) that prevents me from going on. Generative model I assume my data is ...
3
votes
1answer
27 views

Is the harmonic mean the maximum likelihood estimator for some common continous distribution's parameter?

If $y$ is a vector of continuous data the arithmetic mean is the maximum likelihood estimator for $\mu$ when assuming $y \sim \text{Normal}(\mu,\sigma)$ (not uniquely, of course). The geometric mean ...
0
votes
1answer
19 views

constraint optimization in maximum likelihood estimate

I have a sequence of iids defined by: $f(x|\theta) = exp(-(x-\theta))\;\;\;\; \theta<x<\infty$ To find the maximum likelihood estimate, i should maximize the log likelihood with respect to ...
1
vote
1answer
60 views

How to find a maximum likelihood estimator?

Let $X_1,X_2$ be a random sample of size $n = 2$ from a distribution with density function given by: $$f(x) = 2(x - q) , q \lt x \lt q + 1.$$ a) Show $E[(X - q)^k] = 2/(k+ 2)$ for $k \gt 0$. b) ...
0
votes
0answers
27 views

Marginal Distribution of Sample Variance with Inverse Gamma Prior on $\sigma^2$

Assume the following model with $S^2$ being the sample variance based on $n$ samples. \begin{align*} S^2 & \mid \sigma^2 \sim \sigma^2 \chi_{n-1}^2/(n-1)\\ \sigma^2 & \sim \textrm{inverse ...
0
votes
0answers
22 views

Method of moments estimate of a function of a parameter

Let $X_i$ $\left(i=1,2,3,....n \right)$ be a sample drawn from a Poisson distribution with parameter $\theta$. Find the Method of Moments and MLE estimators for $P\left( X_1+X_2+X_3=0 \right)$.
1
vote
0answers
21 views

Fitting multiple models to a noisy measurement

I have measured a quantity for a set of data contains couple of thousands objects. Since the measurement is very noisy, I need a set of data contains a lot of objects. Then I have a model based on the ...
0
votes
0answers
7 views

Do the marginalised posterior and likelihood function converge in the limit of a large number of observations

Short question Do the likelihood function evaluated at the ML estimate and the marginalised posterior converge in the limit of a large number of observations? Long question I expect the two ...
1
vote
0answers
21 views

MLE == MAP under Uniform Prior? [duplicate]

Does Maximum Likelihood Estimation always yield the same result as Maximum a Posteriori with uniform prior?
2
votes
1answer
68 views

Maximizing likelihood versus MCMC sampling: Comparing Parameters and Deviance

I am working in R. I use lm() for maximizing the likelihood in the first analysis, and STAN to sample from the posterior in a second analysis. ...
0
votes
0answers
14 views

Degrees of freedom when splitting populations

I have a question relating to degrees of freedom when I have a number parameters in a model for a population, and then I want to split the population. Say for a certain population I have a the ...
1
vote
0answers
42 views

“weight” input in glm.nb function in R. How exactly does the weight affect the likelihood?

I would like to understand how the weight argument of glm.nb is affecting the likelihood function. I understand that glm.nb find the MLE in an alternating iteration process where for a given theta the ...
0
votes
1answer
66 views

weighing the maximum likelihood

I would like to define a log-likelihood (starting with a gaussian distribution) for an observed value of a quantity ($x_i$), compared to the measured value based on a given model ($\hat{x_i}$) and ...
5
votes
1answer
67 views

Assuming a probability density for MLE to do model selection

Motivation: I am trying to use Akaike Information Criterion to assess model ranking and over-fitting risk for a set of nonlinear models. I am an electrical engineer with no formal statistical training ...
0
votes
0answers
28 views

jump in the sign of loglikelihood

I am trying to find the maximum of a loglikelihood function in a two dimensional parameter space__ e.g. X,Y positions are the free parameters__ by making grids in the parameter space and compute the ...
0
votes
0answers
28 views

Maximum Liklehood estimator of Poisson

I am trying to solve the following problem, I was able to solve first three bits of the problem but after that i am stuck and do not have clue to solve the (iv) bit. The local government has ...
4
votes
0answers
28 views

asymptotic unbiasedness of weibull mle

It's known that the MLEs of the two-parameter Weibull distribution scale and shape parameters are not available in a closed form. It is, however, known that they do exist, are unique, and moreover, ...
1
vote
0answers
49 views

What is relationship between Fisher Information and Variance in natural exponential Family?

I know that $Var(\hat\theta)\geq 1/I(\theta)$ where $I(\theta)$ is Fisher information. Let take an example of natural exponential family with density $f(x)=\lambda\exp(-\lambda x)$. In this case we ...
0
votes
1answer
35 views

Maximizing Log-Likelihood Estimation for Changepoint Detection

I'm trying to code the changepoint detection algo described here: ...
0
votes
1answer
46 views

Calibrating Generalized Hyperbolic distribution in R - which parameters are valid and allow for a numerical calculation of absolute moments

I am using the R-package ghyp in order to calibrate and model. In fact my coding is based on this paper. I know that I could do quite a robust fit using ...
2
votes
2answers
138 views

Gaussian noise model derivation

I have the following linear regression model, $y = f(x;w) + n$, where $y$ is the vector of true labels, $x$ is the observed data, $f(x;w) = w^Tx$, and $n$ ~ $N(0, \sigma^2)$ is the noise. Why then ...
0
votes
0answers
48 views

Estimating correlation hyperparameters of a Gaussian Process

I have an actual function that I need to simulate using a GP model. I've not done this before so I'm unclear of the steps. I have used the true function at different values of the inputs ($\vec X1, ...
3
votes
0answers
39 views

Fitting Multivariate Bernoulli distribution

I want to fit a model to a number of observations, each of them being a k-dimensional binary vector $(x_1, x_2, ..., x_k)$ where $x_i \in \{0,1\}$. Naturally I would like to fit a multivariate ...
0
votes
0answers
19 views

What is a convex support? (Bickel&Doksum, Mathematical Statistics, Basic ideas…Vol1)

Bickel&Doksum, Mathematical Statistics, Basic ideas...Vol1 page 122, just above Cor2.3.1, it says: Define the convex support of a probability P to be the smallest convex set C such that ...
1
vote
1answer
75 views

Power-law fitting and testing

I want to test the distribution that best fit a specific metric (that I call SD) extracted from the source code of systems. I have a guess that they follow a power-law behavior. My sample: 20 ...
0
votes
2answers
61 views

Maximum Likelihood estimator from sample distribution $N(0,\sigma^2x_i^2)$

Let independent random variable $Y_1,...,Y_n$ have respective distributions $N(0,\sigma^2x_i^2)$, where $i=1,2,...,n$ are known constants such that $x_i\neq 0$ for all $i=1,2,...,n$. Find the maximum ...
2
votes
2answers
70 views

Maximum Likelihood estimator of population variance and its derivation process

I have 2 questions about maximum likelihood and using it to calculate variance: Question #1: The question is about finding the derivative of the score function with respect to the parameter $ ...
0
votes
0answers
103 views

How to derive OLS through MLE? [duplicate]

I am just curious on finding about this derivation