a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

0
votes
2answers
19 views

Why can multicollinearity be a problem for logistic regression?

Let's say that I want to run a logistic regression on a dataset with n observations and p variables and I have a bad model. I can't understand why running again a logistic regression but this time ...
0
votes
0answers
6 views

Using linear learning to rank algorithm in a mixture language model

How do I tune the parameter alpha automatically?In this formula S(Q|D) = alpha * P(t|D) + (1-alpha) P(T|D) Friend advice me to use linear learning to rank algorithm ( any example?) The formula is ...
1
vote
0answers
18 views

Role of Distribution , likelihood function, and MLE in regression ?

I'm currently a beginner in learning statistics. In regression part, we started to learn (software) using MLE method for estimating parameters in the models. In order to calculate MLE, likelihood ...
0
votes
0answers
7 views

Estimation of scale parameter and shape parameter of using 'maxLik' package [on hold]

I want to test the closeness of my sample data with the Generalized Exponential (GE) distribution. For that, I am using ks test in R. In the documentation of ...
0
votes
1answer
22 views

MLE of Gamma when only given observations [closed]

i've been given 10 observations of X, where X is a random variable. the observations are 141 16 46 40 351 259 317 1511 107 567 and now given they are gamma distributed, how do you find the MLE using ...
0
votes
2answers
37 views

Why it is popular to use stochastic gradient descent in neural networks rather than the BFGS algorithm?

I have made two solvers to implement neural networks, one is based on stochastic gradient descent (SGD) while the other is based on the BFGS (Broyden-Fletcher-Goldfarb-Shanno) algorithm. I have read ...
0
votes
0answers
22 views

How to estimate parameter k in hyperbolic discounting equation using maximum likelihood?

I've been learning and trying to estimate parameters using maximum likelihood, and I'm trying to understand the hyperbolic discounting equation. Here's the equation for hyperbolic discounting: $$y = ...
1
vote
2answers
42 views

Example of learning methods based on bayesian inference

People often say that Bayesian learning and maximum likelihood are two approaches used in machine learning. Main difference is that Bayesian learning tries to include the existing knowledge in the ...
0
votes
0answers
20 views

M-Estimation with Biases

I am currently trying to solve an m-estimation problem with a bias component. To illustrate, I will use a quick example. Let's say you have a laser range finder at position $x$ and a wall at position ...
1
vote
0answers
17 views

Restricted Maximum Likelihood Estimation for Linear Mixed Model

The maximum likelihood estimation procedure for linear mixed model is described in this link. It seems to me that something is wrong there. In their Restricted Maximum Likelihood section the first ...
0
votes
0answers
28 views

How to estimate parameters for each observation using MLE

Given a sample of variables $x_i$, which are each a function of the known variables $y_i,b_i$ and an unknown parameter $\alpha_i$ $$x_i=y_ib_i-y_i^{\alpha_i}$$ I thought I might solve for the ...
0
votes
0answers
22 views

Does the expectation maximization algorithm apply to this problem?

I have a sample of variables $x_i$, where each one is a function of known variables $y_i$ and $b_i$, and of an unknown variable $\alpha_i$. $$x_i=y_ib_i-y_i^{\alpha_i}$$ The density function of ...
0
votes
0answers
7 views

Estimate parameters of transformed AR1 process

I have a process, $W_t$, which follows an AR1 process, where the error $\epsilon_t\sim N(0,\sigma)$: $W_t = \rho W_{t-1} + (1-\rho) \bar{W} + \epsilon_t$ However, I don't have $W_t$, whereas I do ...
2
votes
2answers
139 views

Bias in the MLE of variance component in a multivariate Gaussian?

Given an $n$-vector $y$ (responses) and a design matrix $X$, I wish to fit them with a simple linear regression model $$y=X\beta+e,$$ where $e\sim\mathcal{N}(0, \sigma^2I)$. Then, we have ...
1
vote
0answers
14 views

Which iterative algorithm lmer uses for REML estimation?

For mixed model, when we estimate variance component by restricted maximum likelihood estimation procedure, an iterative algorithm is required to solve the estimating equations for variance component. ...
0
votes
0answers
36 views

Restricted Maximum Likelihood Estimates of Multilevel Regression Model

A two-level regression model : $$Y_{ij} = \gamma_{00} + \gamma_{10}X_{ij} + \gamma_{01}Z_j + \gamma_{11}X_{ij}Z_j + u_{0j} + u_{1j}X_{ij} + e_{ij}$$ where $e_{ij}\sim N(0,\sigma^2_e)$ and , $$ ...
0
votes
0answers
10 views

Truncated/censored AR1 normal likelihood

I have a model for some data that I am analysing which is of the form: $W^*_t = \rho W^*_{t-1} + \epsilon_t$ Where $\epsilon_t\sim N(0,\sigma^2)$. $W^*_t$ is a latent (hidden) process, which ...
3
votes
2answers
71 views

Maximization of a nasty Gaussian likelihood

I asked this question in math.SE before. One answer so far, and we were unable to reach a conclusion. It is more related to statistics, so I wanted to post this here. I have a Gaussian likelihood ...
1
vote
0answers
18 views

Composing a likelihood function for a convolved normal- and lognormal- dataset

I am trying to use some implementation of MLE to solve for the parameters of a mixed normal and lognormal population, and have had no end of trouble. A colleague and the internet has me convinced ...
1
vote
1answer
29 views

Residual deviance difference between multinom() and vglm() function

I have used two functions multinom() in package nnet and vglm() in package (VGAM) to make a multinomial logistic regression. ...
0
votes
1answer
47 views

Interpretation Maximum Likelihood Plot

When using the likelihood method, plotting the relative likelihood or evidence against the range of possible values for the parameter ($\theta$) being estimated results in a curve. The maximum value ...
3
votes
0answers
45 views

Is the likelihood a valid statistic to assess p-value?

The p-value is defined as the probability, under the assumption of hypothesis H, of obtaining a result equal to or more extreme than what was actually observed (Wikipedia). By "a result" it is ...
1
vote
1answer
47 views

Find confidence interval via pivotal quantity?

Suppose $X_1, X_2, ..., X_n$ is a random sample from a population with pdf $$f(x|\theta) = \dfrac{1}{2\theta}e^{-|x|/\theta},x\in \mathbb{R}$$ The pivotal quantity is $\frac{2}{\theta}\sum_{i=1}^n ...
1
vote
1answer
35 views

Why is coxph() so fast for survival analysis on big data?

I frequently do survival analysis on large data sets. One million samples or more is typical, and this seems to be much more than typical research usage. Many algorithms I've used are prohibitively ...
4
votes
1answer
134 views

Maximum Likelihood Estimator for Negative Binomial Distribution

The question is the following: A random sample of n values is collected from a negative binomial distribution with parameter k = 3. Find the maximum likelihood estimator of the parameter ...
1
vote
0answers
12 views

EM algorithm for Blind separation and deconvolution of noisy signals using mixture models

I have a question from the paper: Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models (pdf). In section 2, equations 7, 8, 9 read: \begin{align} ...
2
votes
1answer
139 views

MLE and Bayesian methods

I saw in some lecture the fact that as the number of data points N goes to infinity, the prediction of the Bayesian method goes to the prediction of the MLE. Can someone explain what exactly this ...
3
votes
1answer
60 views

Find the unique MVUE

This question is from Robert Hogg's Introduction to Mathematical Statistics 6th Version problem 7.4.9 at page 388. Let $X_1,...,X_n$ be iid with pdf $f(x;\theta)=1/3\theta,-\theta<x<2\theta,$ ...
0
votes
0answers
57 views

MLE of a multivariate Hawkes process

I'm struggling with implementing the maximum likelihood estimator for a multivariate Hawkes process (HP). Specifically, while the analytical expression for a log-likelihood function of a univariate HP ...
0
votes
0answers
15 views

derive loss function for gamma regression

In the R package mboost there is a family called "GammaReg" which implementes "negative Gamma log-likelihood with logarithmic link function". Still, I don't really ...
0
votes
0answers
25 views

What is the requirement for Kalman filters

I have few conceptual Questions about Kalman filters and their role. When classical estimation techniques like Maximum Likelihood estimation exists which assumes some information such as the states ...
0
votes
0answers
7 views

Finding specific instances between quantitative variables

I have two sets of data that share a common date/time stamp. the first set of data is power and the second is radiation. from both sets i am trying to identify significant instances: minimum power ...
1
vote
2answers
82 views

How to constrain cumulative Gaussian parameters so that the function will intersect one given point?

I am analyzing data from one study where participants had to choose (between two stimuli) the one with higher intensity. One way to look at the data is to fit the proportion of correct choices as a ...
0
votes
1answer
19 views

Detect multicollinearity in maximum likelihood scenarios

I'm estimating a binary logit discrete choice model with BIOGEME and want to check for multicollinearity of my predictors. BIOGEME uses maximum likelihood estimation (MLE) and not ordinary least ...
0
votes
0answers
29 views

Standard Error of estimate of variance

I am not understanding how is the standard error of estimates of two-factor factorial random effect model calculated ? For example , In the book Design and Analysis of Experiments , by Douglas C. ...
0
votes
1answer
36 views

How can I obtain the log-likelihood value in a OLS estimation?

So, in some papers using panel data, I noticed that in the estimate results inherent a pooled OLS regression, they report the value of the log-likelihood. I was wondering how this is possible, in ...
1
vote
1answer
35 views

How to prove the identifiability of a likelihood

Consider the likelihood function for parameter vector ...
0
votes
0answers
19 views

CRF Training: Max-margin vs max-likelihood

I'm trying to use PyStruct's CRF implementation. In its user guide, it says the following: I call these models Conditional Random Fields (CRFs), but this a slight abuse of notation, as PyStruct ...
0
votes
2answers
42 views

How can maximum likelihood be used to estimate parameters for a Weibull distribution? [duplicate]

Consider the following survival data (cumulative): Month 0 - 100% Month 1 - 50% Month 2 - 33% Month 3 - 25% Month 4 - 20% (meaning 20% of all initial units have survived by the end of Month 4) ...
0
votes
0answers
10 views

Instances of this Gaussian variance model with non-convex log-likelihood

On page 4 of Thomas Minka's Beyond Newton's Method, he mentions the following Gaussian variance model with a non-convex maximum likelihood objective. What are some examples/instances of models where ...
-1
votes
1answer
47 views

Maximum likelihood estimation for Cauchy noise

What is the maximum likelihood estimator of the covariance matrix for a given vector in the presence of Cauchy noise? How can we calculate it given that the Cauchy distribution has infinite variance? ...
1
vote
0answers
27 views

Conditional versus Unconditional MLE parameters estimation?

I have read about both conditional and unconditional MLE parameters estimation for ARIMA models but I have problem understanding the concept from the statistical books . Does conditional MLE mean that ...
0
votes
0answers
15 views

why use weighted sum of component densities in unsupervised parametric estimation?

In unsupervised parametric estimation why do we take $p(x|\Theta)$ as $ \sum_j P(w_j)p(x|w_j ,\Theta_j ), j = 1 - c$? That is weighted sum of component densities. where $p(x|\Theta)$ is mixture ...
-1
votes
1answer
46 views
0
votes
0answers
14 views

convergence assumptions for expectation maximization

in andrew ng lectures notes for expectation maximization, i believe the only assumption invoked for the convergence of the EM algorithm is the jensen inequality that operates on the function Log(x), ...
1
vote
0answers
15 views

Mean of sample means from samples of different variances

Suppose I have three samples $x_i \sim \mathcal{N}(\mu, \sigma_b^2)$. I cannot measure the $x_i$ directly, so instead I estimate the value of each one by averaging $n_i$ draws from a distribution ...
0
votes
0answers
29 views

Need for iid in MLE

I am studying about parametric estimation in supervised learning using maximum likelihood estimation. Here is what I learned: Separate our training data according to class; i.e., we have c data sets ...
1
vote
0answers
11 views

Given an transition matrix what is the likelihood an observed markov chain was derived from this matrix

To give a bit of background, I'm creating a MLE of a transition matrix from a set of empirical data. I'm then creating a simulation of the system that also produces a markov chain. I am looking for a ...
0
votes
0answers
19 views

Estimate betabinomial distribution parameters from weighted observations

Suppose we observe $N$ independent Bernoulli sequences of different lengths. Let $k_i$ be the number of trials in each observation, and $o_i$ be the number of "successes": ...
0
votes
1answer
43 views

Variance of the $\hat{\sigma^2}$ of a Maximum Likelihood estimator

Given some normally distributed observations $x_1,x_2,...,x_n$ $\forall i\ x_i\sim\mathcal{N}(\mu, \sigma^2)$ the ML estimator decides that the variance that maximizes the likelihood function is ...