All Questions
4,023 questions
8
votes
1
answer
221
views
How to match my prior beliefs to beta distribution?
I have some data that I believe comes from the binomial distribution. I also have some old data from a past-experiment that I would like to base my prior beliefs on. The old data observations are: $$6,...
- 163
7
votes
4
answers
618
views
What is the Maximum Likelihood Estimator for my nonlinear problem?
In the region $0\leq x\leq 1$ and $0\leq y\leq 1$ I have 1 radio broadcaster placed in $(x_b,y_b)$ and N receivers. The position of the broadcaster is not known.
The position of each receiver however ...
- 601
0
votes
0
answers
34
views
Numerical Optimization of Marginal Likelihood that Explodes
I have a model with a marginal likelihood of the following form:
$$\mathcal{L}(\theta_1, \theta_2, \theta_3|\{x_{i,j}\}_{i=1, j=1}^{N, M_i})=\prod_{i=1}^{N}\int_{0}^{1} f(p_i;\theta_1) \prod_{j=1}^{...
- 248
1
vote
0
answers
54
views
Manually program EM in r to updated multiple parameters and solve missing data [closed]
I am trying to use EM (Expectation-maximization) to fill in missing data in R, but am not sure how to model/code it for my specific case. I am generally trying to follow the example format used in ...
- 65
1
vote
0
answers
31
views
Normalization for time series comparison
I have a time series Markov Switching model, which is estimated in about 15 different versions. One or two of the time series had to be normalized in order to converge. That is 1-2 out of 15. My ...
- 31
1
vote
0
answers
61
views
How to fit using a model, which has two highly correlated parameters? [closed]
I am trying to fit a dataset that depends on one observable, i.e. x. The model has two parameters that are highly correlated, of the form:
$$f(x, \frac{\alpha}{\...
- 11
1
vote
1
answer
95
views
Is this a typo on P.75, Theorem 5.52 of the book "Asymptotic Statistics" by Van der Vaart?
Let $\Theta$ be a compact metric space, $\theta \in \Theta.$ Let $m_{\theta}:\mathbb{R}^d\to \mathbb{R}: x\mapsto m_{\theta}(x)$ be a family of measurable function indexed by $\theta \in \Theta.$ Let $...
- 761
2
votes
1
answer
88
views
Maximum Likelihood Estimation for a Unique Probability Density Function
In the context of estimating parameters for a uniquely distributed set of independent and identically distributed random variables, I am examining the following probability density function $ f(x|\...
- 425
3
votes
0
answers
135
views
How can we justfify the assumption of equal scale/variance in the definition of R-squared from Deviances in GLMs?
If we take the R-squared to be the comparison of Deviances between models (the model of interest, the saturated model, and the constant model), we can write it as (see this answer CC BY-SA 4.0):
$$R_{...
- 19.5k
0
votes
0
answers
97
views
How to do hypothesis testing for Minimum value?
I have a sample with a size of n=100, and I want to show that the minimum value of the underlying distribution is not less than a certain threshold, with a confidence level of 95%.
The distribution of ...
- 101
3
votes
0
answers
120
views
MLE has any competitors?
I am learning about characteristics of statistical estimators such as Efficiency and Minimum Variance. For example , on the topic of MLE, it says, here (1) it says that : "MLE is popular for a ...
0
votes
0
answers
39
views
Maximum likelihood in linear regression
My understanding is that when we do maximum likelihood we want to choose parameters $\theta$ such that the probability of observing the actual, fixed data is maximized. That's how I understood it ...
- 126
2
votes
0
answers
76
views
Is this a correct explanation of the asymptotic bias of maximum likelihood?
I want to be sure I understand, so please critique the following:
In regular parametric statistical models, the non-linear maximum likelihood estimator is biased. Given some data, $y_i$, parameters, $...
- 61
2
votes
1
answer
98
views
How can Null model likelihood be higher than Fitted model likelihood
As far as I know, when fitting a GLM, the fitted model should always have a higher likelihood compared to the null model (with only an intercept) for the same training set. When I run a small ...
- 1,465
1
vote
2
answers
90
views
Finding the temperature value that gives optimal value
I'm trying to analyze some sleep data from kaggle (this example data does not have correct temperature data but the actual data I will use in the future will have precise temperature) to try to find ...
- 11
0
votes
0
answers
49
views
How do continuous partial derivatives depend on $n$ in maximum likelihood estimation?
I'm reading Tensor Methods in Statistics by McCullagh 1987, (P209 for this question) and I can't understand one step he uses.
He begins with the usual log-likelihood
\begin{equation*}
l(\theta; Y) =...
- 61
3
votes
1
answer
82
views
What standard error should I use with correlated clusters in maximum likelihood estimation of multinomial logit
I have a dataset with 14 clusters. Each cluster is a time series of 80 periods with autocorrealtion, and I am doing maximum likelihood estimation of a structural multinomial logit model. I suspect ...
- 357
2
votes
1
answer
104
views
Finding the limiting distribution of $\sqrt{n} (\hat{\tau} - \tau)$ as $n \rightarrow \infty$ for $N(\mu, \mu^2 \tau)$
Let $X_i$ for $i = 1, ..., n$ be a random sample from the distribution $N(\mu, \mu^2 \tau)$ with unknown parameters $\mu \in (\infty, 0) \cup (0 ,\infty), \tau > 0$.
Find and justify the mle $\hat{\...
- 133
0
votes
0
answers
82
views
MLE of multivariate normal distribution when the VCV matrix is full of equations [duplicate]
Short Version: Given a variance covariance matrix for my multivariate normal distribution where the entries are equations of other parameters, how do I find which of those parameter values maximizes ...
- 425
0
votes
0
answers
42
views
Why isn't X treated as a random variable in linear regression MLE? [duplicate]
I am very confused by this because when I watch videos or read about MLE with linear regression it seems to be commonly assumed that $X$ is fixed or that if it is random we don't care for the purposes ...
- 126
0
votes
0
answers
20
views
Bayesian account for maximum likelihood estimate over infinite parameter space
Suppose I have some samples $x_1, \ldots, x_n$ from $\mathcal{N}(\mu, 1)$ for unknown $\mu$. Then the maximum likelihood estimate for $\mu$ is just $\overline x = \frac1n \sum x_i$. Ideally, we can ...
3
votes
3
answers
178
views
What exactly is likelihood? [duplicate]
My understanding about likelihood, given some reading, is that it is how likely we are to observe the actual data given a certain parameter or parameter values $\theta$. Like with the coin toss ...
- 126
2
votes
0
answers
83
views
Asymptotic normality of the maximum likelihood estimator with dependent data
In the setup, assume $\left(\mathbb{R}, \mathscr{B}\left(\mathbb{R}\right), P\right)$ is the underlying probability space and suppose that $\left\{\mathcal{F_n}\right\}_{n\in \mathbb{N}}$ is a ...
- 549
3
votes
2
answers
160
views
Why is the asymptotic bias of the maximum likelihood estimate $b(\theta) = \frac{b_1(\theta)}{n}+\frac{b_2(\theta)}{n^2}+...$?
Firth (1993) states in his introduction that for a $p$-dimensional parameter $\theta$ the asymptotic bias of the maximum likelihood estimate $\hat{\theta}$ may be written as:
$b(\theta) = \frac{b_1(\...
- 61
1
vote
0
answers
46
views
Large samples property of bayes procedures
I was reading through Wasserman's All of Statistics and I came across this property in the Bayesian statistics chapter:
I think I don't really get what is supposed to be the intuition behind it, and ...
- 41
4
votes
4
answers
299
views
Interpretation of Maximum Likelihood Value
I have a question about Maximum Likelihood values, and how to interpret them.
In order to explain the question, please see the Figure below. I will add explanation for how this figure has been created ...
- 333
0
votes
0
answers
31
views
Estimating number of occurrences of binomial tests
I have data representing a counting of the number of successes in a series of $n$-trial binomial experiments, however each experiment might have a different $n$ and is unknown.
So, if I for example ...
- 101
0
votes
0
answers
29
views
How to prove that the MLE of a uniform distribution is biased using the formula given below? [duplicate]
I've calculated the MLE of the uniform distribution on [0,theta] as maxi{Xi} but don't know how to prove it is biased. The formula I have learned to prove it is unbiased is E(θ^)-θ=0.
Was stuck on how ...
- 1
2
votes
1
answer
107
views
How to tell the difference between a Latent Variable and a Non-Latent Variable?
I am struggling to understand how statisticians define a Latent Variable.
Suppose the joint probability associated with a statistical model is:
$$ p(x, Z, \theta)$$
$x$ are the random variables, $Z$ ...
2
votes
1
answer
77
views
Centering Priors on MLEs vs. Using MLEs as Initial Conditions for MCMC [duplicate]
Here:
Centering prior distributions on MLE/OLS estimates
I ask about centering priors on MLEs in the context of a logistic regression (in my case with only categorical predictors), which I've seen a ...
- 1,649
1
vote
0
answers
79
views
Fitting a truncated normal to data
I consider the normal distribution truncated to the half-interval $[0,\infty)$,
$$
P(x) = \sqrt{ \frac{2}{\pi\sigma^{2}} } \frac{1}{\operatorname{erfc}\left( -\frac{\mu}{\sqrt{ 2\sigma^{2} }} \right)} ...
- 4,552
0
votes
0
answers
26
views
Measuring maximum error of choosing MLE vs OLS/Moment [duplicate]
Here is my understanding of these topics:
MLE is better when you know about the type of distribution that generated data (ex: formula for mean of normal distribution is different from mean of ...
0
votes
1
answer
87
views
Why do we hear less about OLS as we move away from beginner class? [closed]
As I understand, in simple linear regression, we have two options:
OLS: OLS estimator is BLUE: Best (Lowest Variance) Linear Unbiased Estimator ... also normally distributed when sample size is large ...
1
vote
1
answer
155
views
When is OLS=MLE and when is it not? [duplicate]
In linear regression, OLS and MLE are equal to each other. Is this just a coincidence or is there a reason? Is there some way of knowing when/why they will be equal to each other and when they will ...
1
vote
0
answers
87
views
Threshold choice for Peaks-Over-Threshold
I'm trying to estimate equivalent performances at different events, using Peaks-Over-Threshold from Extreme Value Theory. The challenge is to find the threshold and preferably with same number of ...
4
votes
1
answer
445
views
Where does Quasi-Likelihood formula come from?
In regular likelihood/log likelihood, if there is random variable "$Y$" with pdf (probability distribution functions) $f_Y(y)$... the likelihood of this can be written as: $\mathcal{L}(y_i) =...
1
vote
1
answer
53
views
ARIMA parameter estimation from scratch
I am implementing ARIMA from scratch, and I am trying to understand how to estimate the parameters by the MLE + innovations algorithm approach. The likelihood is given by (as in Shumway and Stoffer: ...
- 11
3
votes
1
answer
111
views
The "detectseparation" package. How to interpret its results?
I am running a logistic regression on financial data of several companies. The dependent variable is the company to be a smoother (binary variable: 0, 1), and there are several independent variables: ...
- 171
2
votes
1
answer
66
views
Is a Bayesian calculation still Bayesian if you don't explicitly include priors?
A simplification of my problem that I think conveys the essential parts of my question:
I am trying to calculate the most likely values for the mean and the standard deviation of a Gaussian ...
- 1,243
0
votes
0
answers
35
views
Extreme Value Analysis - Nonrandom/Preferential Sampling
I am doing an extreme value analysis (EVA) but there is a nuance in my problem that I believe is not addressed in extreme value theory. I have not been able to find information about this in textbooks ...
1
vote
0
answers
167
views
Survey Package in R - svyglm with quasipoission link function for binary outcomes data, yielding relative risk? [closed]
I'm planning to run a regression analysis to estimate [adjusted] relative risk of a binary outcome data (essentially, analyzing the relative risk or risk ratio of an event happening), using a number ...
- 81
4
votes
1
answer
844
views
Difference between KDE, MLE and EM for density estimation
I'm reviewing kernel density estimation (KDE), maximum likelihood estimation (MLE) and expectation maximization (EM) algorithm for density estimation and struggling to differentiate what each ...
- 143
4
votes
1
answer
126
views
Does the "log-likelihood" measure cover all details about model fit, like covariance structure, adjustments, robust variance estimator, etc?
Just a general statistical question: when any statistical software returns log-likelihood of some model, does it account for all details in it?
For example, when we employ generalized least square ...
- 51
0
votes
0
answers
24
views
Likelihood based CI with L2 regularization
I apologize if my question seems basic, but I'm attempting to derive confidence intervals for certain parameters whose estimates were obtained through nonlinear least squares regression. Unfortunately,...
1
vote
1
answer
72
views
Does increasing number of observations lead to the decreasing of Mean Square Error of consistent estimators?
I know that not all weakly consistent estimators exhibit MSE-consistency : https://stats.stackexchange.com/a/610835/397467.
Anyway, does increasing the sample size leads to a reduction in their mean ...
- 11
-1
votes
2
answers
141
views
why are errors normal in OLS? [duplicate]
In maximum likleihood, we believe that the y-variable is conditionally normally distributed. So this means that errors are also normally distributed.
In ols regression, things seem to be more algebra/...
0
votes
0
answers
40
views
Calculating MLE under restriction on coefficients
Consider the following simple linear regression model:
$y_i = a + b \cdot x_i + \epsilon_i \space\space\space\space\space\space\space where \space i= 1, 2, 3, \cdots , n$
here $\epsilon_i \space's$ ...
- 115
0
votes
0
answers
141
views
MLE of weibull distribution with survival data
I would like to ask about estimating parameters of Weibull distribution (a, and b)
I am trying to code likelihood of weibull distribution with survival data $(T_i, \Delta_i),$ which I believe is: $(ab)...
- 1
0
votes
0
answers
107
views
Expectation Maximization on Multivariate Gaussian Mixture Model for clustering
I have a dataset with 1000 observations and two features that define those N=1000 data points. Hence it is 1000*2 input matrix. I need to cluster them into k clusters.
I am not understanding the E-M ...
- 1
0
votes
0
answers
25
views
Separating components of a likelihood maximization
Apologies for the naive question, but I have a problem I would like to solve.
Suppose I have a two dimensional likelihood of the form
$L \propto \exp\{-\frac{1}{2}\} \begin{bmatrix}x & y\end{...
- 1