All Questions
712 questions with no upvoted or accepted answers
2
votes
0
answers
72
views
$1-F$ is rapidly varying if and only if there exists $b_n$ such that $\frac{\max X_i}{b_n} \to 1$ in probability
The following is a problem from Extreme Values, Regular Variation and Point Processes by Resnick.
We will say $1-F$ is rapidly varying as $x \to \infty$ if $\lim_{t \to \infty} \frac{1-F(tx)}{1-F(t)} =...
- 656
2
votes
1
answer
140
views
Loss function for estimating the conditional variance by fitting $y_i^2$
I'm trying to detect anomolies in a dataset $i \in \{1,2,...,N\}$ where a random variable $y_i$ is expected to be drawn from a normal distribution with mean $\mu_i=0$ and variance $\sigma_i^2 (X_i)$ ...
- 187
2
votes
0
answers
55
views
Which likelihood function is correct?
I have a confusion related to the likelihood function. I suppose that users waiting time $W$ follows an Exp distribution with the rate $\lambda$, and the prior of $\lambda$ follows Gamma($\alpha$, $\...
- 57
2
votes
0
answers
169
views
Problem with the Fisher information matrix in case of N measurements of two observables
Let consider two observables, $x$ and $y$. Suppose that $y$ depends on the independent variable $x$ through the model $m(x; \boldsymbol{\theta})$, where $\boldsymbol{\theta}$ is a vector of model ...
- 21
2
votes
0
answers
84
views
Calculating confidence Interval for a return time curve, via non-parametric bootstrapping
I have some precipitation data (yearly extremes), which I have fit with a Gumbel distribution (CDF), from which I have calculated a return time distribution. I want to calculate the 95% confidence ...
- 21
2
votes
0
answers
133
views
Is there any intuitive explanation for MoM in estimating parameters?
I found from some literature that when we use the method of moments to fit the Gumbel distribution, the estimated
(On page 24) A comparison of the variance formulas in (1.66) with the CramBr-Rao ...
- 747
2
votes
1
answer
248
views
Extreme value theory for detrended series
I'm reading "An Introduction to Statistical Modeling of Extreme Values" by Stuart Coles, and using the pyextremes package for exploring the data which is time to return (in days). After ...
- 21
2
votes
0
answers
57
views
Likelihood of a random vector with each component following a different distribution
How do you write down the likelihood for random vectors when each component follows a different distribution with a dependence structure?
For example, Suppose there are n-random vectors, mutually ...
- 131
2
votes
0
answers
65
views
Bayesian inference when distribution depends on unobserved outcome with known distribution
Let's say we have an observed outcome $Y_i$ for an object $i=1,\ldots,I$ that arises like this:
For each object a coin is tossed (outcome $X_i$ = $H$ or $T$).
We know the coin is fair, so $X_i \sim \...
- 35.2k
2
votes
0
answers
125
views
Is it circular reasoning to compute the ELBO using MCMC?
Let's say we have a posterior distribution $q(\theta) = p(\theta \mid D, \mathcal{M})$ over parameters $\theta$ given data $D$ and a model $\mathcal{M}$. As is often the case, computing $q$ is hard, ...
- 21
2
votes
0
answers
54
views
Bayesian inference, likelihood on positive data
Suppose I have a parameter $\theta$, that I know is positive, and some data $(x_1,x_2,\dots,x_n)$ on noisy realisations of the $\theta$. I then assume a prior with positive support on $\theta$ (...
- 321
2
votes
0
answers
40
views
Can log-likelihood test be applied to test two models which are not nested but nested within a full model?
If we have a response variable y and three predictor variables x1, x2, and x3 and M1 and M2 are nested within M3 where
...
- 131
2
votes
0
answers
42
views
Find the likelihood threshold for a Goodness-of-Fit test for multinomial data
Given a sample size $n \in \mathbb{N}$, a null hypothesis $H_0 = \langle p_1, p_2, \dots p_k\rangle$ which is an element of the $k$-dimensional probability simplex, and a significance threshold $\...
2
votes
1
answer
133
views
How to choose between mean squared error and likelihood?
I have a very simple data set with just one real valued feature ($x_i$) and a real valued target ($y_i$).
My model assumes that the targets depend on the feature in a very simple way: for the features ...
- 724
2
votes
0
answers
177
views
Limit distribution of the joint distribution of maximum and minimum of a sequence of random variables
Assume we have a sequence $\mathsf{X}_1,\mathsf{X}_2,\mathsf{X}_3,...$ of iid random variables. Then the Fisher-Tippet-Gnedenko theorem shows that
$$ \mathbb{P}\left(\frac{\max\{\mathsf{X}_1,\mathsf{X}...
- 167
2
votes
0
answers
76
views
Tail-equivalence implying same domain of attraction
Suppose two distributions F and G that have the same extreme point ($x^F = x^G$) and
$$\lim_{x \to x^F}\frac{\bar{F}(x)}{\bar{G}(x)} = c \in (0, \infty)$$
Show that F and G belongs to the same domain ...
- 21
2
votes
0
answers
93
views
Hidden Markov Model observing sequences
I have been trying to understand Hidden Markov Models but I often find myself confused. I have discussed with my tutor for further help however, he is often rude and does not help and so I have ...
- 143
2
votes
2
answers
150
views
Simulations based noisy likelihood function
I have a problem where I have a measured data vector $D$ with Gaussian uncertainties (covariance matrix $\Sigma$). I am now trying to model this data with a generative model with parameters $\phi$. ...
- 837
2
votes
0
answers
31
views
Likelihood calculation w.r.t. uniform discrete distributions
I am working on a little project where I use observations to infer a hidden parameter in Pokemon battling. Without delving into the the mechanics too much, I will attempt to describe the context of ...
2
votes
0
answers
41
views
maximizing log likelihood with missing variables
Suppose I have data $(x_{it},y_t)_{i=1,t=1}^{N,T}$, where $N\rightarrow \infty$ and $T<\infty$. My likelihood function for $i$ will be
$f(x_{it},y_t;\theta_0)$ where $\theta_0$ is the parameter. ...
- 657
2
votes
0
answers
2k
views
How to interpret Hill estimate of tail index
I'm seeking a non-technical explanation of how to interpret the Hill estimate of the tail index for fat-tailed data, and, if possible, some explanation of seemingly contradictory results that ...
- 21
2
votes
0
answers
3k
views
Converting Log-Likelihood to Chi-square
I'm using two different algorithms to get a periodogram. One outputs log-likelihood and the other outputs chi-squared test statistic, but I would like a way to convert from log-likelihood to $\chi^2$ ...
- 21
2
votes
0
answers
150
views
A non statistical/mathematical analogy to max vs argmax
I recently had a discussion on the topic 'usefulness/awareness of the function argmax() in non descriptive analysis'. That means areas, where you do not want to ...
- 1,658
2
votes
0
answers
60
views
Sampling distribution of loss function
So I believe the sampling distribution of the likelihood function is a basic idea in frequentist statistics. For example, the Fisher information $\text{Var}_x(\nabla_\theta \log P(x|\theta))$ which ...
- 399
2
votes
0
answers
119
views
MLE for the sum of independent Bernoulli trials with common factor
Suppose I am computing the sum of different bernoulli trials with probability $p_i = P s_i$, where $P$ is a common factor to all trials and $s_i$ is given, how can I compute the MLE for $P$? I realize ...
- 21
2
votes
0
answers
70
views
ABC Pseudo Marginal
Suppose, that we have observed data denoted as $y_{obs}$, a likelihood function $l(y|\theta)$ where the parameter $\theta$ follows a prior distribution $\pi(\theta)$.
The posterior in the usual ...
- 2,286
2
votes
1
answer
204
views
Optimizing HMM log-likelihood with time-dependent prior
I have a HMM (Hidden Markov Model) which emits an observation Z.
The parameters of the HMM are $\boldsymbol\theta$. $$\boldsymbol\theta = {\boldsymbol{A},\boldsymbol{B},\pi}$$
Where $\boldsymbol{A}$ ...
- 121
2
votes
0
answers
104
views
Multivariate Mixed-Effects Model Likelihood
Say we have a mixed-effects model with a single grouping factor, indexed with $i$:
$$
y_i = X_i\beta + Z_ib_i + \epsilon_i
\\
\epsilon_i \sim \mathcal N(0, \sigma^2)
\\
b_i \sim \mathcal N(0, \Sigma)
$...
- 261
2
votes
0
answers
303
views
Log likelihood, aic and aicc values suggest different models should be selected
I am trying to determine which evolutionary model is best for my discrete data using the function fitDiscrete() from the geiger ...
2
votes
0
answers
71
views
Do most scientific discoveries commit the conditional probability fallacy?
Usually, in an experimental scientific paper where we have observations $X$ taken in laboratory conditions, we either reject some model $\mathcal{M}$ under the basis that $P(X \mid \mathcal{M}) \ll 1$,...
- 3,454
2
votes
0
answers
49
views
MLE for the number of samples given $k$ largest values
I have the views on the top 100 videos using a tag in TikTok and want to estimate the total number of videos in that tag. I know the distribution for other tags so I can make a guess as to what it is ...
- 2,608
2
votes
0
answers
51
views
Maximum Likelihood Estimator for a given density function
I have the following problem: Assume you observe $Y_1,...,Y_N$ independently from the distribution $f_y$:
$$
f_{Y}(y)=\frac{12}{12-\theta}\left\{\begin{array}{ll}-\theta(y-0.5)^{2}+1 & \text { if ...
- 121
2
votes
1
answer
141
views
What are some common prior/likelihood choices for Bayesian logistic regression?
I'm not really clear on the Bayesian approach to logistic regression. From everything I've read, the prior and likelihood can be can be whatever you want them to be. Well, I've a couple things; namely,...
- 3,520
2
votes
0
answers
64
views
Covariance matrix of regularized likelihood
My question is how to estimate the covariance matrix of parameters in a regularized likelihood maximization.
Lets assume we have constructed some negative log-likelihood with a set of parameters and a ...
- 21
2
votes
1
answer
221
views
Can I see Log-likelihood values for two-step clustering in SPSS?
I need to compare Two-step clustering with latent class analysis. As LCA is not possible in SPSS I did it in R, however, 2-step clustering in R is quite challenging, so I did it in SPSS. To compare it ...
2
votes
0
answers
41
views
Likelihood as a test statistic in a hypothesis test
Suppose I have two samples $S_1,S_2$ of categorical data, and I'd like to design a hypothesis test to check the null hypothesis that they are both iid samples from the same underlying distribution. ...
- 6,738
2
votes
1
answer
79
views
Understanding Loss functions in Stacked Capsule Autoencoders
I was reading Stacked Capsule Autoencoder paper published by Geoff Hinton's group last year in NIPS. While reading section 2.1 about constellation autoencoders I couldn't understand how the expression ...
- 121
2
votes
0
answers
12
views
can I compare orignal arima with sqrted arima model?
I use arima to fit my data. I use data and it's root sqrted respectively. Can I compare two model by comparing their AIC, loglikelihood etc.?
Thanks.
- 55
2
votes
1
answer
79
views
Maximum likelihood inference by estimating the parameters of the probability distribution
I'm wondering if the following two formulations of maximum likelihood inference yield the same result.
Let $Z$ be a 0-or-1 latent random variable and $X$ a random variable that depends on $Z$ ...
- 6,738
2
votes
1
answer
113
views
Is the conditional likelihood of a sample of $(X,Y)$ a conditional joint distribution of all $Y$ on all $X$'s?
Given random variables from a random sample $(X_1,Y_1),\dots,(X_n,Y_n)$, the conditional likelihood of observing $y_i |x_i$ (for all $i$) given parameters $\theta_1,\dots,\theta_n$ is usually written ...
- 1,209
2
votes
0
answers
443
views
Fisher Information | Second Derivative of Likelihood Vs Second Derivative of Log Likelihood
I watched this video on Fisher Information and it is mentioned that in Taylor series expansion of the likelihood function the second derivative is parabola which is not a good approximation and a ...
- 900
2
votes
0
answers
67
views
How would radical probabilism/Jeffreys updating/probability kinematics come into play in practice here?
I started looking at the Wiki entry for radical probabilism after I saw a paper from ArXiv this morning. The main idea is that it's an alternative to Bayes' rule for updating probabilities in light of ...
- 21.5k
2
votes
1
answer
202
views
In the context of likelihood, why is the log-density considered to be more "natural" than the density?
Working through some notes and it says that one of the reasons for using the log-likelihood rather than the likelihood is that the "log-likelihood is a the more "natural" and relevant quantity" in ...
- 1,617
2
votes
0
answers
226
views
Choosing between two normal distributions
I have two normal distributions with different means and variances:
N(u1, s1)
N(u2, s2)
And I have some data points (X) that were sampled from each of them. For each data point, I want to calculate ...
- 191
2
votes
0
answers
130
views
On likelihood functions and characteristic functions
Let me preface this by saying that if someone manages to provide a solution to my problem, I will forever be indebted to them, as this problem has driven me crazy. Let us first assume that the ...
- 1,226
2
votes
0
answers
105
views
Can the likelihood ratio estimate multivariate confidence levels?
Wilks' theorem describes the log-ratio between the highest likelihood of a distribution $\mathcal{L}$ (aka the dominant mode, given at $\vec{x}_{m}$) and the likelihood of a distribution at a given ...
- 21
2
votes
1
answer
247
views
Bayesian update vs optimization
Say I have a multivariate normal vector
$$ r \sim N(\mu , \Sigma ) \Rightarrow Pr \sim N(P\mu , P'\Sigma P ) $$
and I observe that
$$ Pr = Q $$
Now I can use Bayes rule to calculate the ...
- 633
2
votes
0
answers
48
views
Exponential Inequality For Probability of Being Close to Maximum
Given $n$ independent identically distributed random variables $X_1, X_2, \ldots, X_n$ that have $|X_i| < \lambda$ for all $i$. Let $\max(X)$ be the maximum of these $n$ variables.
Is there a ...
- 103
2
votes
0
answers
106
views
How to fully estimate a probability density from only a sample of minimum values?
We are given a sample $\{ z_i \}$, $i=1,2,\ldots,N$, such that each value $z_i$ corresponds to the minimum of $n$ random variables $x$, i.e., $z = \min \{ x_1, x_2,\ldots,x_n \}$.
By means of ...
- 1,733
2
votes
0
answers
55
views
Extreme Value Theory - Determining the positive normalising constant in the Extremal Types Theorem
I am working through the following question and cannot seem to work out how the final result is obtained from the last inequality involving $a_n$. Can someone shed some light?
- 309