All Questions
774 questions
1
vote
0
answers
39
views
Statistical assessment of block size for bootstrapped distribution fitting
I have a set of intensities from unordered independent events (with no date or timestamps), many of which constitute extremes, and I want to generate an extreme value distribution. The only ...
- 111
2
votes
0
answers
84
views
Calculating confidence Interval for a return time curve, via non-parametric bootstrapping
I have some precipitation data (yearly extremes), which I have fit with a Gumbel distribution (CDF), from which I have calculated a return time distribution. I want to calculate the 95% confidence ...
- 21
32
votes
3
answers
17k
views
Extreme Value Theory - Show: Normal to Gumbel
The Maximum of $X_1,\dots,X_n. \sim$ i.i.d. Standardnormals converges to the Standard Gumbel Distribution according to Extreme Value Theory.
How can we show that?
We have
$$P(\max X_i \leq x) = P(...
- 1,271
6
votes
2
answers
433
views
Do you need large amounts of data to estimate parameters in extreme value distributions?
There is probably not a hard answer for this, but I am wondering if you need to collect more data when trying to estimate the parameters of generalized pareto distribution well?
The reason I ask is ...
- 99
4
votes
1
answer
2k
views
Live peak / trough detection (data provided)
At the bottom of this question is the data of three time series in CSV-format. All are of same length and they all contain measurements of the same event "A". But each time series is using a ...
- 91
1
vote
0
answers
144
views
Resource recommendation for extreme value theory
I'm look to learn about extreme value theory, starting from univariate case and then moving onto the multivariate case.
I have tried the textbook by de Haan, but I'm constantly lost trying to read the ...
8
votes
2
answers
763
views
Intuition behind Weibull distribution?
I don't understand the physical meaning of Weibull distribution's $k$ parameter. Here is a simplified formula of cumulative probability function of Weibull in the simplest form:
$$p(\xi \geq x) = e^{-(...
- 378
3
votes
0
answers
113
views
What likelihood to use to model sample means from a Pareto-like distribution?
Suppose there is a random variable with Lomax (Pareto Type II) probability density
$$
P(x; c) = \frac{c}{(1 + x )^{c + 1}}, \quad x \ge 0, c > 0.
$$
Let's draw n_samples=30000 samples of length ...
- 31
0
votes
0
answers
41
views
Identifiability of a bivariate normal distribution with identified minimum
I am suffering from to understand a proof of a paper.
(Nádas, Arthur. "The distribution of the identified minimum of a normal pair determines' the distribution of the pair." Technometrics 13....
- 441
1
vote
1
answer
219
views
How do I use MLE for non-iid actual data?
In this paper, the author try to fit the Gumbel distribution based on the r largest value of each year using the maximal likelihood estimators: the likelihood function for r largest values $X_{n1},\...
- 747
1
vote
1
answer
268
views
Why my fitted genextreme distribution have no mean/variance?
I have the following code for estimating a generalized extreme value distribution from scipy.
...
- 7,351
4
votes
3
answers
1k
views
Balkema-de Haan-Pickands theorem, generalized Pareto and lognormal
On the wikipedia page on the Balkema-de Haan-Pickands theorem, en.wikipedia.org/wiki/Pickands-Balkema-de_Haan_theorem, it is said the "for a large class of underlying distribution functions",...
- 533
0
votes
1
answer
198
views
Fitting Gumbel distribution based the maximal observation
Assume that we only consider $$G(x)=\exp(-\exp(\frac{x-\mu}{\sigma}))$$ is the Gumbel distribution.
Question: Suppose we have a set of maximum values $\{Y_i\}_{i=1}^m$, why can the article directly (...
- 747
2
votes
1
answer
50
views
Probability of sample minimum below a certain value
I have a list of 1000 songs with their bpm (beats per minute). If I were to sample 30 songs, is there a way to find the probability that the sample minimum is below a certain value like 100 bpm?
- 23
0
votes
0
answers
77
views
Linear Combination of Bounded Pareto RVs
I am working with bounded Pareto distributions and was wondering whether I can say anything about the distributions of linear combinations of Pareto RVs? Suppose the PDF
$f(x; \alpha_i, L_i, H_i) = \...
- 65
15
votes
2
answers
21k
views
What is the distribution for the maximum (minimum) of two independent normal random variables?
Specifically, suppose $X$ and $Y$ are normal random variables (independent but not necessarily identically distributed). Given any particular $a$, is there a nice formula for $P(\max(X,Y)\leq x)$ or ...
- 355
1
vote
3
answers
557
views
How to robustly present a min and a max value?
I have a set of measurements from an air polution sensor. I want to determine the min and the max value in a period of time (let's say in a day).
The min and the max don't have to be the true ...
- 11
30
votes
7
answers
21k
views
How to calculate Zipf's law coefficient from a set of top frequencies?
I have several query frequencies, and I need to estimate the coefficient of Zipf's law. These are the top frequencies:
...
- 319
24
votes
6
answers
48k
views
Why doesn't k-means give the global minimum?
I read that the k-means algorithm only converges to a local minimum and not to a global minimum. Why is this? I can logically think of how initialization could affect the final clustering and there is ...
- 589
6
votes
2
answers
4k
views
Programming inverse-transformation sampling for Pareto distribution
I am having trouble deriving a formula, and running a simulation with its distribution. The Pareto distribution has CDF:
$$F(x) = 1 - \bigg( \frac{k}{x} \bigg)^\gamma
\quad \quad \quad \text{for } x \...
- 203
1
vote
1
answer
2k
views
Return level plots for GEV-distribution
I was reading An Introduction to Statistical Modeling of Extreme Values by Stuart Coles, and I ran into a problem whilst trying to replicate a basic return level graph in R. For context, I first ...
- 79
0
votes
0
answers
25
views
Find extreme values in relative frequencies
I have the relative frequencies of elements in roughly 450 samples (with varying sample sizes). These elements are organisms in fecal samples.
I am interested in finding extreme values of these ...
3
votes
1
answer
1k
views
Method of moments and MLE estimates for Lomax (Pareto Type 2)
I have this dataset, on which I am supposed to fit Lomax distribution with MM and MLE. Lomax pdf is:
$$f(x|\alpha, \lambda) = \frac{\alpha\lambda^\alpha}{\left(\lambda+x\right)^{\alpha+1}}$$
For MM, ...
- 151
1
vote
0
answers
49
views
How to calculate Gumbel with LMoments and GEV with method of moments
I need to calculate the values for certain return periods of a flood event (up to 5000). It has to be GEV with method of moments and Gumbel with L-Moments. But I am not sure about how to calculate ...
- 11
0
votes
0
answers
75
views
What can be concluded when standard deviation plus mean exceeds largest value?
The sum of the mean and standard deviation of a non-normal distribution can exceed the value of the largest sample. For a good explanation of why, see Can mean plus one standard deviation exceed ...
- 101
1
vote
0
answers
51
views
Distribution of the difference between the maximum of $n$ identical and correlated Gaussian random variables and any one of them
Suppose, there are $n$ identical and correlated Gaussian random variables namely, $X_1, X_2, ..., X_n$ with $X_i\sim\mathcal{N}(0,\sigma^2)$ for all $i\in\{1,2, ...n\}$. The correlation coefficient ...
5
votes
1
answer
446
views
Deriving the limiting distribution of a sum of Pareto distributed variables
For a series of independent and identical Pareto distributed variables $X_i$ with $\alpha > 2$, their sum $S_n = \sum_{i=1}^{n} X_i$ has a normal distribution as limiting distribution for $n\to \...
- 86.6k
2
votes
0
answers
177
views
Limit distribution of the joint distribution of maximum and minimum of a sequence of random variables
Assume we have a sequence $\mathsf{X}_1,\mathsf{X}_2,\mathsf{X}_3,...$ of iid random variables. Then the Fisher-Tippet-Gnedenko theorem shows that
$$ \mathbb{P}\left(\frac{\max\{\mathsf{X}_1,\mathsf{X}...
- 167
2
votes
2
answers
2k
views
The relationship between GEV and GPD
source: Embrechts pg 165, 354
(3.52) G is Generalized Pareto Distribution
Base on that theorem, could I conclude that
1) if I have a data and the excess fit with Generalized Pareto Distribution, then ...
- 29
1
vote
1
answer
46
views
Estimating parameters of a Pareto-like distribution and examining its goodness-of-fit
I have developed a theoretical distribution in the form of
$$
f(x) = \frac{\beta}{\alpha}\left(1+\frac{x}{\alpha}\right)^{-\beta - 1}
$$
Where $\alpha$ and $\beta$ are parameters of the model with ...
- 65
1
vote
0
answers
49
views
Is the variance of the maximum of a set of variables higher than the variance of the other variables?
Does the maximum of a set of random variables have high variance compared to the other variables in the set? If so, can someone give an intuitive explanation of why?
Some details about the motivation:
...
- 41
11
votes
4
answers
41k
views
How to check if my data fits log normal distribution?
I'd like to check in R if my data fits log-normal or Pareto distributions. How could I do that? Perhaps ks.test could help me do ...
- 585
0
votes
0
answers
34
views
Find maximum of bimodal posterior pdf
can you help find the maximum (analytically) of the following posterior pdf?
$p(\theta|x) = \frac{\alpha}{\sqrt{2\pi}}e^{-\frac{1}{2}(\theta-x)^2} + \frac{1-\alpha}{\sqrt{2\pi}}e^{-\frac{1}{2}(\theta+...
- 1
0
votes
0
answers
142
views
Connection between forms for Generalized Pareto Distribution
On Wikipedia (https://en.wikipedia.org/wiki/Pareto_distribution#Pareto_types_I–IV) one can find the relation between the different types of Pareto Distribution and the Generalized Pareto Distribution (...
- 363
24
votes
5
answers
5k
views
Why use extreme value theory?
I'm coming from Civil Engineering, in which we use Extreme Value Theory, like GEV distribution to predict the value of certain events, like The biggest wind speed, i.e the value that 98.5% of the wind ...
- 1,285
1
vote
1
answer
534
views
Pareto/NBD and New/Existing Customers
I am looking at implementing a Pareto/NBD model to forecast customer lifetime value in a non-contractual business setting. One thing I haven't got my head around yet is whether such a model is equally ...
- 153
6
votes
2
answers
2k
views
Generalized Pareto distribution (GPD)
I would like to understand the functional form of the Generalized Pareto distribution (GPD) presented in Wikipedia. My questions are:
what is the rationale for replacing $z$ with $\frac{x-\mu}{\sigma}...
- 591
6
votes
1
answer
762
views
How to fit newer cohorts using Pareto/NBD or Beta/Geo for CLTV
I am trying to fit the popular Pareto/NBD or Beta/Geometric models for non-contractual, continuous customer data. On top of that I then fit the Gamma/Gamma model for monetary value (using the very ...
- 4,845
1
vote
0
answers
256
views
Multinomial Logistic Regression as a latent variable model
I was reading the wiki entry for multinomial logistic regression https://en.wikipedia.org/wiki/Multinomial_logistic_regression#As_a_latent-variable_model
and it states that we can view the multinomial ...
- 11
4
votes
1
answer
353
views
How to extract the shape parameter of a Fréchet fitted model using the R SPREDA package?
I'm trying to follow this post, which fits a Frechet distribution to some wind measurements as follows:
...
- 26.9k
2
votes
0
answers
2k
views
How to interpret Hill estimate of tail index
I'm seeking a non-technical explanation of how to interpret the Hill estimate of the tail index for fat-tailed data, and, if possible, some explanation of seemingly contradictory results that ...
- 21
8
votes
1
answer
3k
views
Expectation of the maximum of two correlated normal variables
I am curious what the derivation for the expectation of the maximum of two jointly normal random variables $X$ and $Y$ with correlation coefficient $\rho$.
I could start with the following but the ...
- 259
1
vote
0
answers
727
views
Fitting distributions to censored and uncensored data in R
I need to fit lognormal, Pareto, and generalized Pareto distributions to some empirical data that is a combination of censored and uncensored data. I tried using the function ...
- 11
1
vote
0
answers
82
views
Latent variables for spatio-temporal Extreme Value in R [closed]
Latent variables models are often used for spatial extremes modeling
see e.g., Davison, Padoan and
Ribatet. A typical application
use block maxima such as annual maxima of temperature, assumed to ...
- 5,716
4
votes
1
answer
486
views
Student's t as a power law distribution
I'm currently reading about power laws and I have came across an answer stating:
The density function of a Student's t-distribution with $n$ degrees of freedom is:
$$f(x) \sim (1 + x^2 / n)^{-(n+1)/2}...
- 750
2
votes
2
answers
25k
views
Determine density of $\min(X,Y)$ and $\max(X,Y)$ for independently uniform distributed variables
Two independent random variables, $X$ and $Y$, are uniformly distributed on the unit interval $(-1,1)$.
Determine the density for $U=\min(X,Y)$ and for $W=\max(X,Y)$
- 23
0
votes
0
answers
319
views
Choose best binning for binned maximum likelihood fit?
I am trying to find the strength of signal over a background using a continuous variable, whose distributions are known for the expected signal, the expected background, and the observed data, along ...
- 11
1
vote
1
answer
4k
views
How to fit distributions to data in R?
I have 6 sets of Volume(v) & Duration(d) data. I have fitted a quite few distributions to the data such as Weibull, Gamma, Log-Normal, Exponential, GEV, Pareto, Log Logistic, Poisson, and GP. This ...
- 31
0
votes
2
answers
237
views
Sum of squares for a Dirichlet distribution
I have some data that takes the form of vectors $(a_0,...,a_n)$ lying on the simplex $\Sigma a_i = 1$ (all $a_i$'s non-negative). I have noticed that the maximum $\max_i a_i$ is very highly correlated ...
- 3
0
votes
0
answers
128
views
Probability bound of the difference of order statistics for correlated and identical Gaussian random variables
Suppose, there are $n$ identical and correlated Gaussian random variables namely, $X_1, X_2, ..., X_n$ with $X_i\sim\mathcal{N}(0,\sigma^2)$ for all $i\in\{1,2, ...n\}$. The correlation coefficient ...