33
votes
Accepted
Is it possible to understand pareto/nbd model conceptually?
Imagine you're the newly appointed manager of a flower shop. You've got a record of last year's customers – the frequency with which they shop and how long since their last visit. You want to know how ...
- 561
9
votes
Accepted
Programming inverse-transformation sampling for Pareto distribution
I'm assuming you are referecning to Inverse Transform Sampling method. Its very straight forward. Refer Wiki article and this site.
Pareto CDF is given by:
$$
F(X) = 1 -(\frac{k}{x})^\gamma; x\ge k>...
- 8,645
8
votes
Accepted
Do you need large amounts of data to estimate parameters in extreme value distributions?
It's good to have more data , always :) However, consider why we have EVT: to work with less data! Why would you need EVT if you could collect infinite amount of data? You'd simply fit the underlying ...
- 62.3k
7
votes
Accepted
Distribution of $\sum_{j=1}^n\ln\left(\frac{X_{(j)}}{X_{(1)}}\right)$ when $X_i$'s are i.i.d Pareto variables
A simpler approach might be to use the fact that if $x \sim \text{Pareto}(\theta,a)$, then conditioning upon $x \geq b$ results in $x \sim \text{Pareto}(b,a)$. Consequently, $x | x_{(1)} \sim \text{...
- 41k
7
votes
Accepted
Variance of unbiased estimator for the shape parameter of Pareto distribution
I will write the standard Pareto distribution with density
$$
f(x;\alpha,x_m)=\frac{\alpha x_m^\alpha}{x^{\alpha+1}}\cdot I(x > x_m),
$$ for some $\alpha>0, x_m>0$. Then the loglikelihood ...
- 82.8k
7
votes
Accepted
Does statistically simple algos qualify as AI algos?
There's no universally accepted definition of artificial intelligence. Since this directive came from your superiors, it does not matter whether anyone here considers the projects you've proposed to ...
- 350
6
votes
Expected value of a Pareto distribution between two values
Note that $\mathbb P(X \ge \theta)=\int\limits_{\theta}^{\infty} \frac{k\theta^k}{x^{k+1}} \,dx = 1.$
That will not be the case for $\mathbb P(\theta \le a \le X \le b) = \int\limits_{a}^{b} \frac{k\...
- 42k
5
votes
Accepted
Is there a closed-form solution for the tail index of a GB2 distribution?
The "tail index" of a distribution function $F$ describes the rate of decrease of the survival function $1-F(y)$. (Thus, since the $b$ is merely a scale parameter, it cannot possibly influence the ...
- 334k
5
votes
How to determine the estimator of the asymptotic variance of the MLE estimator of the Pareto distribution?
(I am going to fully answer this home study, because it appears that the OP is rather away from any path that could be steered in the right direction by simple hints).
It can be shown that the MLE ...
- 60.7k
5
votes
Accepted
Besides the Pareto and Zipfian distributions, which distributions obey the power-law?
That's basically the complete list in your question, (the Pareto and the zeta/Zipf).
A power law is one where the pdf/pmf is proportional to $x^{-p}\,$ ($1$).
People use power laws for either ...
- 290k
5
votes
Accepted
Method of moments and MLE estimates for Lomax (Pareto Type 2)
The issue appears to be the greatly different scales of the two parameters and how that interacts with BFGS. When I try optim using BFGS on the raw data, I get ...
- 41k
5
votes
Accepted
Student's t as a power law distribution
$$\dfrac{(1 + x^2 / n)^{-(n+1)/2}}{x^{-(n+1)}} = \left(\dfrac{1}{x^2} +\dfrac1n \right)^{-(n+1)/2} \to \left(\dfrac1n \right)^{-(n+1)/2}$$ as $x\to \infty$ while $n$ remains fixed, i.e. the limit of ...
- 42k
4
votes
Fitting a distribution on Income data
I would have thought that what is most suitable must be empirically justified.
I am used to looking at income distributions something like this (from the UK), though strictly speaking this is only ...
- 42k
4
votes
Accepted
What's the relationship between Pareto shape parameter (alpha) and exponential rate parameter (lambda)?
In some respects your question doesn't give enough information to make it possible to address all that you ask, but here I will address the question in your subject line:
What's the relationship ...
- 11k
4
votes
Accepted
Discrete Pareto Distribution vs Zipf Distribution and Power Law vs Zipf Law
[In relation to the relationship between the Zipf and the zeta distributions, the Wikipedia definitions absolutely address your main question. It's possible that you didn't understand what was there.]
...
- 290k
4
votes
Accepted
Obtaining cdf from pdf when pdf is defined on limited region/support
Let's write the calculus formally.
$$f(x) = \begin{cases}
160x^{-6} & x\ge2\\
0 & x< 2
\end{cases}
$$
To get from $f(x)$ to $F(x)$, integrate:
$$F(x) = \int_{-\infty}^{...
- 67k
4
votes
Accepted
Chebyshev's inequality for Pareto distribution (3 sigma rule)
Let $X$ be a random variable with mean $\mu$ and standard deviation $\sigma$. By simple probability rules, we have
$$P\left(\mu-3\sigma \leq X \leq \mu+3\sigma\right) = P(X \leq \mu + 3\sigma) - P(X \...
- 8,757
4
votes
Deriving the limiting distribution of a sum of Pareto distributed variables
Partial results
Below is a trial by comparing the sum of Pareto variables (with $\alpha = 0.5$) with a Levy distribution. The shifting and scaling are done based on the median and interquartile range....
- 86.4k
4
votes
Accepted
Bayes estimator of possion distribution with Pareto prior
I will simplify the analysis by using the parameter $\alpha = 1/\beta$. Letting $\phi = \ln (\theta)$ we have $d \theta /d \phi = \exp(\phi)$ so we can write the prior for $\phi$ as:
$$\begin{align}
\...
- 133k
3
votes
Accepted
Are the Feller-Pareto and the generalized beta distributions really the same?
Summary
Comparing parameterized families of distributions is usually a matter of algebraic manipulation of the most convenient form of expression (usually a density (PDF) or the distribution function (...
- 334k
3
votes
If the best-fitting distribution has infinite variance, should low observed variance be troubling?
It is possible to bypass any issue with the variance and look directly at the plausibility of the Pareto distribution by forming ancillary statistics and then making a QQ-plot of their distribution. ...
- 133k
3
votes
Accepted
If the best-fitting distribution has infinite variance, should low observed variance be troubling?
I guess it depends on what you mean by "best fitting." I would point out two things. First, your sample variance will always be finite, regardless of how much sampling that you do. It is invalid as ...
- 7,810
3
votes
Is there a closed-form solution for the tail index of a GB2 distribution?
This is just an extended comment (and definitely not an answer) adding some details and checks to @whuber 's answer. (Mathematica is used when some code is needed.)
From Qi (2010) the tail index of ...
- 4,505
3
votes
How to calculate Zipf's law coefficient from a set of top frequencies?
The Maximum Likelihood estimates are only point estimates of the parameter $s$. Extra effort is needed to find also the confidence interval of the estimate. The problem is that this interval is not ...
- 2,565
3
votes
How to determine the type of probability distribution for a dataset?
How to determine the type of probability distribution for a dataset?
You can use the fitdistrplus package in R. First, you can plot a Cullen AC and Frey graph ...
- 119
3
votes
Accepted
Fitting Pareto distribution to data example in SciPy
You're correct that fit is defined as the PDF of the Pareto distribution. And as expected, plotting fit by itself (rather than <...
- 146
3
votes
Verifying the statistics are complete and sufficient for two parameter Pareto distribution
Given the density is
$$f(x; a, \theta) := \theta a^{\theta} x^{-(\theta+1)}\boldsymbol 1_{(a,\infty)}(x).\tag 1\label 1$$
The pdf of the first order statistic $X_{(1)}$ can be easily shown to be $$g(...
- 10.3k
2
votes
Is it possible to understand pareto/nbd model conceptually?
To add to Lyuba B.'s wonderful answer, they are using a Poisson process with a gamma mixing distribution and a shifted geometric distribution with a beta mixing distribution because they found that ...
- 21
2
votes
Accepted
How to use method of moment to find Pareto distribution estimator?
If I understand your concern, it's that the true $\alpha$ might be less than 1 (or 2). In this case, the logic behind MoM is that the true mean and the empirical mean should be close. However, the ...
- 1,327
Only top scored, non community-wiki answers of a minimum length are eligible