All Questions
774 questions
1
vote
1
answer
308
views
Pearson 5/Inverse Gamma/ Double Pareto
My current research deals with landslide size frequency distribution which has been fit to a three-parameter inverse gamma function and a double Pareto function by previous researchers. I am trained ...
- 11
4
votes
1
answer
2k
views
Significance of Difference between Two Extremes (Maxima/Minima)
I am comparing two different populations (with different sizes) of models, for each individual I have a score. I'd like to compare not only the difference of the two means but the maximum value. I've ...
- 1,026
2
votes
1
answer
78
views
Modelling the tail only
I'm trying to model a real-world random variable that behaves approximately as a Gaussian, so a Normal distribution fit is reasonable but far from perfect.
However, I only care about its tail, that ...
- 451
1
vote
2
answers
359
views
Understanding Support Vector Machine Algorithm
I understand the SVM classifying algorithm in my book given the assumptions. My book says, suppose we have these two lines passing through the support vectors of the two classes. They then give the ...
- 413
1
vote
0
answers
74
views
Mode of Joint Posterior - Maximization Problems
I have a problem whereby I get two different answers if I try to maximize a function.
let
$ \begin{bmatrix} Y_{o}\\ Y_{a} \end{bmatrix}|\phi\sim N (0,\phi^{-1}A^{-1}) $
$\pi(\phi)=\frac{1}{\phi}$,
...
- 121
10
votes
3
answers
433
views
If $Z_i =\min \{k_i, X_i\}$, $X_i \sim U[a_i, b_i]$, what is the distribution of $\sum_iZ_i$?
Assume the following set up:
Let $Z_i = \min\{k_i, X_i\}, i=1,...,n$. Also $X_i \sim U[a_i, b_i], \; a_i, b_i >0$. Moreover $k_i = ca_i + (1-c)b_i,\;\; 0<c<1$ i.e. $k_i$ is a convex ...
- 60.8k
2
votes
1
answer
315
views
Understanding the tradeoff with regularization in SVMs
In the linear SVM model, one may have the following equation to describe how to achieve a maximal margin while still classifying the data into 2 groups:
\begin{equation}
L(w, \epsilon) = w\cdot w + \...
- 413
3
votes
1
answer
2k
views
Extreme value simulation with Monte Carlo
I would like to seek your help with some questions to simulating extreme values. For example, I have written the following R code to generate QQplots for a normally distributed data, varying the size ...
- 133
0
votes
1
answer
3k
views
Best method to fit a GEV distribution with generalised linear modelling of parameters?
I need to fit a generalised extreme value distribution to my data but I want the ability to perform generalised linear modelling of the parameters, particularly the location. Can anyone recommend the ...
- 1
3
votes
1
answer
158
views
Joint distribution of a random variable and the sample maximum
This is one necessary part of a slightly larger problem, but this part has me stumped.
We have that $X_1, X_2, ..., X_n\stackrel{iid}{\sim} U(0,\theta)$. What is the joint density of the first ...
- 39
3
votes
1
answer
332
views
Extreme value distribution for multivariate normal
I have a series of data sets. Each data set represents a measurement in 3D space relative to a global origin. I want to model the extreme values of my data. If I were to calculate the extreme radius ...
- 1,191
1
vote
1
answer
3k
views
How to calculate max/min scales on a scatter plot
I have 3 log scatter plots that I want to establish smooth maximum and minimum lines. What is the usual mathematical method to do that? (Image and Excel file links below.)
The black lines on the ...
- 39
1
vote
0
answers
24
views
Newton Raphson Algorithm: negative semi definiteness [duplicate]
I would like to minimise the function $l(\theta|Y)$.
Given the Newton's method below
$$\theta^{(t+1)} = \theta^{(t)} - \left[l''(\theta\;|\;Y)\right]^{-1} l'(\theta^{(t)}\; | Y)\quad t = 0,1,...$$
...
- 1,893
2
votes
1
answer
266
views
Lévy stable vs. extreme value distributions
I'm trying to understand the advantages (if any) of employing the Generalized Extreme Value distribution (GEV) vs. a stable distribution in the context of understanding the probability of crossing a ...
- 958
8
votes
1
answer
3k
views
How to derive the $\alpha$ for the Pareto rule
Suppose we have the CDF for the Pareto Distribution given by:
$$ P(X \leq x) = 1-\left(\frac{x_m}{x}\right)^\alpha \;\;\;\;\;\;\;\;\;\; x \geq x_m$$
What is the intuitive way to find the alpha for ...
- 412
2
votes
1
answer
141
views
Why is it that it is self-defeating to use the posterior mode as the bayes estimator in this case?
I am reading through an applied statistics book and in it, it makes a very luminous statement for a posterior case where the likelihood was taken from $X_1,...,X_n$ iid random variables from a ...
- 2,107
3
votes
0
answers
107
views
Repairable system and the sum of GEV random variables
We know that $X\sim {\mathrm {GEV}}(\alpha ,\beta ,0)$ and $Y\sim {\mathrm {GEV}}(\alpha ,\beta ,0)$ then $X+Y\sim {\mathrm {Logistic}}(2\alpha ,\beta )$.
I am wondering, what will be $X+Y+Z$ ...
- 328
3
votes
0
answers
671
views
Conceptual or mathematical motivation for the three extreme value distribution types?
What motivates, justifies, gives rise to the differences between the Gumbel, Fréchet, and Weibull distributions? Glen_b's comment indicates that they are distributions for extreme values generated by ...
- 1,108
1
vote
0
answers
99
views
Fitting of bivariate data to a self-defined probability density function
I have a bivariate set of data points which I want to fit to a self-defined distribution (i.e. not standard normal or chi-square or like that, a different, let's say "new" density function). I would ...
- 11
3
votes
1
answer
4k
views
Fitting a linear model with non gaussian noise
I am trying to evaluate the elasticity of prices of some goods. I am concerned about the gaussianity of the noise in the prices. With non gaussianity I am referring to the non existence of the firt/...
- 2,098
4
votes
1
answer
590
views
expected lowest value of 10 normally distributed values
Consider 10 values that follow a standard normal distribution. What would you expect to be the lowest value?
I tried to simulate this problem in R. I basically just simulated 100000 standard normal ...
- 311
2
votes
0
answers
1k
views
maximum gap between order statistics of normally distributed random variables [closed]
I am currently working on a not-that-easy problem involving order statistics. As I am unsure as to how I could solve it, I thought it might already possess a solution. So here I am, my questions is: ...
- 21
2
votes
1
answer
265
views
KL divergence minimisation equation
I am looking at some literature on KL divergence minimisation and am having trouble understanding the derivation of the second order moment. So, if we have a distribution from the exponential family, ...
- 4,730
7
votes
1
answer
125
views
Measurement error in maximum counts
I'm familiar with the concept of a mean value of data and the variation around the mean. Is it possible to quantify variation around maximum values?
For example, take the below data collected across ...
- 14.6k
2
votes
1
answer
937
views
Fitting a probability distribution to non i.i.d. data? [closed]
I have temperature time series data that I have determined is not independently and identically distributed (from looking at the autocorrelation plots and Ljung-Box tests).
However, I am still able ...
- 21
7
votes
1
answer
249
views
Compare maxima of two Gaussian samples
Suppose $X$ and $Y$ are both normally distributed, with $X \sim \mathcal{N}(0,1)$ and $Y \sim \mathcal{N}(c,1),$ where $c > 0$. Consider $n$ independent draws of both $X$ and $Y$. As $n \rightarrow ...
- 874
2
votes
1
answer
559
views
How to get Pareto IV parameter estimates
I have a serie with 850 observations, and I need to fit the Pareto IV distribution. How could I do this in R?
I read the guide VGAM, however, I'm not able to run it.
If anyone knows, please provide ...
- 21
2
votes
2
answers
2k
views
Most suitable algorithm for optimizing Maximum likelihood function
What is the most suitable optimization algorithm for optimizing maximum likelihood estimator? In excel I used GRG non linear optimization algorithm, is that good enough?
I want to write my own code ...
- 205
4
votes
2
answers
6k
views
Minimizing relative error (or mean square error) and maximizing likelihood
I'm not a statistician, so I would appreciate an answer in the simplest possible words.
I've read that, in some sense, when we minimize the mean square error, we are maximizing the likelihood.
This ...
- 83
5
votes
3
answers
309
views
Estimating the minimum of a finite probability mass function
Suppose we are given a discrete r.v. $X$, distributed according to some unknown, finite probability mass function $p(x)$. We can assume that $p(x)>0$ for every $x$ in its domain. We can sample $X$ ...
- 51
11
votes
1
answer
14k
views
How to know if my data fits Pareto distribution?
I have a sample which is a vector with 220 numbers. Here is a link to a histogram of my data..
And I wish to check if my data fits a Pareto distribution, but I don't want to see QQ plots with that ...
- 585
1
vote
0
answers
166
views
Find trendline for minimum (not mean) values in distribution
I would like to perform something like a linear regression on my distribution of data, but I'm interested in a trendline that estimates the minimum, not mean, value for each time bin. I'd like to do ...
- 125
11
votes
4
answers
41k
views
How to check if my data fits log normal distribution?
I'd like to check in R if my data fits log-normal or Pareto distributions. How could I do that? Perhaps ks.test could help me do ...
- 585
9
votes
1
answer
9k
views
Expected value of minimum order statistic from a normal sample
UPDATE Jan 25th 2014: the mistake is now corrected. Please ignore the calculated values of the Expected Value in the image uploaded - they are wrong- I don't delete the image because it has generated ...
- 60.8k
2
votes
0
answers
184
views
Posterior distribution
Suppose $X1,..,X4$ be iid from pdf $f(x|\theta)=\frac{1}{\theta}$ ,for $0<x<\theta$.
The prior distribution is
$\pi(\theta)=\frac{2}{\theta^3}$ , for $\theta>1$
I have to obtain:
a)...
- 851
3
votes
1
answer
870
views
$\mathbb{E}$ and Variance of the maximum of independent $\mathcal{N}(\mu_i, \sigma_i^2)$
I am interested in the expectation and the variance of the maximum of several independent, normal distributed variances. That is, given a set of $I$ different RVs with $X_i \sim \mathcal{N}(\mu_i, \...
- 14k
5
votes
1
answer
169
views
Distribution of Extreme Spread for n, sigma
Simple form provided by WHuber: What is the distribution of the diameter of n points in the plane drawn iid from a bivariate Normal distribution? (Diameter is the greatest distance among any pair of ...
- 1,176
2
votes
0
answers
589
views
Convergence in Probability of the minimum
This is a homework question. I think I have the correct answer, but I am not sure. Also, the wording sounds very awkward. Is there a better way to show this (or better way to word this)?
Let $X_1,\...
- 10.2k
1
vote
2
answers
779
views
Joint cdf of extreme values
A die is rolled twice,
$X_1$ : the minimum value to appear in the two rolls
$X_2$ : the maximum
I would like to derive $\ F_{X_1,X_2}(x_1,x_2)$.
I know that that the CDF of $\ X_1 $ = $\ 1- [1-...
- 13
4
votes
1
answer
1k
views
Convergence in probability of minimum
This is a homework problem. Suppose we have a random sample $X_1,\ldots,X_n \overset{iid}{\sim} F$ with density $f(x) = 2(x-\theta)$ for $x\in (\theta,\theta+1)$. Let $X_{(1)} = \min{\{X_1,\ldots,X_n\}...
- 10.2k
9
votes
3
answers
2k
views
Extreme value theory for count data
I am aware of extreme value theory for continuous distributions. I need to fit an extreme value distribution to the maximum observation of number of events on a day, per month. This seems to be the ...
- 205
5
votes
2
answers
2k
views
Expected maximum given population size, mean, and variance
How would one estimate the maximum given population size, a few moments, and perhaps some additional assumption on the distribution?
Something like "I'm going to do $N_s≫1$ measurements out of ...
- 205
2
votes
0
answers
395
views
Is it appropriate to estimate a Pareto regression's α by maximising R-squared?
Fitting a power law regression. What are the downsides of estimating α by maximising the R-squared of (a) the theoretical values output by the regression with the candidate α and (b) the empirical ...
- 679
0
votes
1
answer
186
views
Expectation of the Pareto distribution [closed]
I would like to know if my understanding of the following is correct. This has been tripping me up for a long time now.
Compute $\lim_{x\rightarrow \infty}x^{1-\beta}$.
This is part of a homework ...
user30602
1
vote
1
answer
572
views
Why am I getting this result in modeling a Pareto Type II distribution in Excel?
In Excel for a project I'm trying to model the density, distribution and survival function ($1-F(X)$) and I can't get the density to sum to one and I can't get the distribution to go to one. For the ...
- 1,239
5
votes
1
answer
733
views
OLS robust to outliers
I am facing the following problem: I have a training sample and estimate a model on that training sample. My model is simply OLS: $y_t = a + \beta x_t + \varepsilon_t$. The model is estimated on ...
- 853
1
vote
1
answer
1k
views
What is loc parameter in GPD distribution in POT package for R?
I fitted the Generalized Pareto distribution (GPD) using the POT package in R.
The fitted object provides shape and scale ...
- 131
3
votes
1
answer
2k
views
Can the mean deviation about mean exceed the standard deviation for the Pareto distribution?
Can the mean deviation about mean exceed the standard deviation for the Pareto distribution?
I just went through some books and found they are claiming that it cannot.
How can I prove that? What is ...
3
votes
1
answer
406
views
How to use the Pareto distribution in fitting survival curves?
I have a series of numbers, which are some survival probabilities that form a decaying curve. I would like to fit them with a "Pareto" distribution. I expect to have a smooth fitted curve, thus I can ...
- 275
2
votes
3
answers
12k
views
Fitting data in a generalized Pareto distribution and parameter estimation
I have log(return) data as time series, how I can fit this data in a Generalized Pareto distribution and estimate the parameters of this distribution, any kind of resource
pointer with clear code ...
- 151