Questions tagged [quantiles]
The quantiles of a distribution refer to points on its cumulative distribution function. Some common quantiles are quartiles and percentiles.
840
questions
3
votes
0
answers
43
views
Can we write $P(A > B)$ in terms of their quantile functions?
Suppose $A$ and $B$ are two independent, continuous random variables with bounded supports $[min, max]$ and densities $f_A(a), f_B(b)$ and CDFs $F_A(a), F_B(b)$. Can we write $P(A > B)$ in terms of ...
- 31
1
vote
1
answer
32
views
Prediction interval for continuous positive data including zeros
I am trying to figure out given a set of positive continuous numbers (including zeros) what would be the most appropriate way to calculate a prediction interval, or a normal range, i.e. a range in ...
- 5,406
2
votes
2
answers
855
views
Should I treat these data points as outliers?
Currently, I am building my analytics portfolio as part of the Google Data Analytics course. I chose the option to analyze Divvy Bike Sharing data for the year 2021. But now I'm currently stuck in the ...
1
vote
0
answers
14
views
Combining distribution stats of multiple populations
I have the following statistics of two independent populations:
First population:
...
- 51
0
votes
0
answers
29
views
Will taking mean of sample quantiles be an unbiased estimate the population quantile?
Say I have $N$ samples of 100 numbers all drawn IID from the same distribution $\mathcal{D}$. For each sample, I take the 95th quantile to get $N$ sample quantiles $\hat{q}_n$. Will taking the average ...
- 231
0
votes
2
answers
35
views
qBCCGo function in GAMLSS package (quantile function)
I am working with GAMLSS technique and I use LMS (with three parameters of mean,variation and skewness) method. I found the follwoing formula to calculate the lower limit of normal as the link ...
- 79
1
vote
2
answers
35
views
Correcting for sample size in skewed data
Empirically I have noticed that when data is skewed, and when for example interested in the 99th percentile of latencies, that it is difficult to compare experiments with different sample sizes
...
- 2,289
0
votes
0
answers
40
views
Estimates of the Quantile Regression
In the Quantile regresion model we assume that$$y_i=x_i^T\beta +\epsilon_i,\ i=1, 2, 3, ....,n$$ such that the conditional quantiles of the error terms are zero and for the OLS model we assume that ...
1
vote
0
answers
32
views
Does the zeroth and 100-percentile exist and do the minimum and maximum of the data belong in those respective sets?
Does the zeroth and 100-percentile exist and do the minimum and maximum of a set of data belong in those respective sets?
0
votes
0
answers
18
views
0
votes
0
answers
29
views
Observation in Quantile Regression
i have 30 observations of yearly data and i try to do quantile but results is all of my variables is not significant. im wondering if we can use quantile regression on yearly data? or is there any ...
0
votes
0
answers
27
views
Best practice for estimating values between 5% and 95% CIs for multimodal distribution
How can I improve my quantile estimation method for estimating values between the 5% and 95% confidence intervals for a multimodal distribution (as a mixture of gaussians) ? Where the CI are only for ...
- 547
0
votes
1
answer
34
views
How to average multiple non-normal distributions?
I have the following statistics of two independent random variables:
First random variable:
...
- 51
0
votes
0
answers
16
views
The best way to determine what percentile a range of numbers closely aligns with?
I have a table of ~600 numbers and percentiles like below:
Data
40th
50th
60th
70th
80th
90th
71
53
55
57
60
63
68
101
78
81
83
88
92
102
121
89
94
98
105
110
123
209
146
153
163
171
176
189
68
...
- 1
1
vote
0
answers
19
views
Asymmetries in the DKW bound
Suppose I have $n$ i.i.d. samples $X_1,...,X_n$ drawn from a distribution with CDF $F$. We use the samples to form the empirical CDF:
$$F_n(x)=\frac{1}{n}\sum_{i=1}^{n} \mathbb{1}_{X_i\leq x}$$
The ...
- 841
2
votes
2
answers
50
views
Finding random number based on a specific discrete distribution using R
I have a discrete distribution with cumulative function:
$$
F(x)=Pr(X \leq x) = \frac{1}{(x-1)!}
\left( \Gamma(x,\lambda)-\frac{\lambda^{x-1} \exp(-\lambda)}{\lambda+1} \right)
$$
for $x=1,2,3,\ldots ....
- 344
1
vote
1
answer
29
views
Quantile(X * k) = Quantile(X) * k?
I just wanted to verify that this is true.
Say the .95th quantile of a random variable, $X$, is $q$. Then the .95th quantile of $\frac{X}{k}$ is $\frac{q}{k}$ where $k$ is a constant.
I am assuming ...
- 1,671
0
votes
1
answer
69
views
Robustness of Quantile Regression
Is the 99th Quantile Regression model a robust model?
From my understanding, Quantile Regression is supposed to be robust in nature, but removing some outliers using IQR, the results obtained by 99th ...
- 21
1
vote
0
answers
29
views
Estimate Conditonal Moments from Conditonal Quantiles
In Chang et al. "The Higher Moments of Future Earnings" (2014), the authors say say that based on (predicted) conditonal quantiles of a variable $y$, one can derive the (predicted) ...
- 939
3
votes
1
answer
106
views
Calculating expected value from quantiles
For probabilities $p_i=\frac{i}{10}$ where $i=1, \dots, 10$, the respective quantiles are $\tau_i$. How can I calculate an approximate expected value?
- 224
1
vote
2
answers
39
views
Expected Shortfall vs VaR
I have a question about how to compute the Expected Shortfall practically. I know from the theory that the ES is the conditional Expectation of the Loss distribution (conditional on the VaR) and that ...
- 25
1
vote
0
answers
20
views
Quantitatively determining a "baseline" value for a dataset
I'm wondering if there's a clean way to quantitatively find the "baseline" of a data set, or more specifically, a threshold below which the majority of samples are clustered. For the example ...
- 11
1
vote
1
answer
27
views
Calculate value at risk from a given discrete distribution
How do I find the value at risk at 90% confidence interval i.e., VaR(10%) for a portfolio with the following profit/loss distribution?
[-120 with prob = 0.0036, -55 with prob = 0.1128, and 10 with ...
- 13
0
votes
0
answers
55
views
Understanding the computation of p-value and its v-test by catdes() of FactoMineR
One of the functions that helps to characterize clusters is the catdesc() function. The output of this function is very informative giving the following bits of information:
...
- 1,085
0
votes
0
answers
22
views
Different Calculation Methods for Theoretical Quantiles of Q-Q Plot
There seem to be at least two different methods to calculate the theoretical quantiles in a Q-Q plot. In the following, the normal distribution is assumed to be the theoretical distribution.
Split ...
- 35
2
votes
1
answer
30
views
Almost sure convergence of the difference of two quantiles
Let $X_1, ..., X_n$ be i.i.d. random variables with the same distribution as the random variable $X$. Let $F$ be the distribution function of $X$. We assume that $F$ and its quantile function $F^{-1}$ ...
- 353
0
votes
0
answers
39
views
How to calculate VaR on DCC GARCH?
I am using a DCC (and CCC) GARCH model to model volatility of electricity price returns (using spot and futures) i.e. I have more variables in my model. I need to calculate VaR using these models but ...
0
votes
1
answer
92
views
Compute value at risk from DCC GARCH (in R)
I am struggling to obtain the Value at Risk from my DCC GARCH analysis. I have done the following code:
...
3
votes
1
answer
177
views
Estimating a quantile from an unknown distribution
I have a variable $x \sim N(0,1)$, of dimension $p$, with each $x_i$ independent. I want to find the quantile, $q$, such that
$$\mathbb{P}\left(\frac{\sum_{i=1}^p (|x_i| - b_i) }{c} \geq q \right) = a,...
- 85
0
votes
0
answers
78
views
Quantile Regression Pseudo R-Squared
When computing the Pseudo R-squared value in quantreg in R or statsmodel in Python, what is an acceptable range to justify goodness of fit? Also, what is the functional form of Pseudo R-Squared ...
- 21
1
vote
1
answer
21
views
90th percentile growth rate distribution
It's been a while since I dealt with statistics, I guess I need to brush upon my knowledge about it but as I was reading an article for my class I had a hard time understanding the author's main point ...
0
votes
0
answers
35
views
How do I measure goodness of fit of data transformations that standardize for variables?
I am modeling a dependent variable which has significantly different distributions when grouped by various independent variables. Consequently, it is difficult to compare previous values of the ...
- 133
0
votes
0
answers
19
views
Should I remove/cap "Extreme Values" from Variables and at which percentile should capping be done?
In my dataset, I have a variable corresponding to the monthly average playtime hours of the players. The dataset has 200K records. I estimated various percentiles of the variable - from 0 to 1 for ...
- 143
3
votes
1
answer
227
views
Compute the quantile function of this distribution
I want to compute the quantile function of a Bin(4, 1/3) random variable and plot it with R, but I don't know where to start from.
Any help or hint is highly ...
- 63
0
votes
0
answers
26
views
Issues with transforming the response using quantile mapping?
I am in a modeling setup where having a normal (or T; unskewed) response distribution is important.
I have chosen to use quantile mapping to transform my response. My process is to use GAMLSS in R to ...
- 11
1
vote
1
answer
100
views
Quantile Regression proof
I'm interested in understanding the formal proof of why Quantile Regression works. That is, show me in which conditions the pinball/quantile loss provides asymptotic consistent estimations of the ...
- 13
0
votes
0
answers
25
views
How to best categorize a continuous variable that has many observations with the same value?
I have an air pollution variable (PM2.5 measurements) for a dataset of 9,506 participants based on geographical location. Because many of the participants live in the same general area, there is a ...
1
vote
1
answer
65
views
How to compute 90-th percentile of data with Inf values? [closed]
I need to compute an effective diameter is 90-th percentile of the distribution of shortest path lengths of a graph.
My attempt is:
...
- 844
0
votes
1
answer
27
views
Percentile(s) - logical explanation
What EXACTLY do percentiles (/does which) mean, how are they distributed exactly (equally/weighed)?
1
vote
2
answers
26
views
How to estimate correlations between the standard errors on empirical quantile estimates from two correlated series?
I have two very long series of samples from a pair of correlated random variables, $x$ and $y$.
(they are fat tailed distributions, roughly Pareto, if that helps).
(if you're curious...$x$ and $y$ are ...
0
votes
1
answer
88
views
Does this plot indicate the data is normal distributed?
I use qqnorm to plot my data as the photo attached. Does this plot indicate the data is normal distributed?
- 11
3
votes
1
answer
32
views
Jacobian of function returning $m$ evenly-spaced order statistics of an $n$-dimensional vector
Let $y\in\mathbb{R}^n$, and let $f:\mathbb{R}^n\to\mathbb{R}^m$ be the transformation that outputs $m$ evenly-spaced order statistics (including the extremes) of $y$.
What is the Jacobian of this ...
- 1,710
2
votes
0
answers
93
views
Predicting percentiles with percentile data [closed]
I have a few independent variables and 5 dependent(target) variables. The target variables are percentiles (10th, 25th, 50th, 75th, 90th) and I want to predict the same in that order. What approach ...
- 21
0
votes
2
answers
249
views
How to correctly bucket data into quantiles: by index or by value?
I've been recently involved in a discussion surrounding the assignment of individual values to n-tiles.
One way of doing it would be to rank all of the data in ascending order, dividing it into $n$ ...
- 31
0
votes
0
answers
13
views
Is there any way to adjust a percentile for number of observations per person?
I've got a program that I'm trying to measure the latency of at the 95th percentile. The data has a long tail, so I'd expect the 95th percentile to be relatively high.
To give a contrived example, let'...
0
votes
0
answers
112
views
What's a good rule of thumb for choosing a sufficient number of quantiles in quantile transformation?
I'm a currently developing a regression model based on the Choquet Integral. To tackle outliers I am using the Quantile Transformer, provided by scikit-learn. I was wondering how the quantile number ...
1
vote
1
answer
61
views
Quantiles of the response variable in generalised linear model with binary outcome (survival analysis)
I have a dataset that looks more or less like this (interval-censored outcome, multiple screening events per patient, not synchronised):
...
- 39
0
votes
0
answers
52
views
Getting normal distribution from spike distribution through data transformation
I'm trying to preprocess my data before feeding it to neural network. The goal is to get a normal distribution. I have 3 different features distributed as follows:
One can see bimodal, skewed and ...
1
vote
0
answers
25
views
Bounds on percentile given z-score
Suppose we have a random variable $X$ that can take real values from 0 to 100, according to a distribution $f$, for which the expectation $E(X)$ and the variance $SD(X)^2$ are well defined and finite.
...
1
vote
1
answer
68
views
How to decide cut off?
I will try my best to explain this simple doubt that I have. I have three responses (Increased, decreased and no change) and I have calculated the percentage for each. For instance, for X case- 46% ...