Questions tagged [quantiles]
The quantiles of a distribution refer to points on its cumulative distribution function. Some common quantiles are quartiles and percentiles.
547
questions
0
votes
0answers
15 views
Why Pearson correlation of two variables is very low but correlation between percentiles is almost perfect?
I have two variables say X and Y and when calculating Pearson correlation
I found a very low value ~.01 but when I calculate the correlation between variables's percentiles, say Xp and Yp I found a ...
0
votes
0answers
13 views
Effective samples size for quantile estimates with weighted samples
I've been reading the paper
Rethinking the Effective Sample Size. In it, the authors provide an estimator for effective sample size given weighted samples (eq. 5.1 in the paper) which specifically ...
0
votes
0answers
17 views
How to apply Quantile Normalization for three datasets : training, testing and validing dataset?
I have a question regarding the Normalization of data for the purpose of classification:
I am applying different classification algorithms and want to normalize my data.
I have three different data: ...
0
votes
0answers
24 views
Does comparing percentiles of datasets with different sample sizes introduce bias?
Imagine an A/B test running on a website where we measure a metric like page load time. 90% of the page views are the 'control' and 10% of the views receive a special treatment that might affect the ...
1
vote
0answers
47 views
Finding Quantiles and Sampling from a Mixture of Distributions
I am trying to replicate a result in this paper, specifically I am trying to implement the mixture distribution on page 9 of the document.
The authors describe a hazard function:
$$
h(x):=h_{1}(x) \...
1
vote
0answers
33 views
Skewness and kurtosis using quantiles and mean/variance
I would like to ask if there is a way to get skewness and kurtosis measures if we only know the distribution's mean, variance, and certain quantiles.
Basically, the problem that I am facing is I have ...
1
vote
0answers
7 views
Estimation of population quantile based on mutually exclusive sub-populations?
I have read somewhere that "it's obvious" that the estimator $\hat{q}\%$ where $q\%$ is the q-quantile% of a $X_i iid \tilde{ } Dn$ for some distribution $Dn$ for $...
0
votes
1answer
36 views
Want to know the differences between exponential and heavy-tailed distributions in terms of first and third quartiles
What are the differences between exponential and heavy-tailed distributions, please illustrate this difference by explaining how well the first and 3rd quartiles describe them.
I know what are ...
0
votes
0answers
6 views
Calculating quantiles of pooled-sample NHANES PCB data
I am comparing PCB laboratory results to NHANES population quantiles. This is straight forward with 2003-2004 data.
R code:
...
0
votes
0answers
4 views
Is it better to rely on the average latency or on the percentile to apply Little's Law?
I was reading this post (Little’s Law- An insight on the relation between latency and throughput) about the relation of Latency vs. Throughput and using Little's Law to have an insight about them. ...
2
votes
1answer
42 views
How do you forecast a time series that is inherently uncertain?
Most time series forecasting models take in fixed (so presumably deterministic) historical values, and then output either:
A point (usually mean) forecast + forecast intervals, so $\hat{Y}_{t+1} = ...
0
votes
1answer
27 views
calculating percentiles (quantiles) from GAM predictions in R
I'm working with a bird migration dataset, and exploring some different metrics to quantify changes in migration phenology (timing) over several decades. There are many different approaches to do this,...
1
vote
1answer
72 views
Estimating Quartiles with Moments
The Wikipedia article on Skewness indicates that the median of a distribution can be estimated from the mean, standard deviation, and skeweness with an error term that goes as $O(skewness^2)$. ...
1
vote
1answer
40 views
How to calculate pinball loss for quantiles and for point forecasts?
I have a few general questions about pinball loss:
Is a pinball loss typically calculated for each point in the forecast horizon or is it calculated across all points in the forecast horizon?
How is ...
0
votes
0answers
40 views
Why does quantile regression not work for the newsvendor model?
I want to use quantile regression to solve the newsvendor model for a critical fractile of 2/3.
Surprisingly the quantile that minimizes the cost in my study is not the 2/3 quantile but the 80%.
...
0
votes
0answers
24 views
Percentile computation of time series
Say I have $x_1, x_2,...,x_{2000}$ all $x_i$ from $N(0,1)$
Say I create another series as follows:
$$y_j= \sum_{i=j}^{j+100} x_i$$
$y_j$ goes from $1$ to $1900.$
Now I compute $p_1 = 5^\text{th}$ ...
0
votes
0answers
38 views
Quantile regression does not give expected result regarding costs
I want to predict the length of an event. When I underestimate the event length it costs me 2€ (per minute) while an overestimation costs 1€ (per minute). To capture this price inbalance I tried using ...
0
votes
2answers
59 views
Quantiles in a continuous random variable
Suppose that X is a continuous random variable and $x_{0.25}$, $x_{0.5}$, $x_{0.75}$ are the 25th, 50th and 75th quantiles respectively. How can I write in terms of probabilities the fact that "...
0
votes
1answer
34 views
How to fill the bins?
Consider 1 million people earning money, sorted in increasing order. The kth decile, i.e. the kth 100,000 of them has an income share of $f(k)$ with $f(k)<f(k+1)$ and $\sum_{k=1}^{10} f(k)=1$. Let ...
2
votes
1answer
33 views
How to calculate seasonal naive quantile forecasts?
I was reading a chapter on prediction intervals and saw that the standard deviation is used to calculate "prediction" intervals. Earlier today I was reading some results from GEFCom2017 where ...
1
vote
0answers
44 views
Estimating/translating odds ratio for different percentile intervals
I have a question regarding a translation in odds ratio curve OR(x). More specifically, let's say we have an OR(x) which we get using logistic regression. We look at the interval bounded with 25% and ...
1
vote
1answer
41 views
How to Average Percentiles
I'm currently writing my Master's Thesis which focuses on building software to do some things.
I ran a few benchmarks for my program (Program A) and current alternatives (Program B) and want to ...
1
vote
0answers
16 views
Is it possible to get equivalences of Western Electric Rules in non normal distribution with percentile values?
I am not a statistician and I am a bit lost with these concepts, but I will try to explain my situation.
I have a non-normally distributed data, and I am trying to apply the Western Electric Rules to ...
1
vote
0answers
41 views
What would a good low variance quantile estimator be?
I'm new to quantile estimation. I have a bunch of iid random variables, say $X_1, \dots, X_m \sim p^\star$. Suppose I don't know their law.
Supposing my cdf is $F$, I want to estimate quantiles $q_1 =...
0
votes
2answers
61 views
How do I calculate the standard deviation of a normal distribution given the mean and a quantile value of that distribution?
How can I determine the standard deviation of a normal distribution with known mean and a known percentile value?
The known percentile value would be on the correct side of the mean (eg, for ...
0
votes
1answer
22 views
Are there two different possible meanings for a quantile group?
I'm aware that when a quantile is mentioned, it may mean a point or it may mean a group. But i'm not asking about the quantile points.
In the case where it means a group, then there seem to be two ...
27
votes
5answers
4k views
Are there 99 percentiles, or 100 percentiles? And are they groups of numbers, or dividers or pointers to individual numbers?
Are there 99 percentiles, or 100 percentiles? And are they groups of numbers, or divider lines, or pointers to individual numbers?
I suppose the same question would apply for quartiles or any ...
6
votes
1answer
189 views
Is the sample quantile unbiased for the true quantile?
I would like to find a way to show whether the sample quantile is an unbiased estimator of the true quantiles. Let $F$ be strictly increasing with density function $f$. I will define the $p$-th ...
1
vote
0answers
17 views
A maximization problem involving random variables: a special case
Consider random variables $X$ and $Y$ that are jointly normally distributed,
$$
\begin{pmatrix} X \\ Y \end{pmatrix} \sim \mathcal{N}
\left[
\begin{pmatrix} \color{blue}0 \\ \mu_Y \end{pmatrix} ,
\...
0
votes
0answers
11 views
Quantiles of the Q values of an MDP
Cross-posted from Math StackExchange:
Consider an MDP with $n$ states, $k$ actions, and discount factor $\gamma \in [0,1)$. We are uncertain of its reward function $R \in \mathbb{R}^{n \times k}$ and ...
0
votes
0answers
24 views
normal quantile integration
Let $X$ be normal distributed with $\mu, \sigma$ and let $F$ and $f$ be the cdf/pdf.
I would like to calculate the following integral:
$I = \int_0^1 e^{(-ap)}F^{-1}(p)dp$
let $p = F(x)$, then
$I = ...
1
vote
1answer
22 views
Upper and lower quantiles
I have a question regarding upper and lower quantiles.
In my lecture notes these are defined as:
$Q_p[X] = \inf\{x \mid P(X \leq x) \geq p\}$
is the lower p-quantile
$Q^p[X] = \inf\{x \mid P(X \...
0
votes
2answers
35 views
Should 99 percentile of a union of two sets be the average of 99 percentile of each set?
For example, the 99 percentile value of list A is p99_a, the 99 percentile value of list B ...
5
votes
1answer
200 views
What is the quantile covariance?
Suppose that $X$ is a p-dimensional random vector and $Y$ is a random scalar. Then, Dodge and Whittaker (2009) indicate that the covariance of these two variables can be formulated as a minimization ...
1
vote
0answers
25 views
A maximization problem involving random variables
Consider random variables $X$ and $Y$ that are jointly normally distributed,
$$
\begin{pmatrix} X \\ Y \end{pmatrix} \sim \mathcal{N}
\left[
\begin{pmatrix} \mu_X \\ \mu_Y \end{pmatrix} ,
\begin{...
3
votes
0answers
129 views
Quantile transform vs Power transformation to get normal distribution
I was introduced to the concept of quantile-based gaussian transform. To my understanding, it changes the value of the original data by each percentile to the matching percentile of gaussian ...
0
votes
0answers
19 views
Sample size for n-th percentile order statistic with no approximation for non normal distribution [duplicate]
Given we have population (N) say 1e9, how to estimate sample size proper sample size for obtaining valid n-th (say 99) percentile value with small margin of error ? can we use this calculation
This ...
0
votes
0answers
10 views
Understanding percentiles of child or constituent distirbutions
Consider an example where a user performs some operation on a web page and total time taken for that operation is some of 2 parts. I.e.,
Total time = Part1 time + Part2 time.
What we currently have ...
2
votes
2answers
51 views
What is the variance of the mean, conditional on being between two order statistics or quantiles?
Suppose I have a simple random sample drawn from a population with a known distribution on some population characteristic like height or income, with probability density function (pdf) $f(x)$. Order ...
0
votes
1answer
25 views
Definitions of VaR (Value at Risk)
Here is the definition of VaR (Value at Risk) taken from McNeil, Alexander J., Rüdiger Frey and Paul Embrechts (2015), Quantitative risk
management: Concepts, techniques and tools:
$$
\textrm{VaR}_{\...
1
vote
0answers
32 views
Statistician's quantiles versus policymaker's quantiles: calculating the variance correctly
When speaking of income, although statisticians and policymakers (and their staffs) both refer to deciles, quintiles, and the like, they mean quite different things by these terms. When a statistician ...
1
vote
0answers
15 views
Convergence of empirical quantiles to theoretical quantiles - mixed type distribution
It is well known that under certain (not too restrictive) conditions empirical quantiles of a distribution converge to the corresponding theoretical quantiles in probability as the sample size $n \to \...
0
votes
0answers
10 views
Calculate quantile using z-score in a normal distribution [duplicate]
I've got some z-score values and need to get the percentile (normal distribution)
Of course I can use tables like: https://www.pindling.org/Math/Learning/Statistics/z_scores_table.htm
But shouldn't ...
1
vote
1answer
48 views
Getting different Percentile results when using two different software
I'm calculating 90th percentile values in two different places using two different pieces of software, and I'm getting different results.
These results need to be comparable. I understand there are ...
0
votes
0answers
20 views
Calculating percentile over time [duplicate]
I have a time series data and I want to calculate p90 percentile.
Because there's a huge amount of data I need to split my calculations somehow.
I was thinking that maybe could calculate the ...
0
votes
0answers
12 views
Matching percentiles from aggregate distribution on individual components
I have monthly data on fund flows from Jan-2007 to June 2019 for 100 funds.
I compute two series: :
one series where I collate all funds flows into a single series (let's call it collated series, ...
1
vote
1answer
36 views
Statistics probability help - can someone confirm my solution? [duplicate]
In a certain population, 15% of the people have Rh-negative blood. A blood bank serving this population receives 100 blood donors on a particular day.
(a) What is the probability that 15 to 20 (...
0
votes
0answers
27 views
Usefulness and validity of Alternative definitions of “quantile”
According to textbook, the $p\,$th quantile of a random variable $X$ is any real value $x$ satisfying $P(X \geq x)\geq 1-p$ and $P(X \leq x) \geq p$.
Why isn't the alternative definition,
a $p\,$th ...
2
votes
2answers
94 views
Would this Normal Quantile Plot be considered normal?
How would I describe this QQ plot? Would it indicate fair normality?
Or would it be skewed? I have attached the histogram also.
I have been asked to the answer the following:
"By referring to the ...
1
vote
1answer
32 views
find decile bounds for each group in r
Let's say that I have two variables: x1 is numeric and x2 is factor. I need to find the upper and lower value for each decile of ...
