New answers tagged

0 votes

Betting on a sample from a known distribution

The median minimizes expected absolute distance between observations sampled from the distribution and a single value. That is: $$ \underset{m}{\text{min}}\text{ }\mathbb E\left[ \left\vert X - m\...
user avatar
  • 31.1k
0 votes

Find pdf of X+Y

The sum of exponential distributions each with it's own parameter turns out to be a hypoexponential distribution (https://en.wikipedia.org/wiki/Hypoexponential_distribution). $$\sum_{i=1}^n \text{Exp}(...
user avatar
  • 6,190
2 votes

how to generate data from beta-Liouville distribution?

Extracting the important bits from this density, with new notations, $$p(\mathbf x) \propto\prod_{i=1}^k x_i^{\alpha_i-1}\left[ 1-x_1-\cdots-x_k\right]^{\alpha_{k+1}-1} \left[ 1-x_1-\cdots-x_k\right]^{...
user avatar
  • 91.8k
0 votes

Betting on a sample from a known distribution

The mean is affected by outliers in the distribution, so does not inform us about the mid-point of the distribution. The median is not affected by outliers. Consider three example distributions and ...
user avatar
  • 476
0 votes

Binary Classification Problem with Predicted Probabilities distribution skewed

It could be that you leaked data in some step of your process, but we hope to achieve such performance. It’s great to be able to look at a case (or have the model look at a case) and say, “Yep, that’s ...
user avatar
  • 31.1k
3 votes
Accepted

Usefulness of KS tests and other similar distribution comparing tests

This is one of those ideas that starts out sounding great but winds up being less helpful than one might hope. For instance, just because KS (or a similar test) says that the feature has a different ...
user avatar
  • 31.1k
0 votes

What distribution to sample X from to get an uniform distribution in Y?

There is no need for absolute values around your sine function. f(x) = sin(x) is a perfectly fine pdf on the sample space [0, pi/2]. As schotti points out, you can create an RV with this pdf by taking ...
user avatar
3 votes
Accepted

Prove $P(X_1+X_2> 2C) \leq P(X_1>C)$ if $X_1,X_2$ are identical, but dependent?

Counterexample $$P(X_1 = 3, X_2 = 7) = P(X_1 = 7, X_2 = 3) = 0.5$$ Then $$P(X_1 \geq 4) = 0.5$$ and $$P(X_1 + X_2 \geq 8) = 1$$ Thus in this case $P(X_1 \geq c) < P(X_1 + X_2\geq c)$ Below is a ...
user avatar
1 vote

$X_i, X_j$ independent when $i≠j$, but $X_1, X_2, X_3$ dependent?

One that's perhaps easier to think about comes from a chessboard. Pick a point uniformly on the chessboard and consider $X_1$: row number (1-8) modulo 2 $X_2$: column number (1-8) modulo 2 $X_3$: ...
user avatar
2 votes

Statistical Models that "Exploit" Distributional Knowledge of the Predictor Variables

In regression models, by definition, we are interested in the conditional distribution of the response variable $Y$ given the observed predictors (covariates) $X$. Namely, if the joint distribution ...
user avatar
  • 3,122
3 votes

Distribution of the Spearman rank correlation coefficient under the assumption of non-zero correlation

It's not possible to do this exactly, as knowing the marginal distributions and a correlation coefficient is not sufficient to determine the joint distribution, which would be necessary to do this. ...
user avatar
  • 32.7k
1 vote

Methods for modelling distributions?

A possible option might be to consider each of the vectors corresponding to a single observation of a distribution as independent observations coming from the same distribution f and then model, not ...
user avatar
1 vote

What distribution to sample X from to get an uniform distribution in Y?

The question doesn't require a specific programming language, which is fine, but I noted that the OP's plot looks like the default style of matplotlib. @jbowman has given a useful r implementation. ...
user avatar
  • 3,423
0 votes

Estimating variance of MLE estimate of Beta-Geometric/NBD without MCMC

the estimated standard errors of the maximum likelihood estimates are the square roots of the diagonal elements of the inverse of the Hessian matrix. If you use an optimizer in R, you should be able ...
user avatar
2 votes

How to convert percentage values from 7 point scale to 5 point scale?

Subject behavior between Likert-7 and Likert-5 is clearly different, but poorly understood. Suppose the following are true. Unless bored or uninterested, subjects tend to avoid the middle value as a ...
user avatar
  • 49.8k
12 votes

What distribution to sample X from to get an uniform distribution in Y?

I suspect the difficulty you are having is in the generation of $x$ from $f(x) \propto |\sin(x)|$. I have coded a very simple acceptance-rejection random number generator in R that will do the job: <...
user avatar
  • 32.7k
13 votes

What distribution to sample X from to get an uniform distribution in Y?

maybe i misunderstand your question, but why don't you sample from a uniform distribution and set X to the arccos of your samples? in R, this would be ...
user avatar
  • 391
3 votes

How to convert percentage values from 7 point scale to 5 point scale?

To "translate" the scales, you would need to be able to make an assumption that the distributions of the underlying phenomenon did not change across time, the only difference was the ...
user avatar
  • 113k
5 votes

How to convert percentage values from 7 point scale to 5 point scale?

As already signalled, you can't do much -- without extra information or assumptions. Here, without any strong claims, is a method from Mosteller, F. and Tukey, J.W. 1977. Data Analysis and Regression. ...
user avatar
  • 48.9k
0 votes

What is the 'same distribution' mean?

Strictly speaking, it means that the CDF is the same. That is, the type of distribution, the mean, the variance, and all parameters are all the same, if they are well-defined. For most of the commonly ...
user avatar
  • 143
1 vote

Studying extreme value r.v. $X=\max_i (c_i+X_i)$ where $c_i$ are constants and $X_i$ are i.i.d. r.v

In my notation $X_i \sim f$ with CDF $F$. If $X$ is the maximum of $n$ draws, it means that one sample lies at $X$ and the other $n-1$ must lie below it. Thus we have $$ p(X) = n \cdot f(X) \cdot F(X)^...
user avatar
  • 1,254
0 votes

Turning sleep schedule data into a statistical distribution

Initial thought: Asleep versus Awake at time $t$ is a binary outcome. Let $ x_{it} \mid \pi_t \sim \text{Bernoulli}(x_{it}\mid\pi_t)$, then have $\pi_t$ be a probit transformation of a Gaussian ...
user avatar
  • 173
1 vote

Turning sleep schedule data into a statistical distribution

Strictly speaking (in my humble opinion), you cannot build up an empirical distribution from the data to then detect outliers in the data (sort of a chicken and the egg problem). You can evaluate the ...
user avatar
  • 51
0 votes

Normal Distribution or Not

It seems to be normal, but the transformation you've chosen breaks that assumption. There are way too fat tails in your distribution (left and right), because of the 0% and 12% data transformation. I ...
user avatar
1 vote
Accepted

Questions the calculation of Z score and confidence intervals in biology

My question/concern is, shouldn’t we adjust for the actual distribution of the data? Yes. For example, this helpful Technical Perspective on Empowering statistical methods for cellular and molecular ...
user avatar
  • 62.4k
2 votes

Questions the calculation of Z score and confidence intervals in biology

Mean and sd have an interpretation that isn't exclusive to the normal distribution, so I don't think such a Z score should be seen as requiring normality (normality may be required for certain ...
user avatar
-1 votes

Shapiro Wilk test of normality

One very basic but useful rule* to interpret this kind of test (normality, heteroskedasticity, autocorrelation) is to remember that when the p-value is less than 0.05, it means that something is "...
user avatar
2 votes

Universal approximation of Gaussians

More generally, a Hilbert space $\mathcal F$ of functions from $\mathcal X$ to $\mathbb R$ is a Reproducing Kernel Hilbert Space if and only if for all $x\in\mathcal X$, the map : $$L_x : f\in\mathcal ...
user avatar
8 votes
Accepted

Universal approximation of Gaussians

$L_2$ integrable functions are equivalence classes of functions that can differ on subsets of measure zero. For an RKHS, you need the evaluation function at a given point $x$ to be well defined, which ...
user avatar
  • 2,958
6 votes
Accepted

Sample uniformly from unit square conditioned on sum and product

Let's solve a generalization, so that we can obtain both solutions at once. Let $h:[0,1]^2\to\mathbb{R}$ be differentiable with derivative $\nabla h=(D_1h, D_2h).$ To avoid technical complications in ...
user avatar
  • 287k
3 votes
Accepted

Question on solution of Casella and Berger Exercise 9.10: Showing that $Q(t,\theta)$ is a pivot

In an effort to clarify the notation I have arrived at an equivalent demonstration based on the distribution functions rather than the densities. Let's see how this plays out. What I aim to achieve ...
user avatar
  • 287k
1 vote

Question on solution of Casella and Berger Exercise 9.10: Showing that $Q(t,\theta)$ is a pivot

In the first equation, the solution writer applied the change of variable formula to change from a density in $t$ to one in $Y = Q(t; \theta)$, as well as applying the given information about the ...
user avatar
3 votes
Accepted

Find the probability of which sample comes from a "higher" distribution based on random sample from two distributions

I draw random samples from each distribution, which we can call "bucket" here, with random length, and I need to predict which "bucket" form the two has the high distribution, ...
user avatar
  • 266
2 votes

Find the probability of which sample comes from a "higher" distribution based on random sample from two distributions

First, I tried a common bayesian rule, which gives me the probability over each sample (starting from a 50/50 prior), like, sample 1 (0.13 high, 0.87 low), sample 2 (0.85 high, 0.15 low). But that ...
user avatar
0 votes

Statistical methods for different parts of a distribution

I believe what you are looking for here is what is commonly referred to as an ordinal regression. This term is used in my field to refer to a few different types of models that are used when the ...
user avatar
  • 425
2 votes

Estimate parameters of an unknown negative binomial distribution based on known distribution

You can re-parameterise the negative-binomial distribution in terms of its mean $\mu$ and variance $\sigma^2$. To do this, you use the parameter relationships: $$p = \frac{\sigma^2 - \mu}{\sigma^2} \...
user avatar
  • 94.6k
1 vote
Accepted

Infinitesimal Robustness, influence function of $T$ at $F$

You are working in a function space, specifically, the space of probability distributions equipped with some useful topology. In this case, we can choose two probability distributions in that space - ...
user avatar
  • 32.7k
1 vote

Get probability distribution function from density function and calculate the cumulative value

Firstly, your density function does not presently integrate to one --- to get the correct density you need to add an additional multiplicative term $2^{-n/2}$. I will make this adjustment in my ...
user avatar
  • 94.6k
2 votes

How to prove that von mises distribution belongs to exponential family?

It is an exponential family distribution An exponential family distribution with parameter vector $\boldsymbol{\theta}$ is one that has a log-density of the form: $$ \log f(x \mid \boldsymbol{\theta})=...
user avatar
  • 143
2 votes
Accepted

Combinations from different sets with weightings

This can be expressed as a binary linear program. Provided the total number of unique combinations is not prohibitively large, it can be solved efficiently. When multiple solutions exist, usually ...
user avatar
  • 287k
3 votes
Accepted

Determining distribution based on first three even moments

Take a linear combination of $X^2,X^4$ and $X^6$: $$g(X)=aX^6+bX^4+cX^2$$ Choose scalars $a,b,c$ such that $E\left[g(X)\right]=0$ and $g(X)\ge 0$ almost surely. This would imply $g(X)=0$ almost surely ...
user avatar
  • 9,120
4 votes
Accepted

n-th quantile for bivariate variable

The documentation says ...
user avatar
1 vote
Accepted

Describing / fitting a highly skewed distribution

You should consider comparing distributions. One handy Python library to compare distributions to describing empirical data is the powerlaw library. It has a paper that explains how to use it: ...
user avatar
  • 56
2 votes

transformation of a kernel density estimate to uniform distribution

Background As stated in this as well in his prior question the OP wants to perform Bayes quadrature of an expensive function against a density, which is a Gaussian mixture as the result of applying a ...
user avatar
  • 1,663
0 votes

Log-normal mean and standard deviation change after sampling

I did this in R and got very similar results to you, even when I tried sampling from the corresponding normal then exponentiating. The issue here is the scale of the values you're trying to sample. ...
user avatar
3 votes

t- test for non normally distributed sample

For testing a difference between the means of both groups, a t-test (aka "Welch test" to make clear that no assumption of equal variances is made) can even be used when the data of both ...
user avatar
  • 3,006
4 votes

transformation of a kernel density estimate to uniform distribution

The multivariate $d$ dimensional extension of the inverse cdf generation is incorrect, both because $F^{−1}(\cdot)$ does not exist and because $F(X)$ is not Uniform (0,1). (For instance, in the ...
user avatar
  • 91.8k
0 votes

Obtaining a tractable expression for P(X<2Y)

This answer is somewhat in the spirit of the video listed in https://mathematica.stackexchange.com/questions/267065/how-to-visualize-the-circle-triangle-probability-problem. And it is a bit of a ...
user avatar
  • 2,153
1 vote

Obtaining a tractable expression for P(X<2Y)

I think I've found the error in my original method. I'd appreciate feedback on my solution! We will solve this in generality; that is, we will show, as @JimB asserted, that $P(X<kY) = 1/2$ for any ...
user avatar
  • 155

Top 50 recent answers are included