Skip to main content

New answers tagged

1 vote

Is considering a specific distribution necessary before computing an average?

The mean (average) is a descriptive statistic, just like the variance (standard deviation), skewness, or the coefficients of a linear regression, etc. They are just the mathematical results of ...
jginestet's user avatar
  • 1,849
0 votes

Parametrizing the Behrens–Fisher distributions

The fiducial approach by Fisher is to consider probability statements for an ancillary statistic $t$ such as $$P(t-\theta > a)$$ and invert it like $$P(\theta < t - a)$$ such that we have a ...
Sextus Empiricus's user avatar
1 vote

Is considering a specific distribution necessary before computing an average?

You do not need to know the distribution of the average to use it. Suppose the observations are assumed to be independent and identically distributed. In that case, the estimated expected value of the ...
Xiaochuan Lu's user avatar
0 votes

How to show that many functions (a hundred, a thousand) have the same shape an distribution of values over an interval?

Based on your description 0.4 +ve derivative from zero to 0.4 and around zero or slightly negative derivative up to 1. You could fit $y = a_0 + a_1 x + a_2 (x-0.4)^+$, which is a piecewise linear ...
seanv507's user avatar
  • 7,268
0 votes
Accepted

How to show that many functions (a hundred, a thousand) have the same shape an distribution of values over an interval?

A Q-Q plot approach would not be appropriate for your objective. Not only that, it would give you misleading results. This is because a Q-Q plot changes the order of the data, arranging it in ...
jginestet's user avatar
  • 1,849
-1 votes

What model for continuous data with excess zeros?

I would formulate a continuous "underlying" model that is scientifically justifiable and then "censor" it to allow certain degree of discreteness to describe the data collected.
Zack Fisher's user avatar
4 votes

How to show that many functions (a hundred, a thousand) have the same shape an distribution of values over an interval?

To show that several functions are more or less the same you could just superimpose them graphically. I don't think quantile plots of any flavour are directly relevant or likely to be helpful. The ...
Nick Cox's user avatar
  • 58.6k
1 vote

How can I find the limiting distribution of $Z_n=\sqrt{n}\frac{X_1X_2+X_3X_4+\cdots+X_{2n-1}X_{2n}}{X_1^2+\cdots+X_{2n}^2}$?

Let $X_1, X_2, \cdots$ be i.i.d. random variables with $E(X_i) = 0$, $\text{Var}(X_i) = 1$, and $E(X_i^4) < \infty$. We are tasked with finding the limiting distribution of $$ Z_n = \sqrt{n} \frac{...
Robert Long's user avatar
7 votes

Estimating mean and SD given the median and IQR values

The long answer is "no" this is not possible to determine without additional information about the distribution of the data. The pragmatic answer is "yes", if you make assumptions ...
Gregg H's user avatar
  • 5,624
0 votes
Accepted

Exercise on finding probability density function

I found a solution for my question (thanks to @Ben's answer). In the figure above I need to find $P(U \le u) = P(Y_1/Y_2 \le u)$. So, I need to find area above the line $u = Y_1/Y_2$. What I found $...
k1r1t0's user avatar
  • 131
8 votes

Can a non-symmetrical distribution have the same areas under the PDF in the two sides around the mean?

A nice example is that any Binomial distribution with integer mean has median equal to its mean (this is hard to prove in general, but easy in any specific example).
Thomas Lumley's user avatar
3 votes

Exercise on finding probability density function

It is simpler to do this with a single integral. For all $u>0$ we can write the distribution function as: $$\begin{align} F_U(u) &\equiv \mathbb{P}(U \leqslant u) \\[12pt] &= \mathbb{P}(...
Ben's user avatar
  • 129k
8 votes
Accepted

Can a non-symmetrical distribution have the same areas under the PDF in the two sides around the mean?

Your title question is identical to asking "Can an asymmetric distribution have mean equal to median" to which the answer is "yes" and many examples are to be found on site already....
Glen_b's user avatar
  • 286k
0 votes

What statistic best estimates the sample mean in case of missing data in a distribution?

Here is an answer assuming that each measured value is reasonably accurate, but that the measurements are incomplete due to overlaps: you might get the median particle length as the $6^{th}$ or $13^{...
Matt F.'s user avatar
  • 5,342
0 votes

Distribution of a rate in a subsample after permutation of the total dataset

The problem involves estimating the distribution of the rate $ p_{sub} $ of positives in a subsample of size $ n $ after permuting a dataset $ D $ of length $ N $ that contains $ pN $ positives and $(...
Lynchian's user avatar
  • 305
1 vote

Calculating the joint pdf of linearly dependent random variables $X$ and $Y=X$

As already explained in comments, there is no joint density in the plane, because all the probability mass of $(X, X)$ is concentrated on the diagonal $y=x$. There is a density on that diagonal, but ...
kjetil b halvorsen's user avatar
1 vote

What statistic best estimates the sample mean in case of missing data in a distribution?

Here is an answer assuming that the only sample statistics which can be calculated reliably are the size $k_i$ and the maximum $M_i$ of the sample. The question is how to estimate the average $\mu_i$ ...
Matt F.'s user avatar
  • 5,342
0 votes

Difference between a Student-T vs Cauchy distribution

When the degrees of freedom is one, the Student's t-distribution becomes the Cauchy distribution. As the degrees of freedom increase to infinity, the Student's t-distribution becomes the normal ...
Bleezy's user avatar
  • 1
0 votes

How do I measure the regularity of the distribution in a list of binary data?

Uniformly Spaced $1$s What @whuber suggests seems to match closest your examples of regular and irregular sequences: [1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1] is ...
Frans Rodenburg's user avatar
1 vote

How to fix intersection of cluster distributions in R

I don't see why you are using cluster analysis (CA). CA is unsupervised learning. Its goal is to find ways that observations "go together" or cluster, with no dependent variable. So, it is ...
Peter Flom's user avatar
  • 125k
1 vote

Unit-Root Asymptotics

It would appear (but check), that your case is, or is close to, Hamilton's book, Case 3, pp. 495-497, although he does not use the differenced dependent variable (his specification is his eq. 17.4.14)....
Alecos Papadopoulos's user avatar
5 votes

Equivalence of inverse transformations under distributional equivalence

Since the unit standard multivariate normal $\mathcal{N}(0, I)$ is rotationally symmetric, let $R$ be any orthogonal matrix, that is, $R^TR = R R^T =I$, then also $RY \sim \mathcal{N}(0, I)$. Since $g$...
kjetil b halvorsen's user avatar
0 votes

How can I show statistically that one of my replicates is likely contaminated?

I personally do not think that there is such a thing as an "outlier"; there is either erroneous data (measurement error, transcription error, etc.), or then there is data. It may be ...
jginestet's user avatar
  • 1,849
8 votes

Vintage of this lower bound on skewness for positive data with given mean and sd?

$\newcommand{\s}{\sigma} \newcommand{\a}{\mu}$ I agree that this could have been known a century ago. In any case it has a quick proof, following my answer to one of the linked questions. Let the ...
Matt F.'s user avatar
  • 5,342
2 votes

What does "Aleatoric and Epistemic uncertainties" mean?

Aleatoric Uncertainty: This is the uncertainty of the process which you are trying to model. Say, you want to train a model with some sensor output where the sensor is itself producing some random ...
hafiz031's user avatar
  • 235
1 vote

Can positive values with sd > mean have skewness = 0?

There is in fact a generic lower bound on skewness of strictly positive data: $$ g_1 > \frac\sigma\mu - \frac\mu\sigma. $$ This shows nicely why you are running into difficulties: the coefficient ...
David C. Norris's user avatar
0 votes

Under what conditions are there pairwise monotonic relationships between mean, variance, and (positive) skewness of a lower-bounded distribution?

It turns out there is a lower bound on the skewness of any strictly positive data set having given mean and sd: $$ g_1 > \sigma/\mu - \mu/\sigma. $$ This doesn't seem entirely consistent with your ...
David C. Norris's user avatar
1 vote

What is meant by the probability of a sample having a value of $x$ is $ng(x)$?

The formula relates to continuous distributions. For continuous distributions it makes no sense to speak about "the probability of one sample having a value of $x$" because the probability ...
Sextus Empiricus's user avatar
0 votes

Estimation of population parameters on the basis of multiple samples regressions

Besides the options proposed by Frans Rodenburg (+1) you could also consider to use gee for gamma distributed data. Such model will run much faster than a random intercept model estimated with lmer. ...
BenP's user avatar
  • 1,838
8 votes
Accepted

What is meant by the probability of a sample having a value of $x$ is $ng(x)$?

With a generous interpretation we can make sense of this quasi-argument. Consider an iid sequence of absolutely continuous random variables $\mathbf X = X_1,X_2,\ldots, X_n$ with density $g$ and ...
whuber's user avatar
  • 328k
2 votes

What is meant by the probability of a sample having a value of $x$ is $ng(x)$?

Looks like the Wikipedia page is a little (very?) loose with definitions. Given the integral in the section from which you pulled the quote, $g(x)$ is almost certainly referring to a probability ...
ischmidt20's user avatar
1 vote

Estimation of population parameters on the basis of multiple samples regressions

Not quite an answer to the question you pose, but I don't think this is the right approach. You write glmer in your question, which makes me think you're using the <...
Frans Rodenburg's user avatar
2 votes

How to work out the expected rate of success when there is a guaranteed success on the nth attempt?

A simple question, stated in an elementary fashion, deserves a short elementary solution. It also deserves some consideration of what it is actually asking for. This post offers both. To establish ...
whuber's user avatar
  • 328k
2 votes

How to work out the expected rate of success when there is a guaranteed success on the nth attempt?

You could simulate it as follows: Perform $n$ Bernoulli trials with probability of success $p$; If any trials were successful, note the first success; Otherwise, note the maximum, $n$; Compute the ...
Frans Rodenburg's user avatar
0 votes

How to work out the expected rate of success when there is a guaranteed success on the nth attempt?

I wonder whether this could be modelled via Hypergeometric distribution. Lets say you have $M=\left(N-1\right)+A$ boxes. And out of them $A$ boxes have a prize. So choosing the boxes randomly you will ...
Cryo's user avatar
  • 783
2 votes

(THEORY) Do Tree models output probabilities?

I think this can be made simpler. Single independent variable, and classification. In this case, I would argue, tree-based model is not that dissimilar from learning the distribution of your binary ...
Cryo's user avatar
  • 783
9 votes

(THEORY) Do Tree models output probabilities?

Your assessment of the situation is excellent. I would just add that in my practice random forests suffer some of the worst miscalibration that I’ve ever witnessed as a statistician. Even a single ...
Frank Harrell's user avatar
-1 votes

Posterior expectation of normal distribution with "truncated" observation

\begin{gather*} E[Y\mid X, Y\in A]=\frac{E[YI_A(Y)\mid X]}{E[I_A(Y)\mid X]}=\frac{\int_A yf_{Y|X}(y|x)dy}{\int_A f_{Y|X}(y|x)dy}=\frac{\int_A\int_R yf(_{\theta}, x,y)d_{\theta}dy}{\int_A\int_R f(_{\...
Speltzu's user avatar
  • 336
5 votes
Accepted

Posterior expectation of normal distribution with "truncated" observation

Since $Y\sim\mathcal N(\theta,\tau_Y^{-1})$, the indicator variable $Z=\mathbb I_A(Y)$ has a binomial distribution $$Z\sim\mathcal B(\mathbb P_\theta(Y\in A))$$ Since $X$ and $Y$ are independent, the ...
Xi'an's user avatar
  • 107k
1 vote

Posterior expectation of normal distribution with "truncated" observation

$$ \begin{aligned} &\mathbb{E}[\theta\mid X=x,Y\in A] \\ &=\int_{y\in A}\mathbb{E}[\theta\mid X=x,Y=y]\, p_Y(y\mid X=x, Y\in A)\, dy\\ &= \int_{y\in A}[\gamma_X x + (1-\gamma_X)y]\frac{\...
Johan de Aguas's user avatar
2 votes

Approximation function for MLP and LSTM

Based on your answer to Sycorax I infer that you want to look at the distribution of residuals and compare the two empirical distributions of the residuals (MLP vs LSTM). For comparison you could look ...
Ggjj11's user avatar
  • 1,543

Top 50 recent answers are included