New answers tagged sampling
0
votes
Mini-Batch Gradient Descent - Why does sampling with replacement work?
It works (and we don’t care about sampling points multiple times) because it’s an unbiased estimator of the full gradient.
Gradient distributes over summation (and expectation). The expected value of ...
- 6,787
0
votes
What distribution to sample X from to get an uniform distribution in Y?
There is no need for absolute values around your sine function. f(x) = sin(x) is a perfectly fine pdf on the sample space [0, pi/2]. As schotti points out, you can create an RV with this pdf by taking ...
- 31
1
vote
Formal definition of "sampling intensity" in terms of probability theory
The context for this definition is sampling of a population, which is modelled by considering some population values $X_1,...,X_N$ where $N$ is the population size. In the model-based approach to ...
- 94.4k
3
votes
Logistic regression simulation with respect to event occurrence (prevalence)
You have an array of explanatory variables $(x_1, x_2, \ldots, x_n)$ ($n=20000$) and a model that assigns a probability to each $x_i.$ You seek a subarray of these variables that has a mean ...
- 287k
0
votes
Logistic regression simulation with respect to event occurrence (prevalence)
Its going to be hard to simulate a with an exact proportion of 1s, but if you can get pretty close if you simulate a lot of data.
Key thing to realize is that the intercept is the log odds of the ...
- 25.8k
0
votes
Choosing an equal number of samples from each strata - what is this called?
This would be a type of "disproportional sampling". In particular it would be "equal allocation".
I've never tried it, but from this Stack Overflow question, https://stackoverflow....
- 1,477
1
vote
What distribution to sample X from to get an uniform distribution in Y?
The question doesn't require a specific programming language, which is fine, but I noted that the OP's plot looks like the default style of matplotlib. @jbowman has given a useful r implementation. ...
- 3,402
7
votes
Can I take a random sample of my very large data set to overcome non-independence?
I will add a couple of points to Tim's answer, focussing on the original question, which was "My question is - is this valid? Am I missing anything here?". I think the approach can be valid, ...
- 411
12
votes
What distribution to sample X from to get an uniform distribution in Y?
I suspect the difficulty you are having is in the generation of $x$ from $f(x) \propto |\sin(x)|$. I have coded a very simple acceptance-rejection random number generator in R that will do the job:
<...
- 32.7k
13
votes
What distribution to sample X from to get an uniform distribution in Y?
maybe i misunderstand your question, but why don't you sample from a uniform distribution and set X to the arccos of your samples?
in R, this would be
...
- 391
2
votes
Why is random sampling good?
A non-random sample may be good for a particular purpose, or it may be bad. A random sample can be shown with high probability to be "good" for many purposes.
In particular, in statistics, ...
- 610
3
votes
Why is random sampling good?
The Central Limit Theorem may be the theory you're looking for. It shows that random sample means follow a Normal distribution (even if the population isn't Normally distributed) and that allows us ...
- 139
17
votes
Why is random sampling good?
You seem to be conflating the idea of random sampling with the separate question of whether objects are sampled with our without replacement. The first method you describe is a simple-random-sample ...
- 94.4k
1
vote
Markov Chain Monte Carlo with known normalisation
Another way of looking at the issue of approximating$$\mathfrak I = \sum_{x\in\mathfrak X} p(x)O(x)$$by stochastic techniques is to aim at adding primarily large values of $p(x)O(x)$. Assuming no ...
- 91.8k
4
votes
Accepted
Can I take a random sample of my very large data set to overcome non-independence?
If you downsample time-series data it would not remove the dependence, it would just dilute it. Say that your data follows the relationship
$$
y_{t+1} = f(y_{t}) + \varepsilon_{t+1}
$$
so the current ...
Tim♦
- 113k
4
votes
Accepted
How to sample from a custom heavy tailed (e.g. custom Cauchy) distribution?
Because $Z$ is symmetric around $0$ and $|Z|$ has a generalized beta prime distribution, there is a simple algorithm to obtain random values of $Z$ efficiently:
Step 1: Draw a random value $Y$ from a ...
- 287k
0
votes
Log-normal mean and standard deviation change after sampling
I did this in R and got very similar results to you, even when I tried sampling from the corresponding normal then exponentiating.
The issue here is the scale of the values you're trying to sample. ...
- 131
1
vote
Accepted
Convert from log-normal distribution to normal distribution
If $X \sim \mathrm{N}(\mu, \sigma^2)$ and $Y=e^X$, then $Y \sim \mathrm{Lognormal}(\mu, \sigma^2)$. The random variable $Y$ has mean $m=\mathbb{E}(Y)=\exp(\mu+\sigma^2/2)$ and variance $v=\mathrm{Var}(...
- 131
1
vote
Accepted
Showing that $E[\hat{\tau}_D] = P(n_D > 0)\tau_D$ and $\vert E[\hat{\tau}_D] - \tau_D\vert \leq \tau_D(1-\frac{N_D}{N})^n$
For (ii) I believe the idea is to get the upper estimate going from the usual Simple Random Sample to one with replacement where the inclusion events are i.i.d.
Is the sample $S$ drawn with or ...
- 131
3
votes
uniformly sample from gaussian distributed data
It doesn't matter what the distribution of the data might be. You ask to create samples in which only the data values appear, yet are as uniformly distributed as possible.
Remove any data not in the ...
- 287k
1
vote
Inverse transform sampling : comparing bias, variance and mse for an estimator
Basically, what you are trying to do is evaluate the MSE : $\mathbb{E}(\hat{\theta} - \theta)^2$ ,through Monte-Carlo simulations. That is, you evaluate the expectation (an integral here) through a ...
- 46
0
votes
How to calculate sampling error for proportionate sampling?
I need a bit more data to help you with the sampling error, but just know its the same as standard deviation, but replace p with p^. That accounts for your standard error.
"Also if I can use some ...
- 26
1
vote
How to sample and compute the likelihood from a Mollified Uniform distribution?
This works.
Another way to view what you're doing is as
$$
\mu \sim U[0, 1]
\quad
\delta \sim \mathcal N(0, \sigma)
\quad
X = \mu + \delta
.$$
The density of the sum of two variables is the ...
- 22.2k
Top 50 recent answers are included