All Questions

Tagged with
Filter by
Sorted by
Tagged with
0
votes
1answer
520 views

How to generate conditioned random variables from a density function?

I want to generate random variables from a distribution function using inverse sampling with the additional condition that the sampling should be conditioned, i.e., random generated variables should ...
7
votes
1answer
576 views

Probability of failure in a finite population

I regularly inspect finite populations for failures (we make custom products in batches of ~500-800). Currently, we inspect every product for failure, which is quite a bit of work. I want to reduce ...
4
votes
2answers
4k views

Sample from distribution given by histogram

Given a histogram obtained using given data points, how do I randomly sample from the distribution predicted by the histogram? Any conceptual comment / R code would be welcome.
0
votes
1answer
243 views

sample of arbitrary length from large list without replacement

I'm interested in sampling a large dataset in nonconsecutive-record sequences of arbitrary length without overlap. I know from How to take many samples of 10 from a large list, without replacement ...
6
votes
1answer
148 views

Generate tail of distribution by a given sample in R

I have a sample of measurements from a real life device which misses all the measurements that are less than some threshold (given device is not precise enough). From theory and also measurements ...
1
vote
0answers
68 views

Compare 2 populations and find subset in one population similar to other

I have 2 different populations who are being offered the same product. I am being asked to find a subset of people in one population (on which I have control to run an experiment) who mimic the other ...
1
vote
1answer
444 views

Central Limit Theorem for R coding assignment

I had a homework assignment asking us to code some basic things in R. Here are the tasks: Generate a Gamma(1,2) population of size N=5000 and show its density plot and descriptive statistics. Create ...
6
votes
1answer
2k views

Problems generating a sample from a custom distribution with log

I'm trying to generate a sample from a family of distributions. In particular I would like to be able to obtain a sample from the survival function: $$1-F(x) = c x^{-a} \log^b(x)$$ with a proper ...
2
votes
2answers
394 views

Sampling method for imbalanced data?

I have an imbalanced dataset with 4995:5 ratio as well as other datasets with less imbalanced ratios. I split this 4995:5 ratio into training and testing for about 2/3 training and 1/3 testing. I also ...
5
votes
1answer
79 views

Testing whether sampling (convex polytope) is uniform

Currently, I am sampling points from: i) a convex polytope (i.e. Ax <= b) ii) a high dimensional simplex The algorithms I am using are hit-and-run and a simple version of Bayesian bootstrap. I ...
2
votes
2answers
819 views

Get quantile function of dynamic mixture model

I have a dynamic mixture distribution fitted to my risk data (i.e., I have all parameters) of Weibull and Generalized Pareto, with a Cauchy CDF mixing function, that can be written as: \begin{align} ...
7
votes
2answers
487 views

How to test if social structure is non-random and resulting from genetic relatedness – and how to deal with demography effect

I am trying to construct (undirected) social network based on co-occurence of individuals. Clustering algorithm will be later applied on this network to find some distinct subgroups. Issue is that ...
8
votes
3answers
3k views

Using Uniform Distribution to Generate Correlated Random Samples in R

[On recent questions I was looking into generating random vectors in R, and I wanted to share that "research" as an independent Q&A on a specific point.] Generating random data with correlation ...
0
votes
2answers
2k views

Systematic Sampling with unequal probabilities

I found the library Sampling in R to conduct a systematic random sampling with unequal probabilities. I got my sample but I really don't know how that sample is selected. Is there anyone that can ...
4
votes
1answer
335 views

understanding proposal distribution sequential importance sampling in R

From the article of wikipedia http://en.wikipedia.org/wiki/Particle_filter I see that one generate samples from the proposal $\pi(x_k^{(L)}\vert x_{o:k-1}^{(L)},y_{1:k})$, however, the role of $y_{1:...
6
votes
1answer
1k views

Difference between calculated inclusion probability and what is returned by sampling function?

I have a (small) population from which I wish to sample. I assign probabilities proportional to $y$. I enumerate the possible samples and then determine the probability of each sample occurring based ...
7
votes
2answers
683 views

How to simulate effectiveness of treatment in R?

Let's say I want to write a simulation for the table below to decide if Xylitol treatment and ear infections are independent. How would I go about doing this?
9
votes
2answers
1k views

Inverse CDF sampling for a mixed distribution

The out-of-context short version Let $y$ be a random variable with CDF $$ F(\cdot) \equiv \cases{\theta & y = 0 \\ \theta + (1-\theta) \times \text{CDF}_{\text{log-normal}}(\cdot; \mu, \sigma) &...
4
votes
1answer
2k views

techniques for sampling graphs? (possibly implemented in r packages)

Let's say I have a very large graph that proves impractical for visualization ends and I wanted to sample a random subgraph. (I know that I can filter out a subgraph via measures like degree, ...
0
votes
1answer
198 views

sum of correlated random sample

Suppose I have 1000 draws each of two random variables X and Y. If I wanted to sample the sum of these variables, I would simply calculate 1000 samples, i.e. $$ S_{i}=X_{i}+Y_{i}, i=1,2,…,1000 $$ ...
2
votes
4answers
3k views

smart sampling techniques in r

I have a large data set of about 1.8 million rows with 80 variables. I would like to find a good technique (code or package) in R that can reduce the amount of training data without damaging the ...
3
votes
1answer
745 views

Has anyone publicly shared an implementation of RUSBoost in R?

There's no package available on CRAN, so I was hoping someone in the community had written their own function/package. I see it's been done in MATLAB, so I may just have to start with that and write ...
1
vote
1answer
708 views

choosing one pair randomly from repeated pairs

I have two vectors, in my case all states in the US, and another vector, which indicates the states that it borders. States in vector 1 repeat for each state it borders. For instance, there would be 3 ...
3
votes
4answers
1k views

How exactly does null hypothesis testing work

I'm trying to gain a better understanding of how null hypothesis testing works. I have 3 questions related to the code below: Am I right in saying the probability of each t value in ...
2
votes
1answer
5k views

Leave-one-subject-out cross validation in Caret

Hi Dear Colleagues, I wonder how to correctly setup a leave-one-subject-out cross validation (LOSO) for train() function in caret. Here is my example code: ...
2
votes
4answers
5k views

Sampling data to have specific mean and standard deviation [duplicate]

I have a data that I want to sample such the resultant distribution of values should have specified mean and standard deviation. I can think of rejection sampling to achieve this however that seems to ...
0
votes
0answers
134 views

What is the sampling design and how to declare it in the survey()

I have a sample design that can be resumed in: Split the city in X areas and, for each area, sample 5% of the residences and get some informations about it; For each residence, interview ALL ...
4
votes
1answer
4k views

Importance Sampling to evaluate integral in R

I have asked the question here also. However, there might be something wrong with my theoretical understanding hence I'm asking here as it is more relevant. Kindly do not diss without looking first. ...
1
vote
0answers
243 views

Importance Sampling MC - a couple of questions regarding PDF

I implore the good people to quickly glance through this thread over at StackOverflow to get a better idea of my question if the following isn't clear. I have these integrals to evaluate using ...
14
votes
1answer
7k views

How to draw random samples from a non-parametric estimated distribution?

I have a sample of 100 points which are continuous and one-dimensional. I estimated its non-parametric density using kernel methods. How can I draw random samples from this estimated distribution?
5
votes
1answer
737 views

How to calculate the chance of getting completely unbalanced groups? (with R)

I want to simulate or calculate probabilities of combinations of group membership for different sample sizes (e.g., n= 3, 4, 5, 10, or 100) for two groups (of the same sample size). Each outcome could ...
0
votes
0answers
71 views

Is this a valid test?

I have 5000 cases vs 5000 controls with positive/negative outcome. chi.test shows no significance for this data. I get significance when subsetting cases on VarX, ...
0
votes
0answers
92 views

Estimating the total number of sample subsets of size 10 from a population of size 1000

If we have $n > 1000$ units to sample from, how can we estimate the number of subsets of a certain size, since the number is too much for R to handle? What I ...
-1
votes
1answer
666 views

How do I randomize a larger population, from an existing population in R?

I have a small 20k (176 variables) test dataset. I want to create a random bigger dataset for testing, using the rows (data) and columns (names) from the small dataset. Currently I have: ...
1
vote
0answers
68 views

Identifying potential for missingness in a seasonal time series to bias period-averages

I have high-frequency time series (observations every few minutes) for which I wish to compute daily averages. The data exhibit a strong diel cycle. Sometimes observations are missing in the time ...
2
votes
1answer
184 views

Why do two equivalent approaches to selecting numbers that sum to 100 produce different results

This question is inspired by two seemingly equivalent attempts to solve a question asked yesterday: In R, how to sample from the output of combn(a,b) if the "a choose b" is too large? ...
3
votes
0answers
83 views

How to testing the probability of two factors being found together?

I know how many times a particular factors has been found: values<-cbind(f=c("S1","S10","S3","S4","S5","S6","S7","S8","S9"),count=c(9,8,4,24,20,4,8,21,5)) I ...
3
votes
3answers
2k views

In R, how to sample from the output of combn(a,b) if the “a choose b” is too large?

I would like to sample 1 million vectors from the set of vectors $$\mathbf{v} = \{v_1,v_2,...,v_n \} $$ subject to $\sum_{i=1}^n v_i = 1$ $\forall i: v_i \ge 0$ each $v_i$ has at most 2 decimal ...
4
votes
1answer
486 views

Correct variance estimation for random sample post-stratification weights

I'm dealing with a random sample and I have a question about its poststratification weights. Let's assume that: this is a random sample of addresses, but post-stratification weights have been ...
2
votes
1answer
14k views

Understanding F-test to compare variances (R's var.test)

I gave run the following codes in R console: ...
2
votes
1answer
3k views

The sample size applied to a non-normal distribution

I have a single variable that represents my population values (sample of data): ...
13
votes
3answers
11k views

Cluster Big Data in R and Is Sampling Relevant?

I'm new to data science and have a problem finding clusters in a data set with 200,000 rows and 50 columns in R. Since the data have both numeric and nominal variables, methods like K-means which ...
3
votes
2answers
2k views

What is behind JAGS (Just Another Gibbs Sampler)?

I have been using JAGS but I am not quite sure how it actually simulates it values. I need to know in a general sense what's going on in the background.
0
votes
1answer
1k views

R: Density estimation vs Histogram Estimation

To draw random samples from a custom distribution, I recall reading that KDE's are better than histograms. (See hadley's comment here.) When I experimented in R, I am finding that the KDE method ...
3
votes
3answers
2k views

Downsampling excessively sampled curves

I'm looking for a quick (event if not so accurate) way to deal with over-sampled curves using R. Consider the following example in which x contains 1000 values in ...
2
votes
1answer
506 views

Tests for quality of permutation (Y-randomization)

Can you please suggest the metrics/tests for accessing the quality of Y-randomization (permutation)? I have some doubts in quality of Y-randomization that I am using with sample.int from R. For 100 Y-...
11
votes
2answers
6k views

Sampling with replacement in R randomForest

The randomForest implementation does not allow sampling beyond the number of observations, even when sampling with replacement. Why is this? Works fine: ...
5
votes
1answer
1k views

Regarding the sampling procedure in Adaboost algorithm

The AdaBoost algorithm states that it is to train a classifier based on the training data according to a weight vector. Assume the size of training data is N, the weight vector is of dimension N as ...
1
vote
1answer
185 views

Sampling method for constructing multiple groups from a given pool

There is a set A having $N$ elements. Based on set A, I would like to construct $M$ groups, where each group has $P$ elements. Naturally, $M P <N$. All of those elements in the $M$ groups are ...
6
votes
1answer
15k views

mtry and unbalanced use of predictor variables in Random Forest

I am working on the Random Forest prediction, with the focus on the importance of predictor variables, and have a question regarding understanding of mtry and the actual usage of variables in the ...