Resampling is taking a sample from a sample. Common uses are jackknifing (taking a subsample, eg all values but 1) & bootstrapping (sampling w/ replacement). These techniques can provide a robust estimate of a sampling distribution when it would be difficult or impossible to derive analytically.

learn more… | top users | synonyms

0
votes
1answer
28 views

parametric bootstrap for low sample sizes

I believe that this question is sufficiently different from previous related ones to warrant a new post. (I apologize if it has been answered already) I need to decide between various resampling ...
0
votes
0answers
19 views

Finding the fewest needed samples for a regression method

I have a regression method that I have applied with success to tens of thousands of well-observed objects. I'm looking to estimate how well it works on poorly observed objects. I selected 50 objects ...
7
votes
1answer
164 views

Is bootstrapping appropriate for this continuous data?

I'm a complete newbie :) I'm doing a study with a sample size of 10,000 from a population of about 745,000. Each sample represents a "percentage similarity". The great majority of the samples are ...
0
votes
0answers
15 views

How to improve results when using sampling in skewed binary classification?

I am using a data set with 18 features with True/False output (Related to mobile ad targeting). True values occurs only 0.4 % of the time. So, I have used sampling to keep the ratio of True and False ...
2
votes
0answers
58 views

Estimating means of correlated distributions with long tails

Suppose I have a relatively large number of samples (~1k) drawn from a series (~40) of increasingly long-tailed distributions (going from approximately normal to approximately log-normal). I want to ...
10
votes
2answers
415 views

resampling / simulation methods: monte carlo, bootstrapping, jackknifing, cross-validation, randomization tests, and permutation tests

I am trying to understand difference between different resampling methods (Monte Carlo simulation, parametric bootstrapping, non-parametric bootstrapping, jackknifing, cross-validation, randomization ...
1
vote
0answers
20 views

Oversampling with categorical variables

I would like to perform a combination of oversampling and undersampling in order to balance my dataset with roughly 4000 customers divided into two groups, where one of the groups have a proportion of ...
2
votes
1answer
34 views

Convenience sampling - Distribution forcing?

I am conducting some experiments on a data set that was collected by convenience. It is a data set based on historical data, most of which is not digitized. I know the exact distribution of the ...
1
vote
0answers
11 views

Correcting for multiple comparisons using simulated distribution

To check if unilateral pairs (defined below*) are coordinated (i.e. move together) in a flock of N coordinated individuals, we generate hypothetical (null) distribution of a certain focal parameter of ...
0
votes
0answers
40 views

Determining characteristics of sampling sets for EFA/CFA/SEM

Dividing sample data into several sets seems to be a common approach in statistics. This is especially evident in predictive modeling, where samples are traditionally divided into two sets, usually ...
0
votes
0answers
21 views

What's the safest way to resample to ensure equal class frequencies in training data?

I am working on a number of EEG data sets for binary classification. A good example of one is publicly available here. If you look at what Matthias Kaper did to classify that set, one thing that ...
4
votes
1answer
104 views

Why not always use bootstrap CIs?

I was wondering how bootstrap CIs (and BCa in barticular) perform on normally-distributed data. There seems to be lots of work examining their performance on various types of distributions, but could ...
1
vote
1answer
71 views

Logistic regression and discrepant sample sizes between 0/1 groups

I currently working in a multivariate logistic model but I have a problem regarding the sample size of my observations: -The "success" (1) event group has a sample size of 249 distinct observations - ...
1
vote
0answers
22 views

To find variance and covariance for a double sampling problem

A simple random sample of size $n=n_1 + n_2$ is drawn without replacement from a finite population of size $N$. Further a simple random sample of size $n_1$ is drawn without replacement from the first ...
2
votes
1answer
17 views

Doing comparisons from two null distributions

I have a situation where I have an observed statistic computed from data, and then I have approximated the null distribution by some sort of resampling. I have used this to calculate p-values for a ...
0
votes
0answers
18 views

Blocked Weighted Bootstrap

Bootstrap is a well known resampling method. But I want to know what is blocked weighted bootstrap sampling? Why we need this?
2
votes
0answers
34 views

What is the correct way of generating a p-value for a correlation with resampling?

I have a vector of gene expression values, across 20 patients. Each patient also has a glucose measure (a continuous numeric vector). I want to find how significant the real correlation value is ...
2
votes
0answers
20 views

Using Resampling to understand a large table

I have a data set that is very large. The attributes (columns) are several thousand. Some are sparse others are not. Some are ordinal, others interval, nominal or ratio. The row size is 10s of ...
1
vote
1answer
71 views

Addressing Non-response in a Convenience Sample

I am studying customer satisfaction in a large hierarchical organization. I plan to administer a voluntary survey to customers across the organization, and need to address non-response in my analysis. ...
0
votes
0answers
18 views

Estimating variance of prediction error in bootstrapped training sets with clustered data

I have C clusters with m elements each. I split the C clusters into a large training set D and a test set T. Hence, each element in D and T has m related elements, so its a cluster. I want to ...
2
votes
1answer
275 views

Optimal sampling strategy for EFA, CFA and SEM

I'm wondering what should be the optimal sampling strategy for my dissertation research. I have four data sources (two open source software projects meta-repositories and two global startup ...
0
votes
0answers
37 views

Analysis of forest inventory data - non-random samples

I apologise, this isn't a single question. It is more of a general problem on which I am working and am seeking guidance for how to proceed. I have been provided with an inventory dataset of plot ...
0
votes
1answer
90 views

What are RMSE SD and Rsquared SD metrics in resampling results using R package:caret?

I've been doing predictive modelling with R package caret. When resampling regression models, I get the traditional RMSE and Rsquared metrics, but also RMSE SD and ...
0
votes
0answers
32 views

Bootstrapping data with only sampling weights given

Suppose you only have these information from a sample data: $X_i$ and $w_i$, $i=1,...,N$, where $w_i$'s are the respective sampling weights(not integers). Is it possible to obtain a valid bootstrap ...
2
votes
0answers
46 views

Bias and variance estimation with boostrap

The Wikipedia article about Jacknife estimation of the bias and variance of an estimator $\theta$ includes the following formulas: Variance of $\theta$: $ \operatorname {Var}(\theta )=\sigma ...
2
votes
0answers
51 views

Statistical thought experiment (possibly Bayesian) about survey sampling and propensity scores

For some practical application I recently came about the following thought experiment. Can anybody help? Suppose we administer a survey A to measure a variable $Y$. The response probabilities may ...
1
vote
0answers
34 views

Bonferroni correction: control vs. groups?

I'm trying to understand how to set up a Bonferroni correction on several different groups and compare it to the control group. The groups and observations are as follows (with Group 0 being the ...
0
votes
2answers
59 views

Advice on resampling scheme for a small sample/ costly computation situation?

I am testing combinations of preprocessing activities on a small data set (n=48, p=30). The script generates 3200 different versions of the original data and measures how they perform in a ...
4
votes
1answer
105 views

Can Bootstrap Resampling be used to Calculate a Confidence Interval for the Variance of a Data Set?

I know that if you re-sample from a data set many times and calculate the mean each time, these means will follow a normal distribution (by the CLT). Thus, you can calculate a confidence interval on ...
2
votes
1answer
299 views

Bootstrapped p-value

I have a p-value that I generate via resampling. Resamples = 5000 Positive findings = 1000 positive findings P-value = 1000/5000 = 0.2 How can I compute the 95% confidence interval for this ...
0
votes
1answer
58 views

Selecting uncorrelated samples from a set of bulk data that contains correlated and dependent samples

i have a set of data that is generated by expensive computational model evaluations, on a total data set of 10000 samples in 40 dimensions. This sample data set is composed of different data sets, ...
2
votes
0answers
132 views

Sampling and resampling data in R

My problem this time concerns sampling size-related errors, resample-based confidence intervals and a possible way to control for this error. My dataset consists of 50 measurements of certain cranial ...
1
vote
0answers
71 views

Sufficient statistics and parametric bootstrapping

Does the resampling step of the bootstrap method require to have the sample entirely, or a sufficient statistic suffices? In general, the bootstrapping can be nonparametric or parametric. In the ...
3
votes
0answers
114 views

Bootstrapping fits to a small sample

I have a sample of experimentally measured survival times that are quite noisy and vary stochastically. The survival probability of these events (number of events with a survival time of t or more) is ...
2
votes
1answer
283 views

Required number of permutations for a permutation-based p-value

I need to calculate a permutation-based p-value with significance level $\alpha$, how many permutations do I need ? From the article "Permutation Tests for Studying Classifier Performance", page 5 : ...
0
votes
0answers
46 views

time series (asynchronous/variable) sample rate conversion

I am having a problen in preparing my data. The data is a GPS track recorded from a lengthy car-trip. This is recorded with a constant frequency - I have a datapoint every 200ms. Now I have ...
0
votes
1answer
49 views

How small can a subsample in bagging be before performance degrades severely?

If I want to perform bagging, would subsamples with sizes of 0.1% of the actual data be appropriate? The reason I want to do so is because my actual data set is very large in the tens of millions.
4
votes
1answer
162 views

Bootstrapping in R using the boot {boot} and Boot {car}

I'm trying my hand at resampling techniques with a dataset I have, and I think either I'm missing a conceptual point with bootstrapping, or I'm doing something incorrectly in ...
2
votes
2answers
138 views

Multiple resampling test/train dataset when choosing new models?

I have been reading several posts on testing multiple models on the same dataset, which can lead to problems controling type-1 errors. Mostly these posts have to do with data-mining on big datasets: ...
0
votes
1answer
50 views

Calculating the probability of 31 of 628 items are sampled (no replacement) more than 10x amongst 150 participants drawing 50 items each

I am sorry for having to ask a simple probability question, but I have been thinking about it for weeks and an extensive google search has given no answers. I have a group of 628 questions. 31 ...
7
votes
1answer
142 views

What method is simulating pvalues from re sampling from the data

A while back I asked a question about correlating times between time stamps and received a response from Peter Ellis that said I could calculate mean distances between codes... This already will ...
6
votes
1answer
149 views

Restricting minimum subgroup size in a bootstrap resampling study - why is this approach wrong?

I'm currently doing a simple re-sampling study where I compare different methods for generating the confidence interval for linear regression models. I'm trying to follow Burton et. al's (2006) ...
2
votes
1answer
72 views

Suitable method to estimate confidence intervals for an extreme order statistic

I have a sampling distribution generated by computing the maximum across many samples. I'd now like to generate an estimate for what the true maximum parameter is within the population I sampled from. ...
0
votes
0answers
50 views

How to resample samples according to a known distribution

I am encountering the following problem: I have been given a number of samples, and my task is to select those from the distribution that we are interested. Suppose the distribution that I am ...
0
votes
0answers
146 views

Is the p value in a permutation test the same as the p value in the original test?

This is an extension from my last post. It seems to be too long to discuss there. When testing some null $H$ versus alternative $K$ by a test statistic $U(X)$, the p-value for $U$ on a sample $X$ ...
1
vote
0answers
56 views

How to bootstrap respecting subject-level information? [closed]

This is my first post and I am not a skilled programmer, so please let me know if the question or code are unclear. I am trying to bootstrap an interaction (that is my test statistic) using the ...
1
vote
0answers
48 views

Resampling and multiple comparisons

I hope someone can help me with the following problem (please have in mind that I am not a statistician). I run an experiment with 4 treatments where I measured the variable ā€œPā€, which can be either ...
1
vote
2answers
521 views

By using SMOTE the classification of the validation set is bad

I want to do classification with 2 classes. When I classify without smote I get: ...
3
votes
2answers
302 views

Testing Classification on Oversampled Imbalance Data

I am working on severely imbalanced data. In literature, several methods are used to re-balance the data using re-sampling (over- or under-sampling). Two good approaches are: SMOTE: Synthetic ...
2
votes
0answers
123 views

Block bootstrap for dependent data with unequal sampling intervals?

I have data from a natural archive (lake sediment). For various reasons it is usually impossible to sample the archive equally in time, and we end up with a time series where essentially we have ...