Resampling is taking a sample from a sample. Common uses are jackknifing (taking a subsample, eg all values but 1) & bootstrapping (sampling w/ replacement). These techniques can provide a robust estimate of a sampling distribution when it would be difficult or impossible to derive analytically.

learn more… | top users | synonyms

0
votes
0answers
8 views

Bootstrap Step-Wise Selection For GLM in R [migrated]

What I'm trying to do is the following: Resample (bootstrap) the dataset containing my predictors and outcomes, say 1000 times For each bootstrap sample fit a glm using stepwise selection and then ...
0
votes
0answers
16 views

Comparing differences in pairs of ratings using a bootstrapping like approach

I'm pretty new to bootstrapping like approaches. So I do apologize for any inconvenience. I have two independent samples, snow (n=120) and nonsnow (n=608), consisting of pairs of ratings on a ...
0
votes
1answer
29 views

Resampling probability

I have a population of n unique items and am taking a sample of r. I am sampling with replacement. I would like to calculate the probability of sampling any specific item x times give the sample size ...
0
votes
0answers
39 views

Stability of boostrap confidence intervals

As a word of background, I want to show that certain result is stable when averaging over a large number of simulations, but could be just a lucky draw with a small number of simulations. I have a ...
0
votes
1answer
27 views

Confidence Interval For Mean By Bootstrapping

The standard deviation in my original sample is very large, about 100 or so. I took many bootstrap samples, found the mean of each bootstrap sample and then took the mean of these means. I found the ...
1
vote
2answers
68 views

R randomForest R replace=TRUE pro's and con's

When using R randomForest package I use replace=TRUE, which then dictates to: if (replace) nrow(x) else ceiling(.632*nrow(x)) I was wondering if anyone knows of ...
1
vote
0answers
40 views

How to interpret bootstraping output

I have a small dataset which has just 8 elements. I thought I could bootstrap to compare my sample with a normal distribution. I simply want to answer the question: how likely is it that the sample is ...
0
votes
0answers
37 views

Bootstrap resampling for constructing hypothesis test

I need to use bootstrap resampling to test the significant difference between two datasets (data1 & data2). I have already used bootstrap resampling to estimate the confidence interval of the mean ...
0
votes
0answers
6 views

Accuracy results on subsamples

Given a binary classification task, let us suppose that a classifier achieves an accuracy result of $x\%$ on a sample of size $N$. What would be a way to calculate the conditional probability that ...
0
votes
1answer
90 views

Custom resampling method in caret

I need to create a custom resampling method in R package caret where: For each leave-pair-out-cross-validation, from the training set I derive new data using a function I implemented. Then it is used ...
0
votes
0answers
47 views

Can you perform bootstrap resampling from a sampling distribution?

The quick and to-the-point question I have is: Can you perform bootstrap resampling on a sampling distribution, using the sampling distribution as if it were an original sample of observations? What ...
0
votes
2answers
38 views

Test if sampled data are randomly sampled

Is there a way to test if data are (or at least seem) randomly sampled? In other words, is there a way to measure if my data are randomly sampled -- instead of coming from a complex survey sampling ...
1
vote
1answer
93 views

Assessing fit of binomial glmer() in R with only categorical predictors

I am trying to validate a mixed effects logit regression model with a categorical dependent variable and categorical predictor variables - I have nothing that is continuous. One of my predictor ...
24
votes
2answers
1k views

How well does bootstrapping approximate the sampling distribution of an estimator?

Having recently studied bootstrap, I came up with a conceptual question that still puzzles me: You have a population, and you want to know a population attribute, i.e. $\theta=g(P)$, where I use $P$ ...
1
vote
1answer
113 views

what is the effect of bootstap resampling in bagging algorithm(ensemble learning)?

In ensemble learning with bagging, why is it important to do bootstrap resampling (sampling with replacement) instead of just sub-sampling (sampling without replacement)?
0
votes
1answer
53 views

Optimal method of generating training, validation, and test sets

Suppose that I have a single model that needs training (e.g., a neural network), that I have $N$ data points in a dataset that can be used for this purpose, and that I am not too concerned with ...
6
votes
2answers
348 views

what are the assumptions of permutation test?

It's often stated that permutation tests have no assumptions, however this is certainly not true. For example if my samples are somehow correlated, I can imagine that permuting their labels would not ...
8
votes
2answers
372 views

Best suggested textbooks on Bootstrap resampling?

I just wanted to ask which are in your opinion the best available books on bootstrap out there. By this I don't necessarily only mean the one written by its developers. Could you please indicate ...
1
vote
1answer
31 views

When do randomization methods outshine classical and non-parametric test

In which situations is it advantageous to use resampling methods as opposed to classical tests (if the data fit a certain distribution) or non-parametric tests? Is there a dataset/example that can ...
0
votes
0answers
34 views

How to use/interpret bootstrapping?

I am working on an stochastic optimization problem. Now I have come across the idea of using Monte Carlo sampling approach to solve it. I need the empirical distribution or the true distribution of ...
0
votes
0answers
56 views

Observational data, matching, and resampling. Is this method defensible?

I am interested in comments on the validity (or not) of the following method for propensity score analysis. I’ll simplify the scenario a bit for clarity. I have 1000 subjects that received the ...
0
votes
1answer
39 views

Bootstrap significance test

I'm using the bootstrap method to test my experiment results for significance. I have two sets (say A & B) of 50 grades, for which I want to test whether their means are significantly different. ...
0
votes
0answers
51 views

Quantifying the predictive ability of a model developed from a huge data set? (variation of bootstrapping?)

I have a statistical model with around 20 predictor variables, built on 90% of a dataset consisting of over 600k observations. The original developer held out 10% of the original dataset for the ...
0
votes
0answers
31 views

Fundamental Issues with Influence weighted resampling for bootstrapped predictions

I have a large database 1mill+ from which it is known that there are many influential points and outliers. I am interested in generating a series of predictions from subsets (1,000+) of the data and ...
4
votes
1answer
127 views

Bootstrapping the data to set up a prior

I am using a Gaussian model with a conjugate Normal-Inverse-Wishart (NIW) prior, as described here. The advantage of this approach is that the marginal likelihood $p(y)$, which is what I am interested ...
3
votes
1answer
257 views

Why use stratified cross validation? Why does this not damage variance related benefit?

I've been told that is beneficial to use stratified cross validation especially when response classes are unbalanced. If one purpose of cross-validation is to help account for the randomness of our ...
0
votes
1answer
44 views

Reservoir sampling with a computationally expensive weight function

I have a large dataset, and I want to obtain a small sample of it of size K, weighted by a function f(x) which is expensive to compute (I'm ok computing it O(K) times, but not too much more). Suppose ...
0
votes
1answer
40 views

sampling from confidence intervals

If we are provided a mean and (95%) confidence interval, is it possible to set up a system in which we draw random values that are outside the CIs 5% of the time? My intuition is that if one can ...
1
vote
1answer
67 views

parametric bootstrap for low sample sizes

I believe that this question is sufficiently different from previous related ones to warrant a new post. (I apologize if it has been answered already) I need to decide between various resampling ...
0
votes
0answers
21 views

Finding the fewest needed samples for a regression method

I have a regression method that I have applied with success to tens of thousands of well-observed objects. I'm looking to estimate how well it works on poorly observed objects. I selected 50 objects ...
7
votes
1answer
307 views

Is bootstrapping appropriate for this continuous data?

I'm a complete newbie :) I'm doing a study with a sample size of 10,000 from a population of about 745,000. Each sample represents a "percentage similarity". The great majority of the samples are ...
0
votes
0answers
47 views

How to improve results when using sampling in skewed binary classification?

I am using a data set with 18 features with True/False output (Related to mobile ad targeting). True values occurs only 0.4 % of the time. So, I have used sampling to keep the ratio of True and False ...
2
votes
0answers
92 views

Estimating means of correlated distributions with long tails

Suppose I have a relatively large number of samples (~1k) drawn from a series (~40) of increasingly long-tailed distributions (going from approximately normal to approximately log-normal). I want to ...
21
votes
2answers
5k views

resampling / simulation methods: monte carlo, bootstrapping, jackknifing, cross-validation, randomization tests, and permutation tests

I am trying to understand difference between different resampling methods (Monte Carlo simulation, parametric bootstrapping, non-parametric bootstrapping, jackknifing, cross-validation, randomization ...
1
vote
0answers
162 views

Oversampling with categorical variables

I would like to perform a combination of oversampling and undersampling in order to balance my dataset with roughly 4000 customers divided into two groups, where one of the groups have a proportion of ...
2
votes
1answer
47 views

Convenience sampling - Distribution forcing?

I am conducting some experiments on a data set that was collected by convenience. It is a data set based on historical data, most of which is not digitized. I know the exact distribution of the ...
1
vote
0answers
14 views

Correcting for multiple comparisons using simulated distribution

To check if unilateral pairs (defined below*) are coordinated (i.e. move together) in a flock of N coordinated individuals, we generate hypothetical (null) distribution of a certain focal parameter of ...
0
votes
0answers
74 views

Determining characteristics of sampling sets for EFA/CFA/SEM

Dividing sample data into several sets seems to be a common approach in statistics. This is especially evident in predictive modeling, where samples are traditionally divided into two sets, usually ...
0
votes
0answers
98 views

What's the safest way to resample to ensure equal class frequencies in training data?

I am working on a number of EEG data sets for binary classification. A good example of one is publicly available here. If you look at what Matthias Kaper did to classify that set, one thing that ...
5
votes
1answer
264 views

Why not always use bootstrap CIs?

I was wondering how bootstrap CIs (and BCa in barticular) perform on normally-distributed data. There seems to be lots of work examining their performance on various types of distributions, but could ...
1
vote
1answer
131 views

Logistic regression and discrepant sample sizes between 0/1 groups

I currently working in a multivariate logistic model but I have a problem regarding the sample size of my observations: -The "success" (1) event group has a sample size of 249 distinct observations - ...
1
vote
0answers
31 views

To find variance and covariance for a double sampling problem

A simple random sample of size $n=n_1 + n_2$ is drawn without replacement from a finite population of size $N$. Further a simple random sample of size $n_1$ is drawn without replacement from the first ...
2
votes
1answer
18 views

Doing comparisons from two null distributions

I have a situation where I have an observed statistic computed from data, and then I have approximated the null distribution by some sort of resampling. I have used this to calculate p-values for a ...
2
votes
0answers
73 views

What is the correct way of generating a p-value for a correlation with resampling?

I have a vector of gene expression values, across 20 patients. Each patient also has a glucose measure (a continuous numeric vector). I want to find how significant the real correlation value is ...
2
votes
0answers
21 views

Using Resampling to understand a large table

I have a data set that is very large. The attributes (columns) are several thousand. Some are sparse others are not. Some are ordinal, others interval, nominal or ratio. The row size is 10s of ...
1
vote
1answer
125 views

Addressing Non-response in a Convenience Sample

I am studying customer satisfaction in a large hierarchical organization. I plan to administer a voluntary survey to customers across the organization, and need to address non-response in my analysis. ...
2
votes
1answer
599 views

Optimal sampling strategy for EFA, CFA and SEM

I'm wondering what should be the optimal sampling strategy for my dissertation research. I have four data sources (two open source software projects meta-repositories and two global startup ...
0
votes
1answer
299 views

What are RMSE SD and Rsquared SD metrics in resampling results using R package:caret?

I've been doing predictive modelling with R package caret. When resampling regression models, I get the traditional RMSE and Rsquared metrics, but also RMSE SD and ...
0
votes
0answers
77 views

Bootstrapping data with only sampling weights given

Suppose you only have these information from a sample data: $X_i$ and $w_i$, $i=1,...,N$, where $w_i$'s are the respective sampling weights(not integers). Is it possible to obtain a valid bootstrap ...
2
votes
1answer
90 views

Bias and variance estimation with boostrap

The Wikipedia article about Jacknife estimation of the bias and variance of an estimator $\theta$ includes the following formulas: Variance of $\theta$: $ \operatorname {Var}(\theta )=\sigma ...