Resampling is taking a sample from a sample. Common uses are jackknifing (taking a subsample, eg all values but 1) & bootstrapping (sampling w/ replacement). These techniques can provide a robust estimate of a sampling distribution when it would be difficult or impossible to derive analytically.

learn more… | top users | synonyms

0
votes
0answers
12 views

Estimating performance of an iterated random algorithm

I have an algorithm that has a certain probability of computing an answer after a randomly distributed runtime. The probability may be less than 1 even for infinite time in a single run of the ...
1
vote
0answers
19 views

Visually Compare Different Sampling Methods [closed]

I am working on a project exploring results from a dataset, when using different sampling designs on that dataset. It is really fun thus far, and I have been learning a great deal of things especially ...
2
votes
0answers
115 views

Estimation on evolving distribution with small updates

I have a set $X$ of $10^6$ elements and a time series of probability distributions $\mu_1,\mu_2,\ldots$ on $X$. I want to estimate the expected value of a function $f$ over each $\mu_t$. It is easy to ...
0
votes
0answers
32 views

Training set selection

I have the following question for a project I'm working on. I am trying to find the best strategy to select the best training set in a dataset. I have a dataset with a few billions rows. I am trying ...
0
votes
0answers
19 views

What methods can I use to estimate the uncertainty when a subset of rows are removed from a dataset?

Apologies if I misuse some terminology here, I'm learning as I go along. I have a dataset with a set of descriptors and flags. There are around 60000 rows, but the flags are fairly sparse (covering ...
1
vote
1answer
34 views

Representative resampling

I am working with a population in which each individual has, among others, 6 observed variables that can be 0 or 1: $X_i \sim Bernoulli(p_i),\ i=1,...,6$ . I know the "true" value for the ...
2
votes
2answers
37 views

How to empirically show that a certain quantity approximately follows a normal distribution?

To motivate some theoretical work, I need to show that two certain variables (say $X$ and $Y$) approximately follow a normal distribution in actual datasets. I have one large dataset with about $300$ ...
0
votes
0answers
48 views

Resampling for unbalanced data in cross-validation

Resampling the data prior to classification is one of the techniques dealing with unbalanced dataset. I then consider down-sampling and ...
0
votes
0answers
40 views

Sampling, feature selection and preprocessing in cross validation

To brief my question, I want to clarify the order of parameter tuning and the correctness of the flow in my scheme. In my classification scheme, there are several steps including: SMOTE (Synthetic ...
1
vote
1answer
44 views

Bootstrapping a t-test in R

I have two groups of individuals (22 in each group), which I compared using a t-test. The difference between groups was non-significant (p = .17). Because the p-value was quite low, my supervisor ...
0
votes
1answer
41 views

Random sample of a random sample from a population: Also a random sample?

From population P we draw an adequately sized random sample S1. From the sample S1 we draw an adequately sized random sample S2 (with replacement). Are the distributional characteristics of S2 ...
0
votes
2answers
28 views

Bootstrapping a t statistic - Rationale and procedure

I was hoping someone could help me out with this. I've seen similar questions on the forum, but I need to know if I've understood the correct rationale and procedure for bootstrapping for my ...
2
votes
1answer
219 views

Interpretation of a 95% confidence interval calculated via bootstrapping?

I've been thinking about what exactly a 95% confidence interval means when it is calculated via bootstrapping. The formal definition of a 95% confidence interval is something like this: "if the ...
2
votes
0answers
22 views

Application of Permutation Tests

I don't know how to post the question more formally. Therefore, let me introduce an example. Suppose you want to estimate the following regression: $n_i = f(x_i)+\beta \cdot 1[x_i = j]$, $n_i$ is ...
1
vote
0answers
89 views

Bootstrapping time series data: Circular block bootstrap

I have some very basic questions on circular block bootstrap applied to time series (dependent data). Let's suppose, I have a time series data like the one below. I know it's non stationary, but for ...
0
votes
0answers
17 views

Is there a resampling method that blends subsampling with the bootstrap?

I apologize if this is an inappropriate question. I thought of it in class the other day, and I couldn't find a specific answer in my textbooks. I am familiar with the two basic techniques for ...
0
votes
0answers
18 views

How can I determine the power of a bootstrap confidence interval?

I'm halfway through an exercise for my statistics class, but completely stuck, and unable to find an answer online or from more statistical friends. Simulating n = 6 observations Xi ∼ Poisson(6), we ...
7
votes
3answers
98 views

Enlarging a random sample

In our project we have a population of 1000+ individuals. We picked a random sample of 107 individuals, but then we realized we needed more precision, so now we want to have a larger sample. The ...
3
votes
1answer
57 views

How often will sampling distribution of the mean not be normally distributed?

Kabacoff 2015 suggests that if we're not willing to assume the sampling distribution of the mean is normally distributed, we should use bootstrapping to estimate the sampling distribution of the mean. ...
-1
votes
1answer
49 views

What is Bootstrapping in statistics? How can I use it to determine error in the mean, variance, kurtosis and skewness of a data set?

From what I understood from searching randomly is that it has something to do with resampling. What does this resampling mean? Is it selecting random data from a distribution or is it getting data ...
0
votes
0answers
18 views

Comparing differences in pairs of ratings using a bootstrapping like approach

I'm pretty new to bootstrapping like approaches. So I do apologize for any inconvenience. I have two independent samples, snow (n=120) and nonsnow (n=608), consisting of pairs of ratings on a ...
0
votes
1answer
36 views

Resampling probability

I have a population of n unique items and am taking a sample of r. I am sampling with replacement. I would like to calculate the probability of sampling any specific item x times give the sample size ...
0
votes
0answers
46 views

Stability of boostrap confidence intervals

As a word of background, I want to show that certain result is stable when averaging over a large number of simulations, but could be just a lucky draw with a small number of simulations. I have a ...
1
vote
1answer
49 views

Confidence Interval For Mean By Bootstrapping

The standard deviation in my original sample is very large, about 100 or so. I took many bootstrap samples, found the mean of each bootstrap sample and then took the mean of these means. I found the ...
1
vote
2answers
103 views

R randomForest R replace=TRUE pro's and con's

When using R randomForest package I use replace=TRUE, which then dictates to: if (replace) nrow(x) else ceiling(.632*nrow(x)) I was wondering if anyone knows of ...
1
vote
0answers
52 views

How to interpret bootstraping output

I have a small dataset which has just 8 elements. I thought I could bootstrap to compare my sample with a normal distribution. I simply want to answer the question: how likely is it that the sample is ...
0
votes
0answers
49 views

Bootstrap resampling for constructing hypothesis test

I need to use bootstrap resampling to test the significant difference between two datasets (data1 & data2). I have already used bootstrap resampling to estimate the confidence interval of the mean ...
0
votes
0answers
6 views

Accuracy results on subsamples

Given a binary classification task, let us suppose that a classifier achieves an accuracy result of $x\%$ on a sample of size $N$. What would be a way to calculate the conditional probability that ...
0
votes
1answer
166 views

Custom resampling method in caret

I need to create a custom resampling method in R package caret where: For each leave-pair-out-cross-validation, from the training set I derive new data using a function I implemented. Then it is used ...
0
votes
0answers
55 views

Can you perform bootstrap resampling from a sampling distribution?

The quick and to-the-point question I have is: Can you perform bootstrap resampling on a sampling distribution, using the sampling distribution as if it were an original sample of observations? What ...
0
votes
2answers
43 views

Test if sampled data are randomly sampled

Is there a way to test if data are (or at least seem) randomly sampled? In other words, is there a way to measure if my data are randomly sampled -- instead of coming from a complex survey sampling ...
1
vote
1answer
147 views

Assessing fit of binomial glmer() in R with only categorical predictors

I am trying to validate a mixed effects logit regression model with a categorical dependent variable and categorical predictor variables - I have nothing that is continuous. One of my predictor ...
24
votes
2answers
1k views

How well does bootstrapping approximate the sampling distribution of an estimator?

Having recently studied bootstrap, I came up with a conceptual question that still puzzles me: You have a population, and you want to know a population attribute, i.e. $\theta=g(P)$, where I use $P$ ...
2
votes
1answer
179 views

what is the effect of bootstap resampling in bagging algorithm(ensemble learning)?

In ensemble learning with bagging, why is it important to do bootstrap resampling (sampling with replacement) instead of just sub-sampling (sampling without replacement)?
0
votes
1answer
74 views

Optimal method of generating training, validation, and test sets

Suppose that I have a single model that needs training (e.g., a neural network), that I have $N$ data points in a dataset that can be used for this purpose, and that I am not too concerned with ...
8
votes
2answers
676 views

what are the assumptions of permutation test?

It's often stated that permutation tests have no assumptions, however this is certainly not true. For example if my samples are somehow correlated, I can imagine that permuting their labels would not ...
8
votes
2answers
433 views

Best suggested textbooks on Bootstrap resampling?

I just wanted to ask which are in your opinion the best available books on bootstrap out there. By this I don't necessarily only mean the one written by its developers. Could you please indicate ...
1
vote
1answer
36 views

When do randomization methods outshine classical and non-parametric test

In which situations is it advantageous to use resampling methods as opposed to classical tests (if the data fit a certain distribution) or non-parametric tests? Is there a dataset/example that can ...
0
votes
0answers
37 views

How to use/interpret bootstrapping?

I am working on an stochastic optimization problem. Now I have come across the idea of using Monte Carlo sampling approach to solve it. I need the empirical distribution or the true distribution of ...
0
votes
0answers
61 views

Observational data, matching, and resampling. Is this method defensible?

I am interested in comments on the validity (or not) of the following method for propensity score analysis. I’ll simplify the scenario a bit for clarity. I have 1000 subjects that received the ...
0
votes
1answer
51 views

Bootstrap significance test

I'm using the bootstrap method to test my experiment results for significance. I have two sets (say A & B) of 50 grades, for which I want to test whether their means are significantly different. ...
0
votes
0answers
53 views

Quantifying the predictive ability of a model developed from a huge data set? (variation of bootstrapping?)

I have a statistical model with around 20 predictor variables, built on 90% of a dataset consisting of over 600k observations. The original developer held out 10% of the original dataset for the ...
0
votes
0answers
39 views

Fundamental Issues with Influence weighted resampling for bootstrapped predictions

I have a large database 1mill+ from which it is known that there are many influential points and outliers. I am interested in generating a series of predictions from subsets (1,000+) of the data and ...
4
votes
1answer
133 views

Bootstrapping the data to set up a prior

I am using a Gaussian model with a conjugate Normal-Inverse-Wishart (NIW) prior, as described here. The advantage of this approach is that the marginal likelihood $p(y)$, which is what I am interested ...
4
votes
2answers
621 views

Why use stratified cross validation? Why does this not damage variance related benefit?

I've been told that is beneficial to use stratified cross validation especially when response classes are unbalanced. If one purpose of cross-validation is to help account for the randomness of our ...
0
votes
1answer
54 views

Reservoir sampling with a computationally expensive weight function

I have a large dataset, and I want to obtain a small sample of it of size K, weighted by a function f(x) which is expensive to compute (I'm ok computing it O(K) times, but not too much more). Suppose ...
0
votes
1answer
41 views

sampling from confidence intervals

If we are provided a mean and (95%) confidence interval, is it possible to set up a system in which we draw random values that are outside the CIs 5% of the time? My intuition is that if one can ...
1
vote
1answer
75 views

parametric bootstrap for low sample sizes

I believe that this question is sufficiently different from previous related ones to warrant a new post. (I apologize if it has been answered already) I need to decide between various resampling ...
7
votes
1answer
370 views

Is bootstrapping appropriate for this continuous data?

I'm a complete newbie :) I'm doing a study with a sample size of 10,000 from a population of about 745,000. Each sample represents a "percentage similarity". The great majority of the samples are ...
2
votes
0answers
100 views

Estimating means of correlated distributions with long tails

Suppose I have a relatively large number of samples (~1k) drawn from a series (~40) of increasingly long-tailed distributions (going from approximately normal to approximately log-normal). I want to ...