Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

Resampling is taking a sample from a sample. Common uses are jackknifing (taking a subsample, eg all values but 1) & bootstrapping (sampling w/ replacement). These techniques can provide a robust estimate of a sampling distribution when it would be difficult or impossible to derive analytically.

1
vote
0answers
40 views

Can we sample from a set of biased samples to get unbiased samples?

This is a follow-up question to How to uniformly sample vertices from a large graph with given distance from a fixed vertex?. Suppose I took a set $B$ of $n$ samples from a large but finite universe $...
0
votes
1answer
34 views

Process for oversampling data for imbalanced binary classe

I have about a 30% and 70% for class 0 (minority class) and class 1 (majority class). Since I do not have a lot of data, I am planning to oversample the minority class to balance out the classes to ...
0
votes
1answer
42 views

Precision and recall of imbalanced classes

I'm new and have searched many questions about this problem in this stack, but those answers aren't clear enough for me. The point is the area under PR curve of my binary classes is the same as the ...
1
vote
0answers
70 views

SMOTE, Oversampling on text classification in Python

I am doing a text classification and I have very imbalanced data like Category | Total Records Cate1 | 950 Cate2 | 40 Cate3 | 10 Now I want to over ...
0
votes
1answer
13 views

Is there any point in splitting data before applying NHST?

If I have understood correctly, Null Hypothesis Significance Testing is a device, which eats data and a conservative guess, and outputs a 'probability' that the data was generated by the hypothesized ...
3
votes
1answer
69 views

Oversampling: whole set or training set

I have a rather small dataset of 4 000 points (140 features) to feed to a NN binary classifier. The problem is only ~700 of them represent the second class. Is it more common to resample the whole ...
2
votes
0answers
40 views

“ANOVA on a non-random non-Normal sample from a Normal Population”

How can I run ANOVA or tests for statistical significance on a bi-modal sample that came from a normal population? Context: I was tasked with running an ANOVA to see if genotypes (treatments / ...
0
votes
1answer
28 views

Hypothesis testing on subsamples and cross validation

When working on very large datasets, identifying effects rests more on quantification than significance and many questions/answers give insight on what to do regarding large samples like this (very ...
2
votes
1answer
43 views

Conceptual questions on ensemble learning and Boosting methods in Matlab

The documentation on ensemble methods in Matlab explains different ensemble algorithms for classification and regression tasks. I have normalized the raw feature set and using the normalized data for ...
1
vote
1answer
44 views

Accounting for sensitivity of model to small changes in assumptions

I am running a Monte Carlo simulation of correlated asset returns, based on this Matlab function. The model's inputs are a vector of expected means and standard deviations for 9 assets, and their ...
1
vote
0answers
42 views

Bootstrap Resampling Vs KDE Resampling

Let $\xi\in\mathbb{R}^{m}$ be a random vector with joint desity function $f$, and let $\widehat{\xi}_{1},\ldots,\widehat{\xi}_{N}$ be a sample of $\xi$. We have that the kernel density estimator (...
0
votes
0answers
17 views

Resampling methods in derivation of prediction error

I have made a model which predicts some value $Y_i$ given $X_i$ where $i \in 1,...,N$. My ultimate goal is to predict $Y=\sum_i^n Y_i$ and to be specific the interval in which $Y$ lies within some % ...
1
vote
0answers
52 views

resampling from a frequency distribution (histogram) to test for significance

Suppose I have count data on brands and the number of toys they make. ...
0
votes
0answers
7 views

Using mean estimates of subsets of data or entire data to estimate a parameter

"Suppose that we wish to invest a fixed sum of money in two financial assets that yield returns of X and Y , respectively, where X and Y are random quantities. We will invest a fraction α of our money ...
3
votes
0answers
29 views

Optimal block size for spatial bootstrapping

Take a regression model: $$ y_s = X_s\beta + \epsilon $$ Where $E[\epsilon|X] = 0$, but $cor(\epsilon_s, \epsilon_{near}) > cor(\epsilon_s, \epsilon_{far})> 0$. In other words, $X$ is ...
0
votes
0answers
29 views
0
votes
0answers
80 views

Is Tomek Link undersampling the same as Edited Nearest Neighbours with 1 neighbour?

From what I've read I've understood that undersampling the majority class with Tomek Links or Edited Nearest Neighbours with 1 neighbour should yield the same result. However, I've tried it on this ...
3
votes
1answer
62 views

Variance after resampling uniformly from a sample from a normal population

I have recently been looking into the Bootstrap, and I was wondering, if I were to have a sample $X=\{x_1,x_2,\dots,x_N\}$ that has $N$ samples, all i.i.d coming from a normal distribution, $N(\mu,\...
0
votes
1answer
16 views

Removing confounding variable when comparing distributions

I have the following situation and am hoping that someone can point me in the right direction. I am completing an experiment under 2 conditions (1 and 2) and measuring 2 variables (A and B). I am ...
13
votes
3answers
3k views

Why is bootstrapping useful?

If all you are doing is re-sampling from the empirical distribution, why not just study the empirical distribution? For example instead of studying the variability by repeated sampling, why not just ...
1
vote
1answer
156 views

What's the point of reporting bootstrap bias?

Suppose the data $X = (X_1,X_2,\cdots,X_n)$ is a vector of iid observations $X_i$ where each $X_i$ has marginal distribution $F(\theta)$. Suppose we observe $x = (x_1,\cdots,x_n)$ and $\hat \theta = \...
0
votes
0answers
48 views

K-fold cross validation: confusion over which model is actually chosen and validated

I am watching the online lectures of Machine Learning by Hastie and Tibshirani, which are quite good. But they discussed $k$-fold cross-validation and there is something confusing me. As they describe ...
0
votes
0answers
143 views

Replicate weights and the Current Population Survey

The Current Population Survey’s Annual Social and Economic Supplement (abbreviated, oddly, as ASEC) is the USA’s longest running and most detailed annual survey of income and employment. Along with ...
1
vote
2answers
118 views

Mean of the sampling distribution of the OLS estimator

Suppose b represents the OLS estimator, and B the true coefficient in the regression model without intercept y = Bx + u. Under certain assumptions b is unbiased ...
0
votes
0answers
15 views

ROSE resampling altering certain values in variable

When using the ROSE package's ROSE() function, I resample data to achieve balanced classes. One of the numeric variables in my dataset appears to get altered where negative values present. Now, when I ...
0
votes
0answers
59 views

Analyzing resampled predictions in caret package

Is there a way to compare model performance in the caret package by combining the resampled predictions from every fold/repeat? I am working with a small (<1000 rows) and severely imbalanced (4% ...
3
votes
2answers
157 views

Resample-move algorithm for Sequential Monte Carlo

I'm reading about the resample-move strategy, originally by Gilks and Berzuini, but my question will use the slightly more verbose description from the review of Doucet and Johansen, section 4.4, PDF ...
10
votes
4answers
307 views

Why do hypothesis tests on resampled datasets reject the null too often?

tl;dr: Starting with a dataset generated under the null, I resampled cases with replacement and conducted a hypothesis test on each resampled dataset. These hypothesis tests reject the null more than ...
2
votes
1answer
63 views

What is the difference between SIR and Rejection sampling in this case

Suppose we want a sample of size n from a truncated gaussian distribution with density $f(x) = \dfrac{1}{\sqrt{2\pi}\sigma(1-\Phi((1-\mu )/\sigma ) }e^{-\frac{(x-\mu)^2}{2\sigma^2}}$ , $x>1$ set ...
0
votes
0answers
42 views

Ensemble mean by resampling parameter estimates in OLS nonlinear regression

I have tried to do a nonlinear regression using the 'nls' function in R, with equal weights among all observations and the default Gauss-Newton algorithm. The function takes a form of y = f(x,θ1,θ2,θ3)...
1
vote
2answers
460 views

Bootstrap p-value

Assume you created $B$ bootstrap replicates $T^{(i)}, i\in\{1,\dots,B\}$ for a test statistic $T$ since you don't know the distribution of this test statistic. Why is the p-value approximated by $\...
0
votes
0answers
21 views

Does increasing sample size by reducing universe invalidate previous data?

My team and I are working on a project that aims to measure the impact of adding more public bicycle stations in Mexico City on private vehicle traffic. We selected 100 key locations, which are spread ...
1
vote
0answers
46 views

Raking to converting a convenience sample to representative sample

I have a convenience sample of patients in a ZIP Code Tabulation Area (ZCTA) for whom I know their gender and age. I also have census data for the ZCTA (in particular I have 1) the percentage of ...
0
votes
0answers
271 views

Monte Carlo for a simple linear regression model

Intro: I have several time series of two variables over the course of one year (approx. 2.5k observations). The head of the data frame with the two variables of interest looks like this: ...
0
votes
0answers
10 views

Parameter optimization for re-sampling with noise to tackle class imbalance in Multinomial Naive Bayes

I am creating feature vectors to input into a neural network using a bag of words style approach on large corpuses of textual documents. In order to tackle a heavy class bias in a multi-class neural ...
0
votes
0answers
51 views

How to determine optimal sample size in unbalanced blocked anova-like design to be analysed with resampling methods?

I believe that some R package could deal with this situation, but I did not find one for this specific scenario. I'm only planing the sample design at this stage. The reason why I wish to use ...
2
votes
1answer
172 views

How to choose resample size when drawing without replacement?

Say I have some second-order statistic $m(x)$ where $x$ is a data vector of length $n$. Let's also assume that the limiting distribution of $x$ is gaussian-ish, but generally unknown, so that the ...
1
vote
0answers
48 views

How can I combine point estimates and standard errors from non-independent datasets?

I will conduct a 1:2 or 1:3 matched case-control study in order to examine risk factors related to a rare disease. Under this study, I'm planning to repeat selection of matched controls for at least ...
0
votes
0answers
112 views

How to correctly train a random forest classifier using live churn data (i.e. a snapshot)

I am assisting in the creation of a churn classification model and would like to use a random forest as it is a model that I understand fairly well. I have prepared a large dataset (~600,000 ...
0
votes
0answers
39 views

simulation check (bootstrapping) regression, what is the null hypothesis?

We are asked to do a simulation check on some regression problem. A brief description of the procedure is, roughly speaking, we need to simulate a bunch of fake datasets from the fitted model and ...
1
vote
0answers
27 views

Mean of a Non-Uniform Sample Rate

I have a nonuniform sample rate. I want to take the mean of samples for a given interval, say an hour. Within the samples of this hour, I want to weight each time slice equally. For example: if I ...
0
votes
0answers
34 views

Variance in unbalanced classification problem

I am dealing with an unbalanced classification problem- the dataset has 7000 observations, with 253 positive instances (5.8%), and 243 variables. I have decided to tackle this with Random Forests, and ...
0
votes
0answers
21 views

How can I decide the start of outliers to modify their values?

I have this figure above showing the distribution of my images sizes. As you can see the distribution is right skewed. In order to make the data reasonable I have to cut out in a certain range (ex. ...
0
votes
0answers
11 views

resampling into one two different predictors for a same estimate

Example Data Suppose I have a dataset with rabbit's weigths and lengths, along with weigths and lengths of both its left and right humerus. I plan on using the bones features to predict the overall ...
0
votes
1answer
146 views

testing for differences using jackknife distributions

I have two distributions relative to two experimental conditions. I compute a certain index (i.e. coherence) describing each distribution. I want to see if there is a significant difference ...
0
votes
1answer
29 views

Resampling approaches in multi-sample problems

I'm new to bootstrap and I would need an explanation about resampling techniques for comparing two populations. Suppose we have two samples, of respective sample sizes $n_1$ and $n_2$. The total ...
1
vote
1answer
49 views

Creating a simulation to find a p-value for a slope, is this valid?

I plot Score (y variable) against Year (x variable) and I want to be able to show that there is an increase in the score as the year increases. The scatter plot indeed indicates this but I would like ...
1
vote
0answers
139 views

“Parametric bootstrap” or “Monte Carlo test”

I have a data set on counts of amphibians in buckets around a pond, which I have detailed in a previous post: Circular statistics for discrete, irregular sector data To conduct a test of circular ...
1
vote
1answer
134 views

how to apply bootstrapping in this case?

I have a small sample of marks obtained by a group of students and I would like to apply a binomial test to check if more than half of the size of the classroom has failed due to the application of a ...