Questions tagged [resampling]
Resampling is taking a sample from a sample. Common uses are jackknifing (taking a subsample, eg all values but 1) & bootstrapping (sampling w/ replacement). These techniques can provide a robust estimate of a sampling distribution when it would be difficult or impossible to derive analytically.
293
questions
0
votes
0answers
7 views
Calculating Mean Min Error and Mean Mean Error (with Confidence) in a Set of Samples using Bootstrap/Jackknife
The question I am trying to solve is this: If I take $n$ random samples from a parameter space, what are the means and confidence intervals of the mean and min error? I want to calculate this for ...
1
vote
0answers
36 views
How to bootstrap the time series data for assets portfolio mean - variance optimization?
As known among professionals and amateur investors, the classical Markovitz portfolio mean - variance optimization yields very unstable results that perform poorly in the real world.
One of the ...
0
votes
0answers
9 views
Test and Validation datasets for imbalance classification tasks using SMOTE or other over/under methods
I was reading up more on class imbalance and over/under-sampling methods to help reduce the imbalance and improve ROC. I am working on an extremely imbalanced dataset for click-prediction, so like ...
0
votes
0answers
6 views
Using resampling to estimate confidence interval on
I have historical datasets of prices of two products, $A$ and $B$. These prices are not unrelated, but their relation is complex and depends on more factors, some of which I do not know or cannot ...
2
votes
2answers
58 views
What are the uses of bootstrap resampling?
I'm still wrapping my head around bootstrapping but am struggling to think of how it is applied. I have looked at:
Explaining to laypeople why bootstrapping works
So far, my understanding is that ...
0
votes
0answers
19 views
Forecasting using unevenly sampled data
We have some head position data obtained from the sensors of a Virtual Reality headset. First 200 ms of the data from one user measured while watching one 360-video sequence looks like as follows:
<...
0
votes
0answers
10 views
Is it possible to achieve both stratified sampling and keeping the same train/test dataset split across different runs?
Generally, it is suggested to sample a dataset such that test set and train set remain the same when running the code multiple times, for comparison but also to hide your algorithm the whole dataset. ...
2
votes
0answers
37 views
Estimating Unique Population Sizes
I am trying to estimate the total number of unique visitors to my website.
My website contains 500 different pages.
I am able to:
find the number of unique users to a given page
find the number of ...
0
votes
0answers
20 views
How can a Keras convolutional network be defined such that it outputs images of the same dimensions as the input?
I wanna train a convolutional neural network to convert an input image to an output image, where the input and output images are of the same dimensions (50 pixels wide, 300 pixels high and greyscale). ...
1
vote
0answers
41 views
How to handle missing values in a bootstrap resampling distribution?
I am creating a function that calculates point and interval estimates for a statistic of interest. I get the interval estimate through bootstrapping. However, it occasionally occurs that the statistic ...
1
vote
2answers
53 views
Bootstrap for CIs and Permutation Resampling for Hypothesis Test?
In various scattered places online and StackExchange I've read that bootstrap resampling is more appropriate for calculating confidence intervals while permutation resampling is more appropriate for ...
1
vote
3answers
308 views
Does oversampling/undersampling change the distribution of the data?
I have an imbalanced dataset (10000 positives and 300 negatives) and have divided this into train and test sets. I perform oversampling/undersampling only on the train set since doing this on the test ...
0
votes
0answers
73 views
Is two group comparison test combinig rarefaction and permutation usefull and correct?
I wish to know if the analytical procedure using R environment described below is statistically coherent to enable interpretations and conclusions. I looking for evaluations and opinions about the ...
0
votes
0answers
23 views
bootstrapping resampling for unfair dice
Let's say I've been given an unfair dice. After 1000 throws, these are the empirical frequencies:
1, 100
2, 100
3, 100
4, 100
5, 100
6, 500
I would like to represent this in a barplot, where each ...
1
vote
0answers
27 views
Resampling methods or statistical tests for variables that comprise dependent and independent observations
I am currently working with a dataset that comprises measurements of multiple participants on two variables. I'd like to test the mean difference between both variables on statistical significance.
...
0
votes
0answers
7 views
Is generating data based on subsample statistics useful for bootstrapping regression models?
I am currently reviewing lecture slides about regression models.
One slide reads:
If there are multiple (repeated) measurement points available:
Generate a normal distribution of data per ...
1
vote
1answer
19 views
Group comparison for small sample with repeated measured data
I have very small dataset.
When I conducted rank sum test between the groups (A vs B; A vs C), I found no significant results, but just trends level difference.
Is there any alternative statistical ...
0
votes
0answers
78 views
R-Caret, Regression, different number of PCA components for finalModel and resampling with PCA?
In my project I train models with the "Timeslice" method from the caret package but this question also fits in with other methods, such as cross-validation.
Imagine you have 2584 records and split ...
1
vote
1answer
43 views
Running SMOTE on unbalanced data
I've read a few answers to similar questions that advise the completion of SMOTE after splitting the data set into Train and Test sets however, the documentation and other examples I've seen run SMOTE ...
1
vote
0answers
12 views
Downsampling, AUROC and accuracy equal
I am using downsampling to create perfectly balanced classes in my target feature. I have found that accuracy is exactly equal to the AUROC score. I was thinking that this is because I've used ...
4
votes
1answer
92 views
How to compute confidence intervals from *weighted* samples?
Imagine we have a webserver, which serves a total of N static URLS.
There are users visiting the URLs every day. At the end of each day, we have data like this:
...
0
votes
0answers
24 views
Nested cross-validation with resampling
I am aware that sckit-learn has a provision to perform nested cross-validation. I wanted to perform the same with resampling. My confusion stems from whether their example listed here performs ...
0
votes
0answers
12 views
When is the bias of a statistic of the form a/n + b/n^2 + c/n^3 +
In many books the bias-correction of the Jackknife resampling method is being prooved under the assumption, that the bias has a special form, namely a/n + b/n^2 + c/n^3 * ...
Sometimes it's written "...
0
votes
0answers
90 views
KDE that better-preserves percentile distributions
My understanding is that a Gaussian KDE, because the kernel is symmetric, preserves the statistical mean of a distribution. For my particular case, I'd really prefer a KDE that preserved the median, ...
0
votes
0answers
20 views
Correct approach to define a null distribution by resampling?
I am trying to devise a test for significance by resampling from my data.
The data consists of 60 individuals, grouped in 6 populations (10 per population). There are about 9,000 genes as ...
1
vote
1answer
98 views
Central limit theorem for resampling
What is the analog of the central limit theorem or concentration theorem for resampling, say, an i.i.d. samples? Are there any references for this topic?
Here is a simple example. Suppose there are $...
0
votes
0answers
36 views
Is there any way to resample in order to change the distribution of a sample?
I ran an experiment that involved an independent variable which is known to have a right skewed distribution. However, in the experiment, the independent variable was distributed differently between ...
0
votes
0answers
18 views
Can one construct “original” data from a function of jackknifed data?
Say I have original, uncorrelated data, $x_i$, with $i = 1,2 \ldots N$. I can jackknife this data set (a simple delete-one)
$$
\bar{x}_{i} = \frac{1}{N-1}\sum_{j \neq i}x_{j} \quad\quad (1)
$$
to ...
0
votes
1answer
13 views
Resample dataframe by random subset of years
I have a dataset of 59 years worth of daily rainfall data.
I'd like to resample the data by 6 randomly selected years 1,000 times, using the entire year of data for each of the 6 years (so 2190 ...
1
vote
0answers
48 views
How does one estimate error of extrapolated value with only 2 data points?
I am trying to do a simple linear regression. I have only two data points with errors. How do I estimate the y-intercept with errors?
Currently I have:
...
1
vote
0answers
57 views
When to use ordinary, balanced, antithetic, or permutation resampling for bootstrap?
I am using boot and would appreciate any explanation as to when using each of these resampling methods would be recommended in practice.
I have come across Do who says that:
Simulation results ...
2
votes
0answers
35 views
Correlating Two Time Series with Gaps in Data and of Different Lengths
I am attempting to correlate the time series from 4 separate tilt monitors that sample every 5 minutes. The time series have slightly different base times and end times, and the resulting arrays are ...
1
vote
0answers
44 views
How to calculate p-value comparing bootstrap-based predicted probabilities and observed probabilities
I posted this question on stackoverflow first but I have got no answer so far, so I decided to post it here in the hope that here I might get an answer. I hope my procedure is acceptable.
...
2
votes
1answer
134 views
Bootstrap, Rubin's rules, and uncertainty of sub-estimates?
Can someone provide an intuition for why, when using bootstrap to calculate the variability of an estimate (say a regression coefficient $\beta$) we don't need to incorporate the uncertainty of each ...
0
votes
0answers
14 views
Appropriate visualization and statistical testing using resampling stats
I'm using a linear SVM to perform classification on a dataset.
It seems to me that there are many ways to visualize my results and report their statistical significance, and I'm unsure of best ...
0
votes
0answers
129 views
Adjusting predicted probabilities after resampling
Suppose I've got highly imbalanced data and I want to train a model, for binary classification. So I upsample the minority class or downsample the majority class or whatever. My question is whether ...
0
votes
0answers
62 views
Permutation tests for paired data with several variables
I am trying to answer my own previous question, with a permutation method.
I have a reference method $M_{ref}$ that I would like to compare with other methods $M_1$, $M_2$, $M_3$ when taking ...
1
vote
0answers
65 views
Calibration curve for mixed model (logistic) [closed]
I did some searching using the CV search function and was unable to locate any information on R packages or general approach to creating a calibration curve with a bootstrapped curve overlay (similar ...
1
vote
0answers
43 views
Bootstrap test of means using pooled groups - is there a problem?
In his book "Resampling: the New Statistics" (available for free online), Julian Simon presents several examples where he Bootstraps the differences in group means, assuming that the observations of ...
1
vote
3answers
339 views
How should I resample the training and testing set with imbalanced data whilst having meaningful performance metrics?
I have an imbalanced dataset of approx. 200 positive and 800 negative examples.
I run nested cross-validation where i=5 and j=5; (i is inner and j is outer). The cross-validation procedure isn't the ...
0
votes
0answers
12 views
Adding Bad Events from the past to the risk default model to avoid Down/Up sampling techniques
We have been trying to build a classification model for credit default prediction using two different models one being Random forest and another being the Logistic regression based scorecard model. ...
1
vote
0answers
164 views
how to use boot package to do stratified bootstrapping?
Here's a toy data set that replicates my problem. I am interested in knowing the confidence intervals of an empirical distribution that is composed of the scores of each school at the proportion that ...
0
votes
0answers
14 views
Hypothetical sample size
I have a data set of n=98 of values that is normal distributed. This data set has a certain spread and hence a certain standard error of the mean. I now want to test how many samples (ns) I would need ...
1
vote
0answers
41 views
Practical questions about cluster bootstrap confidence intervals
I want to estimate the accuracy of a machine learning model. I have an independent test set with a vector of trusted labels and a corresponding vector of model-based predictions. If I assume the test ...
0
votes
0answers
50 views
How to approximate continuous function from discrete uneven interval value?
To clarify, I have a discrete value of an unknown distribution. The value of this distribution domain is from 0 to 1. But the value which the discrete value is sampled has an unknown interval.
For ...
0
votes
0answers
43 views
General way to construct a confidence interval for a unknown constant to which a sample estimator converges
Assuming that a sample estimator converges to some unknown constant (a wild assumption to be sure) and without assuming the distribution of either the sample estimator or the variables from which it ...
0
votes
1answer
94 views
ROC and PR curves after over/under sampling in Unbalanced datasets
As I understood till now, ROC curves are not a good presentation of unbalanced datasets and PR curves are preferred because ROC curves are not sensitive to false positives. If we now use resample ...
2
votes
1answer
135 views
Sample size calculation for Kendall's Tau in reproducibility study
Can anyone help with justifying (or rubbishing) a couple of aspects of a sample size calculation for a grant proposal.
A pilot study gave significant results for the correlation of a continuous (non-...
0
votes
0answers
64 views
How to interpolate/resample both dense and sparse points?
Suppose I have data like red points below
I would like to interpolate/resample these points at black ticks.
At right the points are sparse and it is obvious to interpolate them linearly or with ...
1
vote
0answers
226 views
Monte Carlo testing: number of required permutations
I want to perform a statistical hypothesis test, however I don't know the exact distribution of my test statistic under $H_0$. Therefore, I need to calculate a Monte Carlo estimate $\hat{p}$ of the ...
