Questions tagged [subsampling]

Subsampling is a resampling procedure akin to the bootstrap in which fewer than all observations are being drawn with replacement (vs. the original sample size used in the textbook bootstrap method). For creating samples out of your existing data, please consider "sampling" tag instead.

38 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
129 views

Finding a sub-population from dataset matching another target dataset

Let's say one has a finite collection of i.i.d. samples from an unknown source distribution $S=\{x_{i} | i \in [1,n_{S}], x_{i} \sim p_{X_{S}}(x)\}$. Where each $x$ is multidimensional and has ...
3
votes
0answers
126 views

Is there a resampling method that blends subsampling with the bootstrap?

I apologize if this is an inappropriate question. I thought of it in class the other day, and I couldn't find a specific answer in my textbooks. I am familiar with the two basic techniques for ...
2
votes
1answer
90 views

Subsample analysis based on country-level indices?

In a generalized Difference-in-Difference setting from Dasgupta,2019 for multiple event dates (laws staggered implementation) The baseline equation: $Y_{it}$ = $\alpha$ + $\beta$ $(Leniency Law)_{kt}$ ...
2
votes
0answers
16 views

Subsampling the "right" amout of data to train an ML model

I am training a machine learning model (i.e., a classifier) on a large dataset. I know that I can get the same results using less data (about 30%) but I would like to avoid the trial and error process ...
2
votes
0answers
797 views

Formal method to find optimal sub-sample size from large sample for multiple regression

I have labour market data for 9 million observations, for a single time period (i.e cross-section data). I am studying the determinant of wages in a single equation multiple regression with around 300 ...
2
votes
0answers
93 views

Confidence/credible intervals for parameter estimates from structured support vector machine

I am estimating parameters for a conditional random field using a structured support vector machine. The data consists of a flat graph of $i%$ city blocks, where $y_i$ is the assignment of the the $...
1
vote
0answers
64 views

What we should do when the result is only significant to one country but the whole sample?

In a staggered Difference-in-Differences setting, Dasgupta, 2019 has a formula for such a static setting is $Y_{it}$ = $\alpha$ + $\beta$ $pt_{kt}$ + $\delta$$X_{ikt}$ + $\theta$$_t$ + $\gamma$$_i$ +$\...
1
vote
0answers
15 views

Knowing when to stop with spatial point thinning -- is an approach with Clark-Evans test valid?

I need a random sample of presence points from the species' area to use in distribution modeling. I have an excessive amount of spatially clustered presence records, so I use minimum nearest neighbor ...
1
vote
1answer
30 views

Combining Sub-Samples for Factor Analysis?

I am a newbie on the site and a relative newbie to some of the analysis I am trying, so my apologies in advance for any rookie mistakes or for asking what might be obvious to others! Can I run a ...
1
vote
1answer
58 views

Speed Up KNN and Maintaining Accuracy for Anomaly Detection

This question is about using KNN in the context of anomaly detection. If the training dataset is large(10 M data points), KNN will be slow. Is subsampling(i.e. use a small subset of original training ...
1
vote
0answers
51 views

Fitting glmms to data with very unbalanced sampling effort - could subsampling help?

I have a data set comprising measurements of invertebrate species richness from grab samples of seabed sediment collected from a shallow coastal area over a period of 20 years. The sampling effort is ...
1
vote
0answers
133 views

Covariance between sample mean and subsample mean

To estimate $\overline{Y}$(population mean), suppose an SRSWOR of size $n$ is taken from a population of size $N$, and the sample mean $\overline{y}$ is calculated. Then an SRSWOR subsample of size $...
1
vote
0answers
50 views

appropriate error term for a randomized complete block design, possible split plot in time, with subsamples (in SAS PROC MIXED))

I am analyzing the results of an agricultural experiment, as follows: Plots are laid out in a randomized complete block design with 3 replications. They have been either conventionally or organically ...
1
vote
1answer
199 views

Finding the variance of subsample-based estimation

Say, we know that the probability of an object having some property equals exactly $P$. We are given a sample (of size $N$) of these objects - in fact, that is a Binomial distribution with probability ...
1
vote
0answers
26 views

Can I take a random sample of my very large data set to overcome non-independence?

I am trying to run a regression model on a very large time series data set (comparing flow noise to vehicle speed, pitch and dive state). Because my samples are taken about every minute (with some ...
1
vote
0answers
287 views

Subsampling as a method for time series train/validation splits

I have a question concerning train-test splits for time series data: Background I have a dataset of sensor data points for 13 month with datapoints measured every 5 minutes which I downsample to ...
1
vote
0answers
17 views

Sub-sampling to quantify mean and errors

I have a model in matlab that, ultimately, outputs a yes or a no depending on certain input parameters. There's a degree of randomness in the model, so by runnning it 1000 times, I may end up with 200 ...
1
vote
0answers
883 views

Subsampling - Choice of Subsample Size?

I have a question with regard to Subsampling. Subsampling: take samples without replacement of size b from the original sample of size n with b < n Bootstrapping: take samples with replacement of ...
1
vote
0answers
46 views

Significance for the whole dataset of detections in a subsample

I have a fairly simple question, but I am not so familiar with these problems so I would like to ask. If I have, let's say, 4 data samples belonging to the same overall population. If in a test a ...
1
vote
0answers
107 views

Generalized method of moments (GMM): testing equality of parameters across subsamples

I estimate parameters of a panel data model with GMM using Stata. I specify the variance-covariance matrix assuming that the observations are correlated in the same period of time (...
0
votes
0answers
31 views

Subsampling For Class Imbalances and no-information rate

Question has to do with the interpretation of output of the caret package . Subsampling (either up or down) is set up in caret ...
0
votes
0answers
16 views

Friedman's test over multiple dependent datasets

In 'Statistical comparisons of classifiers over multiple data sets', Friedman's test is applied to compare different machine learning algorithm performances over different datasets (obtained from ...
0
votes
1answer
60 views

Linear regression in very unbalanced data

I hope you can help me with this question. I have a dataset with several classes (around 25) and they are very unbalanced. Some classes have thousands of subjects, others hundreds, and others just a ...
0
votes
0answers
10 views

Distribution of a rate in a subsample after permutation of the total dataset

I need to estimate the distribution of a rate in a subsample after permutation of the total dataset. I was wondering whether it's sufficient to use the binomial distribution with the global rate ...
0
votes
0answers
47 views

Independent random sampling without replacement for sampled Gaussian mechanism

I was reading a paper about "Sampled Gaussian Mechanism" which is used in training differentially private ML models. This mechanism is a composition of ...
0
votes
0answers
23 views

comparing the effect and magnitude of two coefficients across two models

I have a panel dataset with 500 banks (in 20 countries) over the period 2000-2015. First, I perform a baseline regression by regressing my dependent variable; bank risk with my independent variable; ...
0
votes
0answers
33 views

R: repeated subsampling for unbalanced design with zero-inflated count data using glmmtmb

I have count data with the number of individuals per observation of 64 different plant species. 3 - 5 observations were made of each plant species, and the number of arthropods on the plants were ...
0
votes
0answers
139 views

Can you use the isolation forest algorithm on large sample sizes?

The original isolation forest paper states that the algorithm works best on small subsamples, but is it okay to use it on large sample sizes or are other anomaly detection algorithms better?
0
votes
1answer
109 views

How to partition a sample into representative subsamples?

The problem is the following: take a sample $X$ of the general population $\Omega$. Each element of $\Omega$ (and hence, each element of $X$) is described by a vector of characteristics, each ...
0
votes
1answer
44 views

Subsampling large dataset before testing?

I'm running a Kruskal test with Conover post-hoc test to assess if there is a statistically significant difference between a numerical and a categorical variable with R. I previously created boxplots ...
0
votes
0answers
20 views

Techniques to account for differences in misclassification "cost" on variables other than the outcome

Suppose you're in a classic classification context: you want to predict whether a patient has a certain virus. You are working in multiple regions (let's say 2 for simplicity: Region A and Region B) ...
0
votes
1answer
351 views

How to compute ESS (Effective Sample Size)?

I implemented the ESS calculation according to this manual like this: ...
0
votes
0answers
14 views

Presenting results of estimation with sub-sampling

I have a dataset that we can partition between three groups: Controls, Treated 1 and Treated 2. I want to run regressions that include the whole Control and Treated 1 groups, but I draw a random ...
0
votes
0answers
46 views

General way to construct a confidence interval for a unknown constant to which a sample estimator converges

Assuming that a sample estimator converges to some unknown constant (a wild assumption to be sure) and without assuming the distribution of either the sample estimator or the variables from which it ...
0
votes
0answers
152 views

Subsampling to account for spatial autocorrelation of observations

I'm wondering to what extent (if any) subsampling of observations can be used to account for spatial autocorrelation within data. Is taking a smaller sample (subsample) of observations (without ...
0
votes
0answers
44 views

Run a regression with data from different measurements

I have a population $N$, that can be divided into several samples. 1 sample, $S$, was taken out from $N$, let's say a piece of paper from a page of a manuscript; this piece of paper is divided into ...
0
votes
0answers
1k views

What is subsampling?

The sample I have is constituted of 100 subsamples. In addition, each subsample is nationally representative and preserves any stratification of the sample from which it is drawn. What does this mean?...
0
votes
0answers
95 views

Help in getting the correct values in the sequential tests of hypothesis in ANOVA in R

Say, I wanted to compare the effect of $t=3$ (8 hours, 12 hours, 16 hours) lengths of exposures to sun on plant growth. I randomly applied these 3 lengths of exposures to $r=3$ pots but let us say ...