# Questions tagged [small-sample]

Refers to statistical complications or problems due to having few data. If your question is about a small sample relative to the number of variables, please use the [underdetermined] tag instead.

620 questions
Filter by
Sorted by
Tagged with
7 views

### What methods can I used to assign cases to groups when some variables are dependant on others?

I have a sample size of 42 cases, with about 5 variables for each case. Most of these variables are measurements, and I want to assign the cases into 2 groups (condition present, and condition absent) ...
70 views

### Kruskal-Wallis test on data with heterogeneous variance and small sample sizes per group

I've been trying to figure out how to test (in R) if there are significant differences between the group means of my data because it seems to violate the assumptions of tests that do this (ANOVA, ...
• 691
27 views

### Modification of Outliers

I have a practical / applied statistics question. I'm dealing with a specialized dataset with a very small sample (i.e. n < 10). In the sequence of observations, it is possible that a new ...
70 views

### How to deal with very little data

I thought I understood the train/validation split basics but this question got me confused. I could acquire very little data. I don't have it already, which is why I cannot just experiment with it. I ...
• 41
1 vote
23 views

### Using Cramér's V to evaluate correlation between categorial variables with small numbers within contingency tables

I am evaluating correlations between categorical variables in a data set. I have seen Cramér's V is one way I can calculate this, and I know it takes the chi-squared statistic as one of the elements ...
• 11
27 views

### special form of a paired design (very small, unbalaned)

I am trying to figure out how to analyze following design. Suppose three samples are taken from the starting material of an experiment which will be performed at larger scale (LS). The same ...
• 131
69 views

### Should stepwise regressions or overfitting also be avoided for exploratory (hypothesis generating) modelling?

In a recent paper, Andrew Tredennick and colleagues (2021) suggested to use the drop1() function in R for exploratory modelling (that is to generate new hypotheses ...
• 383
8 views

### SMALL SAMPLE SIZE: Can i use it for a validation study? What evidence supports this?

I would like to validate submaximal exercise tests; I will correlate the predicted maximum oxygen consumption values using the field tests (values will be derived from standard equation) with the ...
• 1
1 vote
36 views

### How to correct for unequal sample size per group in neg. binomial regression

My data contains information on the time, location, and present conditions of a fish catch. The problem we often have in the field is that not all locations can be consistently visited every time. ...
• 691
1 vote
47 views

### one-sample t-test on zero variance sample

I have to compare many gene-expression values between a group of 3 patients and a control patient. I choose to apply the one sample t-test (ttest_1samp from python <...
33 views

### Wide prediction intervals for short time series: how to fix that?

I have very small time series data(24 points) for sales for different-different regions. I need to build Range Forecast (Confidence/Credible Intervals) for sales around it for every region. I tried FB'...
21 views

### What statistical test of significance should I use when comparing two small, unequal groups of non-normal distribution?

I am trying to analyze quantitative data between two independent groups. One group has 8 data points, and the other has 9. I used a Shapiro-Wilk test for each of the groups to determine normality, and ...
• 21
41 views

### How to handle generate more data in small data-sets for regression?

I have a logistic regression and decision tree classifiation model with a dataset with 70 data entries, yet only 16 responses have said yes for the response variable. I have 8 covariates. How can i ...
• 1
46 views

### How to find an optimal sample size (or range) based on sampling errors?

Intro to problem Increasing the sample size can be very costly in terms of time and money. However, in experiments where we measure proportions within a sample, errors associated with the sample size ...
32 views

### Why is hinge loss suitable for small training data sets?

In this blog post it says "Hinge loss is one such example which makes training with small dataset possible." Why does that hold true? So far, I could not come up with a convincing ...
• 101
31 views

### Small N analysis for a field experiment question, and rare events logit

so, I am running a "Field" experiment, meaning, people in a real life situation are subject to certain treatment, and I am trying to measure the effect of said treatment. In this case its an ...
• 3
34 views

### Do poisson regression and exact poisson regression only differ in confidence interval?

The point estimates from the two approaches are the same, right? I plan to use exact Poisson regression for a study, since the events could be sparse after adjusting for a few covariates. Is there ...
• 475
16 views

### Final model training after cross validation for a regularized neural network

I have a regression problem with a relatively small number of available samples. In order to select the best model and tune the hyperparameters, I am using nested cross-validation (NCV). The common ...
1 vote
15 views

### Formula for Leemis's m

I'm looking for the formula to calculate the statistics and the significance level of Leemis's m. Wikipedia reports the statistics as m = \sqrt{N}\cdot\max_{i=1}^9 \left\{ \left|\Pr (X \text{ has ...
• 223
86 views

### What's the correct tool to compare means between very small groups?

I'm not really used to VERY small samples, so I'm wondering what to do in the following situation: I have 3 groups, A, B and C. Each of them consists of 3 observations. I want to test whether there is ...
• 39
17 views

### Small Sample - Confidence Interval Mean

I have a sample size of 14 and I plan to measure confidence intervals for the mean of different characteristics. I can say this about my data: ...
• 11
20 views

### Using dimensionality reduction to control for more variables in a multiple linear regression without breaking the 15:1 rule - possible?

I'm interested in whether there is a relationship between my particular variable of interest (X) and my dependent variable (Y). I'm planning to explore this using multiple linear regression. My focus ...
• 11
10 views

### Estimate minimum sample size for a rolling time-serie percentage

I have to calculate the conversion rate for a web page, defined as CVR = orders / clicks This is an e-commerce web page, so clicks arrives to the web page ...
8 views

### How can survey data for a small sample be analysed?

A survey was conducted among companies regarding the prevailance of a technological device in their company. It collected categorical data and received 40 responses (from a population of 520). As the ...
10 views

### What is the degree of cell sparsity that Poisson model can tolerate?

For models with no interaction terms, my understanding is that all the marginal cells need to have non-zero counts, in order to have finite estimates. If this minimum requirement is satisfied, will a ...
• 475
1 vote
40 views

### Is it better to stratify or adjust confounders in the model with small sample size?

My questions is for Poisson models and Cox models. The context is rare/small number of events. We want to adjust for a few categorical confounders: sex, age group, baseline risk, but are not ...
• 475
1 vote
51 views

### Will stratified log-rank test work if there is no event in some small strata?

Since the analysis is for a rare event, it is possible that some small strata have no event. Hard to predict. Just wondering if there will be any problem. Cox PH model will have problem for sure, but ...
• 475
9 views

### How to achieve good result in text classification when the data is very small?

I have a dataset with many user comments, I want to classify this dataset with label 0 or 1. The dataset only has 730 comments labeled as 0 and 65 labeled as 1. I have developed a simples model using ...
• 163
78 views

### Is post-variable-selection multimodel inference a bad idea?

If I understood correctly, in this answer, Ben Bolker says that using inferential methods after having performed AIC-based model selection is wrong because "standard inferential methods assume ...
• 383
27 views

### Designing an experiment to test before vs. after unsuccessful payment percentage

I'm working at an online payment processor, which is used by multiple vendors (kind of like paypal). The experiment that I'm working on involves just one vendor. I have 100k data points from that ...
• 161
1 vote
65 views

### Adjust for covariates in small sample size (IPTW, PS match, etc.)

I have a dataset of patients with a grouping variable (groups A (control) and group B (treatment)). The two groups have sample sizes of 170 vs. 30. I would like to compare outcomes between the two ...
33 views

### Choosing best hyperparameters for multiple regression when number of features is higher than number of samples

I am a chemist mostly, and I do not have much experience in statistical learning. However, I am currently starting work on a problem that requires multiple regression. I have a set of molecules, for ...
• 153
28 views

### How to find p value when comparing small amounts of data?

I am comparing shorebird populations and need to determine if they are statistically different across two beaches. I ran a Kruskal Wallis test on another set of data and got good p values but since my ...
1 vote
32 views

### Causal Inference Short Time Series

I am trying to analyse causal inference associated with an intervenion using either Difference-in-Differences or Interrupted Time Series Analysis. I have a discrete time series consisting of data ...
116 views

### HAC standard errors: small-sample correction

The Python package statsmodels provides a use_correction option when computing HAC standard errors for an OLS model, which ...
• 252
17 views

### Compare models with different number of observations

I have a trainingset of molecules with known toxicity property: it can be a numerical value or a class (e.g., toxic / non-toxic). Let's focus on the classification for now. Analyzing the trainingset, ...
• 101
1 vote
220 views

### Can Bayesian Models be used to Compensate for Small Datasets?

I have the following question: Can Bayesian Models be used to Compensate for Small Datasets? Suppose we have a linear regression problem (e.g. predict the age of giraffes based on their height and ...
• 5,688
191 views

### Is the Conover-Iman post-hoc test is better to use than Dunn's with small samples size after the KW test since it use the t-test on the ranks?

I know that we use the t-distribution to be able to know if two small-size samples with a normal distribution (n < 30) are different or not (Student's t-test). If we have a bigger sample with a ...
18 views

### Unbalanced classes, bootstrap roc-auc-score undefined

Setting I have a small dataset with unbalanced classes (N=50 vs N=10) and currently report the roc-auc-score as a classification metric of my model. In addition to the point estimate of the roc-auc-...
• 239
77 views

### How to calculate the confidence interval of a mean on web search findability

I have 30x 1-hour sessions where I assign one random person to each session to search for a specific type of content on a social media website. I record the number of pieces found in each session, ...
• 123
1 vote
53 views

### Can a stochastic optimization in place of deterministic optimization help us dealing with small sample data set?

So there is a radioactive point source and it is traversing at constant velocity though a medium. It emits two gamma at same time (called annihilation gamma of energy 511 keV). Detector detects two ...
• 35
1 vote
19 views

### Estimating the size of a population. Is the method employed valid?

Context: in North America there has been a Drug Overdose Epidemic for several years. In my own municipality, more people have died of overdose than of COVID during the pandemic. My workplace (a non ...
• 400
341 views

### Simple comparison of two Poisson means in R?

I have two samples and I would like to determine whether the difference between them is statistically significant or not: Because this is clearly data of small counts that cannot be approximated by a ...
• 125
23 views

I need to do a content analysis of certain types of Twitter accounts. There are in total 19 of them (N = 19) but I need to take a sample due to resource constraints. I want to have 95% confidence (Z = ...
31 views

### Two-samples Z-test alternative for small samples (categorical data)

I am looking for alternatives to 2 samples Z-test (2 independent samples) for categorical variables. I have two datasets (same indicators measured in 2020 and 2021) I need to understand whether ...
54 views

### Why does the prediction of a VAR dgp diverge from the test set?

I'm working on a multivariate data set consisting in 44 observations (which have to be splitted: the first 34 observations are in the training set, the remaining ones in the testing set) of 9 ...
45 views

### Appropriate forecasting methods for only 20 observations [duplicate]

I am trying to forecast the regional GDP growth of our region in the next five years, I only have 20 observations in my data which is yearly, what forecasting model is appropriate? I tried ARIMA in r ...
581 views

### What is reasonable to do with small -tiny- datasets?. Dealing with rare diseases

I come up from posts like What to do with a small (27) medical dataset? because they are fairly similar. But in a broader context, in rare diseases/studies with a really low sample size -but plenty of ...