Questions tagged [sample]

A sample is a subset of a population. Statistics, in general, is concerned with using samples to make inference about the parameters governing a larger (possibly infinite) population.

Filter by
Sorted by
Tagged with
102
votes
25answers
41k views

Locating freely available data samples

I've been working on a new method for analyzing and parsing datasets to identify and isolate subgroups of a population without foreknowledge of any subgroup's characteristics. While the method works ...
41
votes
5answers
387k views

What is the difference between a population and a sample?

What is the difference between a population and a sample? What common variables and statistics are used for each one, and how do those relate to each other?
28
votes
3answers
7k views

What if your random sample is clearly not representative?

What if you take a random sample and you can see it is clearly not representative, as in a recent question. For example, what if the population distribution is supposed to be symmetric around 0 and ...
26
votes
7answers
4k views

Do we need hypothesis testing when we have all the population?

From what I understand hypothesis testing is done to identify if a finding in the sample population is statistically significant. But if I have a census data, do we really need hypothesis testings? I ...
22
votes
9answers
6k views

How do I figure out what kind of distribution represents this data on ping response times?

I've sampled a real world process, network ping times. The "round-trip-time" is measured in milliseconds. Results are plotted in a histogram: Ping times have a minimum value, but a long upper tail. ...
17
votes
2answers
44k views

What is the difference between sample variance and sampling variance?

What is the difference between sample variance and sampling variance? They seem same. Aren't they?
16
votes
2answers
7k views

What is the difference between random variable and random sample?

These two expressions confused me a lot when I was learning statistics. It seems to me that they are totally different things. A random sample is to randomly take a sample from a population, whereas ...
15
votes
1answer
4k views

Large sample asymptotic/theory - Why to care about?

I hope that this question does not get marked "as too general" and hope a discussion gets started that benefits all. In statistics, we spend a lot of time learning large sample theories. We are ...
15
votes
3answers
4k views

Bootstrap: the issue of overfitting

Suppose one performs the so-called non-parametric bootstrap by drawing $B$ samples of size $n$ each from the original $n$ observations with replacement. I believe this procedure is equivalent to ...
14
votes
2answers
12k views

How do sample weights work in classification models?

What does it mean to provide weights to each sample in a classification algorithm? How does a classification algorithm (eg. Logistic regression, SVM) use weights to give more emphasis to certain ...
13
votes
4answers
751 views

Is any quantitative property of the population a “parameter”?

I'm relatively familiar with the distinction between the terms statistic and parameter. I see a statistic as the value obtained from applying a function to the sample data. However, most examples of ...
13
votes
3answers
2k views

Using mean length and mean weight to calculate mean BMI?

Is it valid to use mean length ($h$) and mean weight ($w$)) from a given population to calculate mean Body Mass Index ($BMI = \frac{w}{h^2}$) for that population?
13
votes
2answers
1k views

Are Kaggle competitions just won by chance?

Kaggle competitions determine final rankings based on a held-out test set. A held-out test set is a sample; it may not be representative of the population being modeled. Since each submission is ...
12
votes
6answers
2k views

Is the mean of samples still a valid sample?

Suppose I sample $n$ times from a distribution $$ x_1, \ldots, x_n \sim p_\theta(x) $$ is the mean of the samples always a valid sample from the target distribution? I.e. is $\overline{x}$ a valid ...
12
votes
4answers
7k views

How to take many samples of 10 from a large list, without replacement overall

I've got a large set of data (20,000 data points), from which I want to take repeated samples of 10 data points. However, once I've picked those 10 data points, I want them to not be picked again. I'...
12
votes
5answers
3k views

If not a Poisson, then what distribution is this?

I have a data set containing the number of actions performed by individuals over the course of 7 days. The specific action shouldn't be relevant for this question. Here are some descriptive statistics ...
11
votes
2answers
1k views

How can the central limit theorem hold for distributions which have limits on the random variable?

I've always taken issue with, and never been given a good answer, for how it is possible that the central limit theorem - the classical version where the distribution of sample means approaches ...
11
votes
1answer
4k views

Estimate variance of a population if population mean is known

I know that we use $\frac1{n-1}\sum\limits_i(x_i - \bar{x})^2$ to estimate the variance of a population. I remember a video from Khan Academy where the intuition given was that our estimated mean is ...
11
votes
1answer
1k views

Does SurveyMonkey ignore the fact that you get a non-random sample?

SurveyMonkey has steps and a chart for you to figure out what sample size you need for a given margin of error or confidence interval, based on your population size. SurveyMonkey sample size Does ...
10
votes
1answer
3k views

Why doesn't a sample proportion also have a binomial distribution

In a binomial setting, the random variable, X, that gives the number of successes is binomially distributed. The sample proportion can then be calculated as $\frac{X}{n}$ where $n$ is your sample size....
10
votes
2answers
1k views

Estimating the size of an intersection of multiple sets by using a sample of one set

I'm working on an algorithm which needs to calculate the size of a set generated by the intersections of at least 2 sets. More specifically: $$ z = \left |A_0 \cap \ldots \cap A_n \right | $$ The ...
9
votes
4answers
2k views

Calculating ratio of sample data used for model fitting/training and validation

Provided a sample size "N" that I plan on using to forecast data. What are some of the ways to subdivide the data so that I use some of it to establish a model, and the remainder data to validate the ...
9
votes
1answer
609 views

Consequences of exceeding sample size after determination of sample in power analysis

Challenge In our office, I have been involved in a discussion about sample size and its influence on effect size - can you help me out and explain further? Base When conducting a power analysis, ...
9
votes
1answer
238 views

How to define what a “sample” is?

If I give you three numbers that are independently and identically drawn from a standard normal distribution, then have I given you three samples or one sample? If the answer is one sample, then is ...
9
votes
2answers
1k views

Consistency of sample quantile

I'm new to statistics and recently I learnt about non-parametric estimation. On the estimation of empirical quantiles, many notes talk about the asymptotic behavior of sample quantiles. The proof is ...
8
votes
4answers
1k views

Detect if there is actually two populations in a sample

I have been counting stomata on fossil leaf material to apply a known relationship between stomatal index and CO2. I thought that the material was all from one population (one species at a given site)....
8
votes
2answers
5k views

How to validate if a sample is independent and identically distributed

How can I check if the data is drawn i.i.d. from an unknown multivariate distribution?
8
votes
3answers
210 views

Clarification on the definition of “population”

In stats classes, I've always learned that a population is always a very broad, almost unquantifiable group (e.g., all voters in a country, all consumers of a company, all viewers of a TV channel), ...
8
votes
2answers
2k views

Subsample of a random sample: random sample?

Let's say you have a large random sample of soccer players in Europe but you are only interested in what happens in Spain. Could you reduce your sample to players in Spain and still call it a random ...
7
votes
3answers
3k views

Correcting biased survey results

Knowing that a population sample (non-random) is biased in terms of its demographics, what are the best practices to correct for this issue? That is, let's say that I can attach an array of ...
7
votes
3answers
1k views

Non-random small sample - What conclusions can I get?

For my thesis I developed a psychotherapeutic method to address social competence in aggressive/withdrawn children. I have obtained a sample of 10 subjects (not random, the professors chose the ...
7
votes
1answer
4k views

What is the distribution of the sample variance for a Poisson random variable?

The mean and variance of a Poisson random variable $X$ are both $\lambda$ but what is the distribution of the $\operatorname{var} X$ across a series of experiments recalculating each time? I would ...
7
votes
1answer
4k views

Sampling 100000 times from normal distribution in R : strange distribution of samples' standard deviation

I generated, in R, one hundred thousand random samples of ten values from the normal distribution with mean zero and unit standard deviation, and registered each mean and standard deviation, in hope ...
7
votes
2answers
10k views

Check if sample is representative of a larger sample

From a larger sample of tabular data, I have picked certain rows that meet a certain condition (this condition is unrelated to the actual data in the rows). Now, I want to know if the distribution of ...
7
votes
2answers
2k views

Sample mean and variance independence in the case of correlated observations

Are the sample mean and sample variance of correlated normal observations independent? The classic theory relies on the independence of observations, but this is not the case. So as an example I ...
7
votes
3answers
325 views

Generating a min{p, 0.5} coin from a p-coin - Bernoulli factory type problem

Suppose we are given a coin with arbitrary (unknown) head probability $p$, I am wondering if there is an easy-to-implement algorithm for generating a $\min\{p, 0.5\}$ coin for any $p\in [0,1]$. ...
7
votes
1answer
2k views

Prove that distribution of sample median (for even sample) is symmetric

Let $\mathbf{X}=(X_1,X_2,\ldots,X_{2k})$ be a sample from normal distribution $\mathcal{N}(0,1)$. Prove that distribution of sample median (for even sample) is symmetric around 0. $me(\mathbf{X}) = \...
7
votes
1answer
1k views

Bounds for the population variance?

Suppose we have i.i.d. samples $x_1$, $\ldots$, $x_n$ for a (potentially non-normal) random variable $X$ with finite moments. We can use these samples to construct an unbiased estimates of the ...
7
votes
3answers
183 views

Enlarging a random sample

In our project we have a population of 1000+ individuals. We picked a random sample of 107 individuals, but then we realized we needed more precision, so now we want to have a larger sample. The ...
7
votes
1answer
287 views

Generalizing results - what is the population?

Take as an example the well-known Boston housing dataset. In the original paper, the regression model was used to model the amount people were willing to pay for improved air quality. But the 'people'...
7
votes
2answers
1k views

Small n: non-parametric or parametric tests?

I am in a situation where I have sample with $n=19$, and the the data are not normally distributed (according to the results of a Kolmogorov-Smirnov test) for all the variables. The data were ...
6
votes
2answers
25k views

Whether to use r-square or adjusted r-square with a small sample size that may represent the entire population?

I read online that it is only necessary to use adjusted-$R^2$ when you are working with a sample rather than the entire population. The data I'm working with is information on a series of live ...
6
votes
2answers
821 views

About Sampling and Random Variables

So I've recently started an introductory course in econometrics and I'm having trouble grasping the idea of Random Variables and Sample distributions If we have a population and we take a sample $Y= \...
6
votes
2answers
998 views

When does one use $\frac{1}{\sqrt{n}}$ and when does one use $1.96\sqrt{\frac{p(1-p)}{n}}$?

I'm not sure when to use $\frac{1}{\sqrt{n}}$ and when to use $1.96\sqrt{\frac{p(1-p)}{n}}$. Are they both used to calculate the confidence interval at 95% for a population proportion? Here are two ...
6
votes
3answers
174 views

Overview of predictive modelling, machine learning, etc.

My text Intuitive Biostatistics is a nonmathematical explanation of conventional statistics. Chapter 3 explains the basic mindset of statistics as analyzing a sample to make inferences from a ...
6
votes
1answer
754 views

Comparison of two means

Which test should I use, if I want to compare the mean of the whole sample with the mean of the fraction of the sample? So for example the mean of the whole classroom taking the test and the mean of ...
6
votes
1answer
4k views

Exercise on Chebyshev inequality compared to the Central Limit Theorem

Problem Take this (easy) problem as an example: An astronomer is interested in measuring the distance, in light-years, from his observatory to a distant star. Although the astronomer has a measuring ...
6
votes
1answer
164 views

Compare maxima of two Gaussian samples

Suppose $X$ and $Y$ are both normally distributed, with $X \sim \mathcal{N}(0,1)$ and $Y \sim \mathcal{N}(c,1),$ where $c > 0$. Consider $n$ independent draws of both $X$ and $Y$. As $n \rightarrow ...
6
votes
1answer
456 views

Does the Central Limit Theorem hold if you replace the true variance with the sample variance?

I'm currently doing a self-study of Wasserman's All of Statistics. Theorem 5.10 states that for $X_1, ..., X_n$ IID with mean $\mu$ and variance $\sigma^2$, we have $\frac{\sqrt{n}(\bar{X}_n - \mu)}{...
6
votes
1answer
2k views

Problems generating a sample from a custom distribution with log

I'm trying to generate a sample from a family of distributions. In particular I would like to be able to obtain a sample from the survival function: $$1-F(x) = c x^{-a} \log^b(x)$$ with a proper ...

1
2 3 4 5
18