Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution.

learn more… | top users | synonyms

0
votes
0answers
8 views

Surveying the whole sample space

Are there instances when surveying just a sample of the sample space is better than surveying the whole sample space?
0
votes
0answers
8 views

Sampling from Study Population for Labeling

I have the opportunity to get labels for a portion of my study population, and I can determine the subjects that will receive a label. Every subject in the population can be targeted. The labels are ...
0
votes
1answer
12 views

Estimating population mean when using biased sampling?

Say you had a sample of 50 items that was constructed by taking selecting values that were in the top 75% (not in the first quartile), and you had to take 61 items to find those 50 (11 were in the ...
1
vote
1answer
36 views

Method for constructing large data set from smaller data set?

Is there a method for creating a large data set from a smaller one? I have a data set of anthropometric variables (e.g. stature, leg length, arm length and so on) So I have 7 variables and 1774 ...
0
votes
0answers
3 views

Statistics sampling on over-lapping categories

I got a finite population. (e.g. a pool of files) The objective of the audit is to determine whether the files are in order. I need to audit the files for 5 different category on the population. Each ...
1
vote
0answers
18 views

Rigorous definitions of sample and population

I am trying to understand some basic ideas of econometrics (and mathematical statistics) from the precise, mathematical point of view, avoiding vague explanations. I am beginning to learn about these ...
2
votes
0answers
11 views

Normalize(Scale) data before sampling or after sampling in binary classification?

I have a binary classification database with imbalance outputs (1 labeled data: 1400 samples, 0 labeled data: 200 samples). I balance data based on a criteria to (200 - 200). Where should I normalize ...
1
vote
1answer
64 views

Why do we need confidence intervals?

I am following a video lecture on Statistics, which introduces the concept of confidence intervals in the following way: "A bank vice president is interested in the average checking account balance ...
0
votes
0answers
18 views

Time-partitions of sample size

I am struggling with explain something I read in a Whitepaper. The essence is as follows. Let's begin with a random variable $X$ defined as number of events in an hours. Further, we assume that $X ...
0
votes
0answers
18 views

Different types of dependent observations [on hold]

Say we want to estimate the mean height of people in population X. If we collect 50 observations from the same person, then they would be dependent. In this case, we would expect the type of ...
0
votes
0answers
26 views

Sampling massive dataset [on hold]

I have a massive dataset (100s of millions of rows and 100s of dimensions). The dimensions are of all conceivable data types. I need to sample the data for predicting a response. How do I arrive at ...
3
votes
0answers
23 views

What would be an efficient way of representing a known discrete distribution with a number of samples?

So, say that I have a known discrete probability distribution, for example: [1] [2] [3] [4] [5] 0.08072 0.0642 0.2853 0.3206 0.2492 What I want to ...
0
votes
0answers
13 views

Stratified Sampling in 1 dimension

I was planning on implementing a stratified sampling (importance sampling) technique in one dimension. My understanding is that certain procedures need to be followed in determining the partition and ...
0
votes
0answers
7 views

Aggregating Equity Intraday Ticks

I am currently working with intraday equity data. The ticks are sourced from Bloomberg API. Bloomberg only timestamps down to the second (not millisecond) and data is not in order. In many instances ...
0
votes
1answer
66 views

Calculating a p-value from permutations

I have 500 genes I am interested in. I want to test whether they are enriched in a certain dataset (Y). Y is a list of cancer genes that are called cancer genes at different levels of likelihood. I.e. ...
6
votes
0answers
74 views

Bayesian inference with sampling and mixture models

I'm having some trouble doing Bayesian inference on an experience I have in hands. I apologize in advance if it is too complex, but I couldn't find a trivial way to split it in several parts. Let ...
0
votes
0answers
41 views

GridWalk sampler with pymc3

I am trying to build a GridWalk sampler (actually PolicyWalk as in BIRL by Ramachandran et. al.. Edit More info: The distribution I am interested in is the reward posterior given by, $$ P(R|D) = ...
3
votes
1answer
35 views

Weighted sampling as a way to eliminate specific source of variation?

I am facing a problem of predicting probability of an event given two correlated predictors where only one of them is of interest. Thus, I’m trying to eliminate one of them from the model while making ...
0
votes
0answers
20 views

Using MARS for imbalanced data

I’m working on my thesis on classification for child labor using decision tree C5.0 algorithm compare with multivariate adaptive regression spline (MARS). I have imbalanced data for child labor (total ...
4
votes
4answers
90 views

Standard deviation vs standard error of the mean for intervals

I couldn't find a question like this anywhere on Cross Validated. Also I struggled to write the title for this question. Imagine I have a satellite picture of a straight road, and I expect that the ...
0
votes
0answers
8 views

Explanations about the parallel tempering

I am reading this paper on parallel tempering but there are a few things I do not really understand. If I'm not mistaken, parallel tempering is a MCMC method which is quite convenient to sample from ...
0
votes
1answer
31 views

Adaptive Rejection Sampling in python?

Adaptive Rejection Sampling is a sampling technique for uni-dimensional variables that takes profit of the log-concavity of the probability density. It is used, for instance, in Gibbs sampling, when ...
1
vote
0answers
9 views

how to calculate a good quantity of samples for this situation?

I have developed a method for performing steganalysis into pictures. Steganalysis consist in the inverse procedure of Steganography, so what it does is to uncover hidden information into a picture or ...
2
votes
1answer
23 views

Probability of a deleterious mutation, given an observed distribution of mutations

This problem has been driving me crazy for days, so I finally resort to asking for help here: Consider a set of genes, where each gene consists of a sequence of letters, for example ...
0
votes
1answer
47 views

Sample size for categorical data

I have a population of phone calls - 200,000. There are different reasons for each call, but lets assume the number of reasons is known. i.e. 7 different call reasons: 1) Check on order 2) Cancel ...
1
vote
0answers
22 views

Comparing logistic regression models b/w non-independent samples

I have run a fully adjusted logistic regression model for a sample whose data was collected in the year 2000. The outcome is retirement, and the predictors are individual characteristics, like age, ...
0
votes
0answers
15 views

Random Sampling Techniques

I have a pool of people who each have at least one email associated to him/her. Two people can also share a single email. This results in clusters of people chained together by email:person units. ...
2
votes
1answer
48 views

Confusion about the sample distribution.. Can you please enlighten me?

I thought that the sample distribution was an approximation of the distribution of the underlying phenomenon. But then the book says: We will denote the sample size by $n$ ($n \le N$) and the ...
0
votes
0answers
5 views

Prototype selection methods for down sampling

I am looking for some methods for down sampling using prototype selection. I am guessing it is based on nearest neighbors, but I do not know what exact steps they are. Is there any good paper for ...
1
vote
0answers
14 views

Sampling of a real time-correlated random variable

I would like to sample numerically the real-value time-continuous random variable $X(t)$, which verifies: $$E[X(t)X(s)]=f(t-s).$$ In my case, $f$ is given by $$f(t)=\frac{1-t^2}{1+t^2}.$$ -My first ...
2
votes
6answers
91 views

How to prevent collinearity?

Ieno & Zuur 2015 describe a number of causes of collinearity among explanatory variables entered into a linear regression. One of these causes is what they call a 'data collection' cause. They ...
0
votes
0answers
12 views

scoring/predicting for new observations

I have two data sets of variables where one of them - the new observations - has no dependent variable. The data set without a dependent variable has around 20 times the number of records. ...
2
votes
1answer
61 views

Example of sample $X_1,X_2,\ldots,X_n$

In the book Statistical Inference by George Casella, it is written that An experimenter uses the information in a sample $X_1,X_2,\ldots,X_n$ to make inferences about an unknown parameter $\theta$. ...
2
votes
0answers
61 views

How many people initially had apples?

Story problem: Assume 10 apples are distributed across $X$ unknown people, where each person has at least one apple. For each apple a biased coin is flipped to see if that apple should be kept or ...
2
votes
1answer
55 views

How does upsampling rare events affect the interpretation of logistic regression?

The answer to this question can be found here Does down-sampling change logistic regression coefficients? I have a dataset of 50k positives and nearly 1M negatives. Instead of taking a random sample ...
4
votes
1answer
41 views

How to make an effective sampling from a database of text documents?

Problem: I want to know methods to perform an effective sampling from a database. The size of the database is about 250K text documents and in this case each text ...
1
vote
3answers
88 views

Why take multiple samples from a population?

I'm fairly new to Econometrics, and I'm curious about, when trying to estimate the population parameter, why it is important to take multiple samples of the population, instead of just combining all ...
0
votes
0answers
21 views

Montecarlo analysis: how many iterations I need?

I am working on a Montecarlo analysis. I have a transfer function from R^m-->R (i.e. m inputs to one single output), whose I do not know deterministically the m inputs. So I generate N random values ...
1
vote
0answers
19 views

Sampling of groceries

I need some statistical advice. I am planning a marketing research to test if gluten free food is more expensive than regular food. I know it is, I want to show just how much more expensive it is. I ...
1
vote
1answer
34 views

Data splitting in regression

Edit: The key point I'm attempting to understand is whether during a regression model building exercise, do I need separate datasets to: search for predictors and settle on a functional form ...
1
vote
1answer
35 views

Searching for signal in a somewhat noisy landscape

I'm trying to implement a system to identify regions of high signal content in the presence of a noisy landscape. By high signal content, I simply mean intensity values significantly above background ...
2
votes
2answers
113 views

How to draw a representative sample from a biased portion of a sampling frame?

I have a question concerning sampling. We are planning a study in which we aim to draw conclusions about the relationship between different sources of research funding and academic impact (measured ...
2
votes
2answers
161 views

Given a series of numbers, generate new numbers that are “similar” to the original numbers

I'm a programmer but I don't know much about stats, so please excuse me if this question sounds naive. I have two computers that are in different locations. I have made N measurements for the ...
1
vote
1answer
15 views

sample from a mixture

Suppose I have two types of students, male or female. Suppose a test score of a male student follows a distribution $F_m$ and suppose a test score of a female students follows a distribution $F_f$. ...
1
vote
0answers
24 views

On sampling from all the continuous functions in [0,1]

I was watching this youtube video for motivation on measure theory and at the 7:20 mark a reason for utilizing measure theory is given; to be able to pick a random function from all the continuous ...
1
vote
0answers
28 views

Is sampling according to several i.i.d. random variables or to just one of them equivalent

Suppose we have n+1 random variables i.i.d. distributed $X_0,X_1,...X_{n}$. Is it the same operation if I generate n samples according to $X_0$ $\{S^0_1,S^0_2..,S^0_n\}$, or if I take from each of the ...
1
vote
1answer
47 views

How to emphasize on specific data points in Linear Regression?

I'm now solving linear regression problems. $y = wx + b + e$ So I have $(x, y)$ data set and want to learn weights $w, b$. Additionally I know that certain data points are not polluted by noise ...
0
votes
1answer
50 views

Bootstrap and MonteCarlo Method

I am trying to make sense of the bootstrap method. I am studying on Rice, "mathematical statistics and data analysis" Here it is its explanation of the bootstrap method: Imagine for the moment ...
0
votes
0answers
20 views

Subset items from a group while controlling for variables between groups

Can anyone offer solutions to the following problem?: I'm trying to extract a subset of items from multiple groups (grouped by 2 separate factors) so that the final groups are: 1) of equal size, and ...
2
votes
2answers
38 views

data are drawn from a probability distribution P?

what does it mean to say data are drawn from a probability distribution P?