Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [randomness]

Randomness is the lack of pattern or predictability in events. Randomness is often modeled with probability distributions, but can also be generated by deterministic processes.

2
votes
1answer
20 views

Measure of randomness that maps well to “hard to guess”?

It was recently pointed out to me that entropy does not necessarily map well to "hard to guess". E.g.: consider a distribution where there's a very high (say, 90%) chance you select a single, ...
0
votes
2answers
49 views

Is there a way to check for sure that optimization was applied to the weights of NN?

Given the first or next steps of NN training, where weights were initially randomly sampled from some distribution, is there a reliable way to determine if some optimization was applied to the weights?...
1
vote
1answer
39 views

Classification accuracy heavily depends on random seed

I want to compare different classification methods and evaluate their prediction measures (such as accuracy etc). I first split the data into training and test set. With the training data I then ...
0
votes
1answer
28 views

LSTM - random and always-different time between data measurements?

I am working with a time series problem where the time between two data measurements is random, and I am trying, without luck, to find an LSTM architecture that can handle this. A very simplified ...
0
votes
0answers
30 views

Testing Randomness in day of year values

I have airline booking data (no. of bookings vs day of year). Day of year is a vector, taking values from 1 to 365, with stepwise increments of 1. We can have one data-set where booking exists for ...
0
votes
1answer
43 views

Stochastic vs Adversarial Multi-Armed Bandit Problems

I know that the multi-armed bandit can be formalised in multiple ways - two of them being the stochastic and adversarial ways. I am familiar with the fact that adversarial way is a game theoretic ...
0
votes
1answer
60 views

The assumptions regarding the t-test

The t-test has an assumption that the sample provided has to be random in nature. Do we test our sample for randomness before carrying out the t-test ?
0
votes
0answers
20 views

Explain the results of the 2-sample Kolmogorov-Smirnov test for two random Gaussian sets of numbers

Suppose I have two sets of random numbers drawn from the Gaussian distribution, $\mathbf{r_1}$ and $\mathbf{r_2}$. I then do a 2-sample Kolmogorov-Smirnov test (KS test) between the two to determine ...
3
votes
3answers
210 views

How do I prove one company is actually doing better than another?

Suppose I have the return information for two reverse logistic companies--See example data. For certain groups companies A and B each receive 50% of the work from a client, but for another group ...
1
vote
0answers
17 views

Random sampling at two different frequencies

In a production line, random samples are drawn to be tested for properties A and B. A has to be tested at a higher cadence than B and the assumption is that A and B have zero correlation. Since the ...
0
votes
1answer
22 views

Cross checking AI Selections vs Random Chance

Fifty Three Images AI Selects 20 images based on various criteria. I Select 20 images based on personal criteria. In a two group selection of all 53 items what is the likely overlap due to random ...
0
votes
0answers
28 views

How to do Monte Carlo with conditional input values?

Please apologise my likely ignorance of the correct terminology and notation. Any edits and suggestions to improve the question are very much appreciated. I want to perform a Monte Carlo simulation ...
7
votes
4answers
3k views

Check if a character string is not random

Background Let's say we have an alphabet of A,B, C, D, then we look through some data and find a "word" which is DDDDDDDDCDDDDDD ...
1
vote
0answers
22 views

Is there a known PRNG that passes TestU01's BigCrunch but fails any test in NIST 800-22?

In other words, does TestU01 incorporate all statistical tests by NIST 800-22?
0
votes
0answers
84 views

Difference between Random Forest and Random Subspaces/Patches

When fitting a Random Forest model, a subset of the features is randomly considered at the splitting of each node. E.g., if $p$ is the number of features, then at each node in each tree, $\sqrt{p}$ ...
6
votes
2answers
1k views

What is the state of the art in statistics tests for distinguishing good from bad random number generators?

There are many packages out there. In particular, PractRand gives out an opinion on a number of them, but it's only an opinion. Is there conventional wisdom about which set of set of statistics ...
2
votes
1answer
83 views

Why are there different statistics tests in NIST SP 800-90B and NIST SP 800-22? [closed]

NIST SP 800-22 (published in 2010) lists out some statistical tests for telling good random bit generators from bad ones. NIST SP 800-90B (published in 2018) has apparently the same purpose but lists ...
1
vote
0answers
15 views

Sample proportion of 30 independent binary random processes [closed]

Suppose you are interested in π=Probability of Heads for a particular coin. Toss this coin 30 times and record the outcomes in a sequence of H and T’s. Then find the sample proportion of Heads. Find a ...
-1
votes
2answers
204 views

Why atmospheric noise is considered the only reliable source for generating “pure random numbers”?

When we say that something is random, it simply implies that the occurrence under observation does not have any cause or at least that cause cannot be determined beforehand in layman's language. For a ...
1
vote
0answers
30 views

Goodness of fit test for observed samples of binary strings

Consider a process that produces binary strings of varying length $n$. A typical sample would include $n\ $ I Number of strings $1\ $ I $\ 3,000,000$ $2\ $ I $\ 800,000$ $3\ $ I $\ 350,...
1
vote
0answers
32 views

why LSTM word based model works better with random input seed than fixed input seed?

I have implemented an LSTM model that have 2 LSTM layers, a dropout layer and a dense layer for predictions. I trained my LSTM model on 1000 XML files. Each file has 4 main markups with very simple ...
2
votes
1answer
118 views

Find if a coin toss sequence is random

Suppose we are given a sequence of heads and tails for 1000 coin tosses. how do we check if it's actually random? I believe I need to do a hypothesis test, I tried to write some examples to get an ...
3
votes
3answers
114 views

Is it possible to calculate the probability the sum of one set of random variables equals the sum of a different set of random variables?

I'm doing data analysis at the business I work for. 23 people were asked for input for two separate lists. The first list the answers could be any integer 6 to 10. The second list the answers could be ...
1
vote
2answers
223 views

NIST Randomness Test, Testu01, Dieharder

the NIST Randomness Test, Testu01, Dieharder all seem to offer a suite of tests to check psuedo random number generators(PRNG) for randomness. I was hoping someone with experience of those tools could ...
0
votes
0answers
16 views

How to properly cross validate a model that is affected by randomness in the training procedure?

Many machine learning models are affected by some amount of randomness as part of their training procedure. For example the performance of a neural network model can be highly affected by the initial ...
0
votes
1answer
34 views

Hypothesis testing with randomised algorithm

Imagine I have some data D, and with a randomised algorithm I construct a parameter $\theta$. I know the distribution of $\theta$ under the null hypothesis of the data. For example, the original data ...
-1
votes
2answers
78 views

Dataset partitioning issue of cross-validation

Every time the cross-validation is run, the dataset is partitioned into k even groups randomly. That means every time the result of cross-validation can be different, so which one should we take to ...
2
votes
1answer
346 views

Confused about how to interpret Chi square results

For my local charity I have to organize random draws, each week to give a prize. I just bought a random machine generator, which randomly spits one ball out of 10. Before using the machine in public ...
2
votes
0answers
40 views

Randomized Block Design

It is desired to use a randomized block design with $4$ blocks of size $6$ each for testing the effects of $5$ treatments A,B,C,D and E. In each block, treatments B,C,D and E are replicated once each, ...
0
votes
1answer
20 views

Within individual independence testing

I have a group of individuals that is independent and identical distributed. Within each individual $i$, one may observe $N_i$ observations. The task is to test whether the within-individual ...
12
votes
3answers
770 views

Why do several (if not all) parametric hypothesis tests assume random sampling?

Tests like Z, t, and several others assume that the data is based on a random sampling. Why? Suppose that I'm doing experimental research, where I care much more for the internal validity than the ...
3
votes
1answer
800 views

Is PCG random number generator as good as claimed?

I have a demand for extremely fast acceptable-quality random number generator (RNG). This means rand()%2 should not alternate between 0 and 1 (that totally kills the simulation, as in one problem I ...
1
vote
1answer
225 views

How to generate randomness using a biased coin? [closed]

Imagine we have three people and we want to select one of them at random. If we had a fair coin we could have tossed the coin twice to get one of the 4 outcomes and could have related the occurrence ...
1
vote
0answers
57 views

Randomly split batches for time series CV

In certain code snippets for time series prediction, I find sometimes that authors split their data to train and test sets (nothing unusual for the moment). But then the train data is split into ...
1
vote
0answers
153 views

Generate sequence of random autocorrelated numbers

Back in the university I was working on a program that generates a sequence of random autocorrelated numbers. Autocorrelated meaning that each next value is close to the previous one and depends on it ...
-1
votes
1answer
43 views

Random sampling a matrix to reflect a real world case

I'm trying to create two matrices one that can be filled with 0's and 0.25's and another with 0's and 1's. ...
2
votes
0answers
54 views

Clarification on the “tickets in a box interpretation” for random variables

I have a question on the definition of a random variable. I have read the tickets in a box interpretation here and here, which helped me immensely, but I have some doubts for the case in which the ...
0
votes
2answers
99 views

But what is a random experiment

I am currently thinking a lot about the definition of probability and one thing that I am really not content with at all is the frequentist definition of probability as long run relative frequencies ...
0
votes
0answers
18 views

Random distribution exercise

I have the following problem to solve and I don't know when to start. Please see example below to get an idea of it. There is 5 persons (A, B, C, D, E) that need to be fed with apples for 10 days. ...
1
vote
0answers
31 views

Random sequence and uniform distribution? Is this always the case?

(Short version) Is this always the case that a truly generated random sequence (of whole numbers) conforms to (or at least expected to conform to) a uniform distribution? (Long version) The most ...
0
votes
0answers
28 views

Algo to identify next numbers in a row that looks random (but isn't) [duplicate]

I was wondering what would be your approach on building an algorithm that needs to guess the next numbers based on a given row of numbers, that seem random but maybe follow some rule - the algo is ...
3
votes
0answers
24 views

How best to form many short, independent random numbers from one long random number?

I have 20 digit, uniformly distributed random numbers, downloaded from random.org (because the numbers from that source are based on variations in atmospheric pressure rather than a mathematical ...
0
votes
0answers
22 views

Statistical tests for PRNG that generates a sequence which is not binary

I have already asked this question on crypto.stackexchange but I think here is the appropraite site for this. Is there a pratical application to PRNG that generates a sequence which is not a binary ...
3
votes
1answer
162 views

Establishing the Probability Distribution that Governs a Random Process

I have a experiment in which a random event that can occur $N$ times in some set period of time $T$ (where $N \in$ $\mathbb{N_0}$ but is typically less that 10 and never exceeds 100). Here I will ...
0
votes
0answers
141 views

Using Kalman Filter to estimate parameters in a CIR process

I am trying to estimate the parameters in a system $ds(t) = r$ $dr(t) = \beta(\alpha-r)dt + \sigma \sqrt{r}dW$ where $W$ follows a Wiener process, so $r$ follows a ...
0
votes
4answers
61 views

Probability and randomness

Assume that we toss a coin n times and the result looks like this HTHTHTHTHTHTHTHT... . Would a (Probability-) Frequentist conclude that that the probability for the coin to land heads is 0.5? It ...
0
votes
1answer
57 views

In Deep Learning, how much does randomness help generalization?

Let's assume at the beginning the NN is initialized with random weights, then the Backpropagation "shapes" the weights but the signal is strongest close to the final layer (where it gets computed) and ...
10
votes
2answers
703 views

Will this introduce bias into what should be random numbers?

Assume a data file with 80+ million ones and zeros, randomly generated. From this file, we want to create a list of random decimal integers. This is the plan to do this conversion. Divide the 80 ...
0
votes
1answer
15 views

Unsupervised learning problem: rate individual survey submissions on randomness of data entered

For my work we collect a lot of surveys which contain a mix of questions: multiple choice, checkbox and numeric. We know that in some instances enumerators might fudge the survey by entering answers ...
3
votes
1answer
59 views

Why does the law of large number hold for pseudorandom numbers

i`m wondering why the law of large numbers holds for pseudorandom numbers. Is this a empirical observation such as that the law of large numbers holds for coin tosses, or is this somehow programmed ...