Questions tagged [randomness]

Randomness is the lack of pattern or predictability in events. Randomness is often modeled with probability distributions, but can also be generated by deterministic processes.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Ljung-box test on weekly percentage of total quarter bookings

I have a data on the weekly percentage of the total quarter bookings. The data looks as follows (note: weekly percentages add up to 100 for each quarter) : (not real data) I used the Ljung-box test ...
1
vote
0answers
23 views

Test for random distribution of values in a sequence

I'm looking to measure the degree to which values in a column or list are ordered or pseudo-randomly aligned. In essence I'm trying to distinguish the following scenarios. Say we have a "sex" column ...
5
votes
1answer
115 views

Informational value of R squared and correlation? [closed]

Taleb has previously undermined the typical interpretation of correlation with regards to the informational value it carries, showing how the uncertainty is reduced in a non-linear fashion. With ...
0
votes
0answers
10 views

What are the necessary steps required for conducting a Wolfowitz runs test of randomness, given that I have all the necessary data?

I am writing a part in my thesis about the random-walk theory of price changes. I plan to set up a runs test of randomness to show whether the price changes do follow a particular random pattern. I ...
0
votes
0answers
15 views

How to spot/measure artifacts produced by several rotations of 2-dimensional data?

I am trying to transform dataset to mulitiply its size but preserve its initial random nature. I have 5-dimensional numerical dataset which represent particles in plane, produced by Monte Carlo ...
1
vote
0answers
37 views

I am drawing balls of three colours from an urn until it is empty. Is the sequence of colours random?

I have the following problem. I have an urn with n balls of k different colours. I draw balls without replacement until the urn is empty. This leads to a sequence of colours. I now want to show that ...
4
votes
1answer
53 views

Typo in Deepleariningbook.org or am I misunderstanding Bayesian stats?

This is on page 133 of the book: https://www.deeplearningbook.org/contents/ml.html#pf10 In the above, it says that the data set is directly observed and so is not random If that data we observe ...
0
votes
0answers
19 views

how to improve lasso regression model? [duplicate]

Im trying to build a linear model and accuracy in the model prediction is same for random set of features and a set of features - I believe should be associated with the variables. Is there anything ...
0
votes
1answer
28 views

My approach to deal with the effects of randomness in a ML model

I'm not a statistician, so please excuse a possibly wrong use of terminology here. My dataset has about 400 - 800 samples and about 800 features. The samples are ordered by time, although it is not a ...
1
vote
1answer
36 views

Need to randomize rank order,

I have a ranking of items, $X = (x_1, \ldots, x_n)$. I want to obtain a random sequence $\hat{X}$, which is a permutation of $x_i$, such that the expected rank-correlation (say, Kendall) between $X$ ...
2
votes
0answers
42 views

maximum randomness and non-randomness

Two questions about randomness of data-sequences: Using a variant of run test like Wald–Wolfowitz to check randomness hypothesis for a one-sample data sequence, which yields a P value denoting the ...
4
votes
1answer
57 views

Measure of randomness that maps well to “hard to guess”?

It was recently pointed out to me that entropy does not necessarily map well to "hard to guess". E.g.: consider a distribution where there's a very high (say, 90%) chance you select a single, ...
0
votes
2answers
56 views

Is there a way to check for sure that optimization was applied to the weights of NN?

Given the first or next steps of NN training, where weights were initially randomly sampled from some distribution, is there a reliable way to determine if some optimization was applied to the weights?...
2
votes
1answer
986 views

Classification accuracy heavily depends on random seed

I want to compare different classification methods and evaluate their prediction measures (such as accuracy etc). I first split the data into training and test set. With the training data I then ...
0
votes
1answer
804 views

LSTM - random and always-different time between data measurements?

I am working with a time series problem where the time between two data measurements is random, and I am trying, without luck, to find an LSTM architecture that can handle this. A very simplified ...
0
votes
0answers
34 views

Testing Randomness in day of year values

I have airline booking data (no. of bookings vs day of year). Day of year is a vector, taking values from 1 to 365, with stepwise increments of 1. We can have one data-set where booking exists for ...
0
votes
1answer
239 views

Stochastic vs Adversarial Multi-Armed Bandit Problems

I know that the multi-armed bandit can be formalised in multiple ways - two of them being the stochastic and adversarial ways. I am familiar with the fact that adversarial way is a game theoretic ...
0
votes
1answer
70 views

The assumptions regarding the t-test

The t-test has an assumption that the sample provided has to be random in nature. Do we test our sample for randomness before carrying out the t-test ?
0
votes
0answers
70 views

Explain the results of the 2-sample Kolmogorov-Smirnov test for two random Gaussian sets of numbers

Suppose I have two sets of random numbers drawn from the Gaussian distribution, $\mathbf{r_1}$ and $\mathbf{r_2}$. I then do a 2-sample Kolmogorov-Smirnov test (KS test) between the two to determine ...
3
votes
3answers
228 views

How do I prove one company is actually doing better than another?

Suppose I have the return information for two reverse logistic companies--See example data. For certain groups companies A and B each receive 50% of the work from a client, but for another group ...
1
vote
0answers
23 views

Random sampling at two different frequencies

In a production line, random samples are drawn to be tested for properties A and B. A has to be tested at a higher cadence than B and the assumption is that A and B have zero correlation. Since the ...
0
votes
1answer
31 views

Cross checking AI Selections vs Random Chance

Fifty Three Images AI Selects 20 images based on various criteria. I Select 20 images based on personal criteria. In a two group selection of all 53 items what is the likely overlap due to random ...
0
votes
0answers
41 views

How to do Monte Carlo with conditional input values?

Please apologise my likely ignorance of the correct terminology and notation. Any edits and suggestions to improve the question are very much appreciated. I want to perform a Monte Carlo simulation ...
8
votes
4answers
4k views

Check if a character string is not random

Background Let's say we have an alphabet of A,B, C, D, then we look through some data and find a "word" which is DDDDDDDDCDDDDDD ...
1
vote
0answers
40 views

Is there a known PRNG that passes TestU01's BigCrunch but fails any test in NIST 800-22?

In other words, does TestU01 incorporate all statistical tests by NIST 800-22?
6
votes
2answers
1k views

What is the state of the art in statistics tests for distinguishing good from bad random number generators?

There are many packages out there. In particular, PractRand gives out an opinion on a number of them, but it's only an opinion. Is there conventional wisdom about which set of set of statistics ...
2
votes
1answer
491 views

Why are there different statistics tests in NIST SP 800-90B and NIST SP 800-22? [closed]

NIST SP 800-22 (published in 2010) lists out some statistical tests for telling good random bit generators from bad ones. NIST SP 800-90B (published in 2018) has apparently the same purpose but lists ...
-1
votes
2answers
1k views

Why atmospheric noise is considered the only reliable source for generating “pure random numbers”?

When we say that something is random, it simply implies that the occurrence under observation does not have any cause or at least that cause cannot be determined beforehand in layman's language. For a ...
1
vote
0answers
55 views

Goodness of fit test for observed samples of binary strings

Consider a process that produces binary strings of varying length $n$. A typical sample would include $n\ $ I Number of strings $1\ $ I $\ 3,000,000$ $2\ $ I $\ 800,000$ $3\ $ I $\ 350,...
1
vote
0answers
79 views

why LSTM word based model works better with random input seed than fixed input seed?

I have implemented an LSTM model that have 2 LSTM layers, a dropout layer and a dense layer for predictions. I trained my LSTM model on 1000 XML files. Each file has 4 main markups with very simple ...
2
votes
1answer
528 views

Find if a coin toss sequence is random

Suppose we are given a sequence of heads and tails for 1000 coin tosses. how do we check if it's actually random? I believe I need to do a hypothesis test, I tried to write some examples to get an ...
3
votes
3answers
123 views

Is it possible to calculate the probability the sum of one set of random variables equals the sum of a different set of random variables?

I'm doing data analysis at the business I work for. 23 people were asked for input for two separate lists. The first list the answers could be any integer 6 to 10. The second list the answers could be ...
1
vote
2answers
962 views

NIST Randomness Test, Testu01, Dieharder

the NIST Randomness Test, Testu01, Dieharder all seem to offer a suite of tests to check psuedo random number generators(PRNG) for randomness. I was hoping someone with experience of those tools could ...
-1
votes
2answers
100 views

Dataset partitioning issue of cross-validation

Every time the cross-validation is run, the dataset is partitioned into k even groups randomly. That means every time the result of cross-validation can be different, so which one should we take to ...
2
votes
1answer
499 views

Confused about how to interpret Chi square results

For my local charity I have to organize random draws, each week to give a prize. I just bought a random machine generator, which randomly spits one ball out of 10. Before using the machine in public ...
2
votes
1answer
75 views

Randomized Block Design

It is desired to use a randomized block design with $4$ blocks of size $6$ each for testing the effects of $5$ treatments A,B,C,D and E. In each block, treatments B,C,D and E are replicated once each, ...
0
votes
1answer
26 views

Within individual independence testing

I have a group of individuals that is independent and identical distributed. Within each individual $i$, one may observe $N_i$ observations. The task is to test whether the within-individual ...
12
votes
3answers
868 views

Why do several (if not all) parametric hypothesis tests assume random sampling?

Tests like Z, t, and several others assume that the data is based on a random sampling. Why? Suppose that I'm doing experimental research, where I care much more for the internal validity than the ...
5
votes
2answers
2k views

Is PCG random number generator as good as claimed?

I have a demand for extremely fast acceptable-quality random number generator (RNG). This means rand()%2 should not alternate between 0 and 1 (that totally kills the simulation, as in one problem I ...
1
vote
1answer
537 views

How to generate randomness using a biased coin? [closed]

Imagine we have three people and we want to select one of them at random. If we had a fair coin we could have tossed the coin twice to get one of the 4 outcomes and could have related the occurrence ...
1
vote
0answers
68 views

Randomly split batches for time series CV

In certain code snippets for time series prediction, I find sometimes that authors split their data to train and test sets (nothing unusual for the moment). But then the train data is split into ...
1
vote
0answers
377 views

Generate sequence of random autocorrelated numbers

Back in the university I was working on a program that generates a sequence of random autocorrelated numbers. Autocorrelated meaning that each next value is close to the previous one and depends on it ...
-1
votes
1answer
47 views

Random sampling a matrix to reflect a real world case

I'm trying to create two matrices one that can be filled with 0's and 0.25's and another with 0's and 1's. ...
2
votes
0answers
89 views

Clarification on the “tickets in a box interpretation” for random variables

I have a question on the definition of a random variable. I have read the tickets in a box interpretation here and here, which helped me immensely, but I have some doubts for the case in which the ...
0
votes
2answers
232 views

But what is a random experiment

I am currently thinking a lot about the definition of probability and one thing that I am really not content with at all is the frequentist definition of probability as long run relative frequencies ...
1
vote
0answers
92 views

Random sequence and uniform distribution? Is this always the case?

(Short version) Is this always the case that a truly generated random sequence (of whole numbers) conforms to (or at least expected to conform to) a uniform distribution? (Long version) The most ...
3
votes
0answers
27 views

How best to form many short, independent random numbers from one long random number?

I have 20 digit, uniformly distributed random numbers, downloaded from random.org (because the numbers from that source are based on variations in atmospheric pressure rather than a mathematical ...
0
votes
0answers
39 views

Statistical tests for PRNG that generates a sequence which is not binary

I have already asked this question on crypto.stackexchange but I think here is the appropraite site for this. Is there a pratical application to PRNG that generates a sequence which is not a binary ...
3
votes
1answer
170 views

Establishing the Probability Distribution that Governs a Random Process

I have a experiment in which a random event that can occur $N$ times in some set period of time $T$ (where $N \in$ $\mathbb{N_0}$ but is typically less that 10 and never exceeds 100). Here I will ...
0
votes
4answers
80 views

Probability and randomness

Assume that we toss a coin n times and the result looks like this HTHTHTHTHTHTHTHT... . Would a (Probability-) Frequentist conclude that that the probability for the coin to land heads is 0.5? It ...