Questions tagged [randomness]

Randomness is the lack of pattern or predictability in events. Randomness is often modeled with probability distributions, but can also be generated by deterministic processes.

61 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
7
votes
0answers
3k views

Precision and recall of a random classifier

My understanding of precision and recall tells me that there is a tradeoff between these two measures: you can improve one at the cost of the other. However, when I think of a random classifier (on a ...
4
votes
0answers
1k views

How to validate a random sample

I have been given two sets of data, one with 10,000 observations which is a subset of the other with 100,000 observations. It is claimed that the small dataset is a random sample of the large dataset. ...
4
votes
0answers
167 views

Sanity check to determine if 'random' data on disk drive may have been tampered with

I've got some code that does a Dept of Defense secure erase of disks, and would like to know about a good sanity check to run after the fact. Bottom line, the last write phase of a secure erase of a ...
3
votes
0answers
27 views

How best to form many short, independent random numbers from one long random number?

I have 20 digit, uniformly distributed random numbers, downloaded from random.org (because the numbers from that source are based on variations in atmospheric pressure rather than a mathematical ...
3
votes
0answers
64 views

A pseudorandom number generator with a seed greater than 80 bits

I need to generate a sequence of pseudorandom numbers with a period larger than $2^{80}$, using a seed larger than 80 bits, on a 32-bit embedded microprocessor. What are my possible options (in terms ...
3
votes
0answers
128 views

Relation between statistical randomness, uniform distribution and independence

In Monte Carlo simulation, we often consider how well a sequence of generated points are. If I am correct, one aspect is statistical randomness: A numeric sequence is said to be statistically ...
3
votes
0answers
833 views

interpreting chi square value for validating random numbers

I am trying to use the chi-square test for identifying random data. I have 256 categories, i.e. 255 degrees of freedom and count the occurrence of byte-values (0-255). As suggested by Knuth, the ...
3
votes
0answers
75 views

Best way to find non-randomness regions in these or similar count data?

Let say I have data in a shape: [0,0,0,0,1,0,0,0,1,1,0,0,0,0,0,0,0,0,1,0,0,2,0,0,0,....] - so mainly zeros.... However I know how long is my 'signal' and how many counts are they. Is it possible ...
2
votes
0answers
42 views

maximum randomness and non-randomness

Two questions about randomness of data-sequences: Using a variant of run test like Wald–Wolfowitz to check randomness hypothesis for a one-sample data sequence, which yields a P value denoting the ...
2
votes
1answer
75 views

Randomized Block Design

It is desired to use a randomized block design with $4$ blocks of size $6$ each for testing the effects of $5$ treatments A,B,C,D and E. In each block, treatments B,C,D and E are replicated once each, ...
2
votes
0answers
89 views

Clarification on the “tickets in a box interpretation” for random variables

I have a question on the definition of a random variable. I have read the tickets in a box interpretation here and here, which helped me immensely, but I have some doubts for the case in which the ...
2
votes
0answers
67 views

When random output is a local maximum, how can a neural network find the global (or a better local) maximum?

I'm trying to train a neural network to play Rock-Paper-Scissors. A random player is a local maximum for the game, as it wins 50% of the time. In a tournament setting, it comes out consistently ...
2
votes
0answers
40 views

Testing for randomness of room assignments

I have data on room assignments for students at a university, as well as certain characteristics on the students in the rooms, e.g. test scores, gpa, nationality, gender. I am told room assignments ...
2
votes
0answers
169 views

Suggested algorithms for time series analysis of sporadic events

I have a series of sporadic, discrete time series events that appear to occur in shorter interval clusters and I am looking for a way to determine the frequencies at which the events occur. None of ...
2
votes
0answers
65 views

Auto-Correlation

I had thought that if I call acf() on a vector of integers, and examine the value correlation coefficient at K (lag) = 1, I would expect: A value close to +/-1 would show that the integers are not ...
2
votes
0answers
296 views

What is the distribution of an outer product of random values mod P?

I haven't been able to answer this daunting question for days now and I am not able to find the right resource for it, so maybe someone here might help me out. Consider an outer product of two ...
2
votes
0answers
87 views

How to select model when different models are preferred with different seeds?

I am trying to apply the lasso or ridge regression to my data set for the feature selection, but different random seeds produce different models. What is a good or universal way to obtain the final ...
1
vote
0answers
23 views

Test for random distribution of values in a sequence

I'm looking to measure the degree to which values in a column or list are ordered or pseudo-randomly aligned. In essence I'm trying to distinguish the following scenarios. Say we have a "sex" column ...
1
vote
0answers
37 views

I am drawing balls of three colours from an urn until it is empty. Is the sequence of colours random?

I have the following problem. I have an urn with n balls of k different colours. I draw balls without replacement until the urn is empty. This leads to a sequence of colours. I now want to show that ...
1
vote
0answers
23 views

Random sampling at two different frequencies

In a production line, random samples are drawn to be tested for properties A and B. A has to be tested at a higher cadence than B and the assumption is that A and B have zero correlation. Since the ...
1
vote
0answers
40 views

Is there a known PRNG that passes TestU01's BigCrunch but fails any test in NIST 800-22?

In other words, does TestU01 incorporate all statistical tests by NIST 800-22?
1
vote
0answers
55 views

Goodness of fit test for observed samples of binary strings

Consider a process that produces binary strings of varying length $n$. A typical sample would include $n\ $ I Number of strings $1\ $ I $\ 3,000,000$ $2\ $ I $\ 800,000$ $3\ $ I $\ 350,...
1
vote
0answers
79 views

why LSTM word based model works better with random input seed than fixed input seed?

I have implemented an LSTM model that have 2 LSTM layers, a dropout layer and a dense layer for predictions. I trained my LSTM model on 1000 XML files. Each file has 4 main markups with very simple ...
1
vote
0answers
68 views

Randomly split batches for time series CV

In certain code snippets for time series prediction, I find sometimes that authors split their data to train and test sets (nothing unusual for the moment). But then the train data is split into ...
1
vote
0answers
378 views

Generate sequence of random autocorrelated numbers

Back in the university I was working on a program that generates a sequence of random autocorrelated numbers. Autocorrelated meaning that each next value is close to the previous one and depends on it ...
1
vote
0answers
92 views

Random sequence and uniform distribution? Is this always the case?

(Short version) Is this always the case that a truly generated random sequence (of whole numbers) conforms to (or at least expected to conform to) a uniform distribution? (Long version) The most ...
1
vote
0answers
65 views

Simulating from the Gaussian Distribution

I attempted to sample random variables from the Gaussian distribution in C using the following simulation method: $$X_n = \sqrt{(-2\log(U_1)}\cos{(2\pi U_2)}$$ Where $U_1, U_2 \sim \mathcal{U}(0,\,...
1
vote
0answers
87 views

purpose of random experiments in Jaynes' Baysian probability theory?

NOTE: I understand the role of random experiments in orthodox probability theory, and its role in providing unbiased estimators and hypothesis tests. Please don't answer this question with basic ...
1
vote
0answers
66 views

the length of a random walk in two dimensions

This is a matlab code that could generate a random walk in two dimensions of length 1000: xy = cumsum (-1+2*rand(1000,2),1). The 'stepsize' is between $[-1,1]$. In this case, we are talking about ...
1
vote
0answers
64 views

Comparing the performance of random and weighted-random classifiers

I have a set of 300 observations of known class and there are 20 classes in total. I want to compare the performance of a random classifier with a weighted-random classifier at classifying the ...
1
vote
0answers
324 views

Uniformity of true random numbers

Are numbers generated by True (hardware) Random Number Generators (TRNGs) more uniform than numbers from a Pseudo Random Number Generator (PRNG)? I understand that there are not much (analytical/...
1
vote
0answers
603 views

How to do cross validation in Random Forest if the objective is to measure variable importance?

As you know, the random forest algorithm and k-fold cross validation (except the case of Leave One Out) involve randomness. Thus a typical 10-fold CV in one run results in a certain variable ...
1
vote
0answers
23 views

What's the best method for figuring out whether clusters occurring in data are random or are indicative of an underlying non-random behavior?

I have a dataset of the times that events happen. I noticed some clustering of events but I know that even assuming a random uniform distribution some amount clustering will inevitably occur. How do I ...
1
vote
0answers
48 views

Detecting extent of randomness of a binary time series

I have several time series that take a value of 1 if an event occurs and 0 otherwise. I would like to measure randomness of each time series is. What would be a good measure for this? The reason I am ...
1
vote
0answers
72 views

Variation of Wald–Wolfowitz detecting seasonal patterns?

I am considering the standard test for randomness based on runs(Wald–Wolfowitz runs test). In it, you basically take a sequence of 1's and 2's, having length $n$, $n_1$ 1's, $n_2$ 2's and you want to ...
1
vote
0answers
66 views

How can I test if a string on distinct numbers/characters is random?

Let's say I have a string of 26 letters. I want to permute them and then test if my permutation is random. How do I do this?
1
vote
2answers
280 views

Calculate probability of random numbers adding up to or being greater than another number

What's a simple formula I can use for calculating the probability of a set of random numbers adding up to or being greater than another number? Where $W$ is the amount of random numbers picked, $X$ is ...
1
vote
0answers
35 views

Question about word/discourse similarity analysis

Question about a investigative journalistic proof involving LDA and prior along with general probability distribution model for an unusual type of problem. Basically I have an email I wrote to a ...
1
vote
0answers
48 views

testing randomness in a grid

I need help with figuring out what kind of statistics I need to run to test for randomness in a grid for small mammal trapping. The grid consists on 64 traps, arranged in 8 rows of 8. The image ...
1
vote
0answers
30 views

A question about random permutations and the probability of their fixed points

I have started off by listing all the random permutations as (1 2 3) (1 3 2) (2 3 1) (2 1 3) (3 2 1) (3 1 2) I have shown that the probability a random permutation has at least one fixed ...
1
vote
0answers
241 views

Accounting for measurement bias using a histogram or violin plot and numerical data in R

The Problem I have a gardener whose job it is to measure trees in meters using a measuring stick. She measures the heights of 1,000 trees. I plot these measurements on a violin plot in R. I am ...
1
vote
0answers
57 views

How can I model this problem?

Genetic algorithms are a kind of evolutive approach to problem solving where solutions are randomly generated and crossed with each other as to produce other solutions. With each generation or ...
1
vote
0answers
44 views

Modeling a list with a tunable degree of disorder/shuffling

Imagine we have a list of ordered numbers $L = (1, 2,\dots, N)$. I want to add an arbitrary amount of "disorder" to that list. For instance: Adding a little bit of disorder would permute a few ...
1
vote
0answers
114 views

How to test the randomness of a sequence of real m-tuples?

1) Let $\{a_i\}_{i = 1, ...,n}$ be a real sequence generated by a pseudorandom number generator. How can I check the randomness of such real numbers using, for instance, Runs test? 2) Assume three ...
1
vote
0answers
66 views

Finding a sample to represent a known population

Entire Population: 30,000 Known properties of population: Age, gender, height, weight, Ideal representative sample size: ??? This is for a mailing list test group selection. Problem constraints: ...
0
votes
0answers
7 views

Ljung-box test on weekly percentage of total quarter bookings

I have a data on the weekly percentage of the total quarter bookings. The data looks as follows (note: weekly percentages add up to 100 for each quarter) : (not real data) I used the Ljung-box test ...
0
votes
0answers
10 views

What are the necessary steps required for conducting a Wolfowitz runs test of randomness, given that I have all the necessary data?

I am writing a part in my thesis about the random-walk theory of price changes. I plan to set up a runs test of randomness to show whether the price changes do follow a particular random pattern. I ...
0
votes
0answers
15 views

How to spot/measure artifacts produced by several rotations of 2-dimensional data?

I am trying to transform dataset to mulitiply its size but preserve its initial random nature. I have 5-dimensional numerical dataset which represent particles in plane, produced by Monte Carlo ...
0
votes
1answer
28 views

My approach to deal with the effects of randomness in a ML model

I'm not a statistician, so please excuse a possibly wrong use of terminology here. My dataset has about 400 - 800 samples and about 800 features. The samples are ordered by time, although it is not a ...
0
votes
0answers
34 views

Testing Randomness in day of year values

I have airline booking data (no. of bookings vs day of year). Day of year is a vector, taking values from 1 to 365, with stepwise increments of 1. We can have one data-set where booking exists for ...