Questions tagged [randomness]

Randomness is the lack of pattern or predictability in events. Randomness is often modeled with probability distributions, but can also be generated by deterministic processes.

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

Deep learning model performance heavily dependent on initial model weights

While my deep learning model is quite robust to randomness coming from train/test data-splits, it is not robust to initial model weights. How should I go about this? Also, I'd greatly appreciate it if ...
0 votes
0 answers
16 views

Variance in Deep-Neural-Network Classifier Performance based on weight-initialization

Currently, I am applying K-fold Cross-validation to a Deep-Neural-Network Classifier with a fixed set of model-hyperparameters. To test the robustness of that fixed hyperparameter-set to randomness, I ...
1 vote
0 answers
10 views

Can we use p-values of NIST Test as a score for Randomness for a given binary sequence?

I am given two sequences and the p-values of NIST Tests for both sequences. Can I use these p-values as a measure of the quality of randomness?
  • 11
0 votes
0 answers
14 views

Quantifying patterns/non-randomness in 2D residual plots

I have many matrices (different but small dimensions) of independent residuals. Each cell should follow N(0,sd). This is one example: I would like to automate pattern detection, so that ideally I get ...
  • 175
1 vote
1 answer
50 views

Two not normal distributed samples & unequal sample size

I got two sample sets a training set around 32k rows and I have a test set of 16k rows. I want to test if the two sets are randomly split. I checked the normal distribution and it is not met for none ...
0 votes
0 answers
71 views

Metric to quantify randomness

I have an $L \times L$ matrix representing a $2D$ region. Each entry of this matrix is a real number lying in the closed interval [0,1]. I want to quantify how different is this from a similar $L \...
  • 1
4 votes
2 answers
105 views

Stochastic modelling, distribution and ergodicity of a particular time series with a given finite history

Let $\Omega$ be a sample space. A stochastic process $\{Y_t\}$ is a function of both time $t \in \{1, 2, 3, \ldots\}$ and outcome $\omega \in \Omega$. For any time $t$, $Y_t$ is a random variable (i....
0 votes
2 answers
210 views

Is using a fixed random seed in production okay?

I have a dataset and am trying to use GMM to cluster it. The algorithm works well but when I run it multiple times I get different results. While the clusters produced in each run are valid my users ...
  • 11
1 vote
1 answer
29 views

How to determine the randomness of coin tosses by its final position?

Suppose a circle is drawn on the floor, and a coin is tossed over it many times. If it jumped out of the circle, then we do not take it into account. How to determine from the statistics of coin end ...
0 votes
1 answer
317 views

Chi Square Test?

Would I use a chi square test to solve the following: Use the attached data set for our company to address the following. ...
  • 1
1 vote
2 answers
63 views

Is there a term for this concept (Randomly Random)?

Let's say I have what appears to be a fair dice that over 10K fair throws it produces very close to 1/6 occurrences of each face value. But then one day it starts a string of throws where the 3 face ...
9 votes
4 answers
983 views

Is anything inherently random?

Is anything inherently random? Or is all randomness observed in data either "errors in measurement" or "lack of understanding"? Assume we could measure everything with infinite ...
1 vote
0 answers
35 views

Can we ever quantify the level of randomness within a (data collecting) experiment?

Can we ever quantify the level of randomness within a (data collecting) experiment? We all know that "randomness" is both directly and indirectly required on different levels for various ...
  • 6,048
3 votes
1 answer
200 views

Are randomness and probability really logically dependent notions?

Background Some have lamented that it is difficult to distinguish randomness from probability, but I seem to be having a contrary difficulty: are they even logically dependent notions? Probability ...
  • 4,650
1 vote
3 answers
79 views

Statistical Test for Randomness

just wanted to ask if it's possible to check whether a given sample of data is drawn from a random distribution? If so, can I use some statistical test like the Anderson-Darling test or Chi-squared ...
1 vote
1 answer
53 views

measure if some binary events happen in random or are correlated

I have thousands of binary processes of uneven length like those below: ...
  • 33
1 vote
0 answers
112 views

Best practices for testing a coin-flip algorithm

I am uncertain what the best approach is to analyze a game's coin flip algorithm. A game that I play uses a coin flip algorithm to determine who goes first (on the play) or second (on the draw), but ...
2 votes
2 answers
200 views

Why is randomness so useful?

Where by "why" I do not mean "list of use cases for randomness." If one has a quantitative question Q about topic X, it does not seem intuitive that values that by definition have ...
  • 2,047
2 votes
1 answer
57 views

Randomness generated by deterministic processes

The tag excerpt of randomness says: Randomness <...> can also be generated by deterministic processes. Is that not intrinsically contradictory? Or does it consider randomness epistemically ...
2 votes
1 answer
31 views

Definition of model in the tag excerpt: random relationship(s) between variables

The tag excerpt of model says: A formalization of relationships between stochastically (randomly) related variables in the form of mathematical equations. This seems questionable to me. I guess ...
0 votes
2 answers
190 views

Is random walk with drift is random?

I see everywhere in the web that lag-plot or acf are used to see if a time serie is random. If there is no structure in the lag plot then the data are random, and if autocorrelation = 0 then data is ...
  • 3
1 vote
2 answers
106 views

Why is a 100 heads run surprising? [closed]

Assume we have a fair coin. We flip it 100 times. The outcome is all heads. Why is it that all heads outcome is more surprising to us than a "more random looking" outcome with less ...
  • 111
3 votes
1 answer
266 views

Is there a way to determine if one picture is more random than another

I have two pictures. I only want to keep pictures that "display" randomness. Is there a way to distinguish more random from less? In these plots, time is the vertical axis. So, the second ...
  • 41
4 votes
1 answer
93 views

Why does k-NN algorithm produce different predictions with same model and data?

I have successfully created a model using Apple coremltools' KNearestNeighborsClassifierBuilder() and trained it. However, each time I run the algorithm, I get different predictions. Here are the ...
  • 141
0 votes
1 answer
46 views

How far from "even" is (or is not) significant?

If a six-sided die is rolled 600 times, an even distribution would be 100 rolls for each number. If I actually rolled 600 times and tallied the results... How far away from 100 would a count need to ...
2 votes
1 answer
78 views

Frequency of mutants in a very large number of observations

During cellular division (generation of two cells from one) bacteria experience mutations, where a newly produced cell harbors altered genetic code. To measure the rate of this happening Max Delbrück ...
  • 131
5 votes
2 answers
316 views

The interpretation of a random variable

A variable is modelled as a random variable, without reference to the question whether it is truly random in reality. For example, when the outcome of a flip coin is modelled as a random variable, no ...
  • 417
0 votes
1 answer
282 views

Fixing random seed on inference

I will be deploying models into production and I'm wondering whether it is appropriate to fix the random number generator seed at inference time. Here's what I think: setting the seed will give ...
  • 363
0 votes
1 answer
81 views

How do you show a sequence of numbers is not random?

Let's say I take the following sequence of numbers: Obviously it's not random - each value is close to either .5, 1 or 2. What test would you use to show that it's not random?
1 vote
0 answers
27 views

Test for random distribution of values in a sequence

I'm looking to measure the degree to which values in a column or list are ordered or pseudo-randomly aligned. In essence I'm trying to distinguish the following scenarios. Say we have a "sex" column ...
  • 111
5 votes
1 answer
245 views

Informational value of R squared and correlation? [closed]

Taleb has previously undermined the typical interpretation of correlation with regards to the informational value it carries, showing how the uncertainty is reduced in a non-linear fashion. With ...
  • 1,040
0 votes
0 answers
13 views

What are the necessary steps required for conducting a Wolfowitz runs test of randomness, given that I have all the necessary data?

I am writing a part in my thesis about the random-walk theory of price changes. I plan to set up a runs test of randomness to show whether the price changes do follow a particular random pattern. I ...
0 votes
0 answers
21 views

How to spot/measure artifacts produced by several rotations of 2-dimensional data?

I am trying to transform dataset to mulitiply its size but preserve its initial random nature. I have 5-dimensional numerical dataset which represent particles in plane, produced by Monte Carlo ...
  • 1
1 vote
0 answers
40 views

I am drawing balls of three colours from an urn until it is empty. Is the sequence of colours random?

I have the following problem. I have an urn with n balls of k different colours. I draw balls without replacement until the urn is empty. This leads to a sequence of colours. I now want to show that ...
  • 11
4 votes
1 answer
60 views

Typo in Deepleariningbook.org or am I misunderstanding Bayesian stats?

This is on page 133 of the book: https://www.deeplearningbook.org/contents/ml.html#pf10 In the above, it says that the data set is directly observed and so is not random If that data we observe ...
  • 2,543
1 vote
1 answer
199 views

Interpretation of Clark-Evans test and Hopkins-Skellam test (complete spatial randomness)

When I’am calculating a Clark Evans or Hopkins-Skellam test to test for complete spatial randomness a value with R/A < 1 signifies a clustered point pattern; R/A = 1 would be a random point pattern....
  • 11
3 votes
0 answers
49 views

Why is Knuth's definition of random sequence (R4) too weak?

Knuth's $R4$ definition of a random sequence is called "too weak" and he presents an example of why --- second paragraph after the definition on the previous link. Definition $R4$. A $[0..1)$ ...
0 votes
1 answer
35 views

My approach to deal with the effects of randomness in a ML model

I'm not a statistician, so please excuse a possibly wrong use of terminology here. My dataset has about 400 - 800 samples and about 800 features. The samples are ordered by time, although it is not a ...
1 vote
1 answer
100 views

Need to randomize rank order,

I have a ranking of items, $X = (x_1, \ldots, x_n)$. I want to obtain a random sequence $\hat{X}$, which is a permutation of $x_i$, such that the expected rank-correlation (say, Kendall) between $X$ ...
2 votes
0 answers
71 views

maximum randomness and non-randomness

Two questions about randomness of data-sequences: Using a variant of run test like Wald–Wolfowitz to check randomness hypothesis for a one-sample data sequence, which yields a P value denoting the ...
  • 21
4 votes
1 answer
100 views

Measure of randomness that maps well to "hard to guess"?

It was recently pointed out to me that entropy does not necessarily map well to "hard to guess". E.g.: consider a distribution where there's a very high (say, 90%) chance you select a single, ...
  • 525
0 votes
2 answers
61 views

Is there a way to check for sure that optimization was applied to the weights of NN?

Given the first or next steps of NN training, where weights were initially randomly sampled from some distribution, is there a reliable way to determine if some optimization was applied to the weights?...
  • 113
2 votes
1 answer
4k views

Classification accuracy heavily depends on random seed

I want to compare different classification methods and evaluate their prediction measures (such as accuracy etc). I first split the data into training and test set. With the training data I then ...
  • 277
1 vote
1 answer
1k views

LSTM - random and always-different time between data measurements?

I am working with a time series problem where the time between two data measurements is random, and I am trying, without luck, to find an LSTM architecture that can handle this. A very simplified ...
  • 31
0 votes
0 answers
34 views

Testing Randomness in day of year values

I have airline booking data (no. of bookings vs day of year). Day of year is a vector, taking values from 1 to 365, with stepwise increments of 1. We can have one data-set where booking exists for ...
  • 1
4 votes
1 answer
808 views

Stochastic vs Adversarial Multi-Armed Bandit Problems

I know that the multi-armed bandit can be formalised in multiple ways - two of them being the stochastic and adversarial ways. I am familiar with the fact that adversarial way is a game theoretic ...
1 vote
1 answer
94 views

The assumptions regarding the t-test

The t-test has an assumption that the sample provided has to be random in nature. Do we test our sample for randomness before carrying out the t-test ?
0 votes
0 answers
175 views

Explain the results of the 2-sample Kolmogorov-Smirnov test for two random Gaussian sets of numbers

Suppose I have two sets of random numbers drawn from the Gaussian distribution, $\mathbf{r_1}$ and $\mathbf{r_2}$. I then do a 2-sample Kolmogorov-Smirnov test (KS test) between the two to determine ...
  • 845
3 votes
3 answers
271 views

How do I prove one company is actually doing better than another?

Suppose I have the return information for two reverse logistic companies--See example data. For certain groups companies A and B each receive 50% of the work from a client, but for another group ...
1 vote
0 answers
25 views

Random sampling at two different frequencies

In a production line, random samples are drawn to be tested for properties A and B. A has to be tested at a higher cadence than B and the assumption is that A and B have zero correlation. Since the ...

1
2 3 4 5