With $N = 365$ and $x = 23,$ randomly generated numbers, your vetting procedure is similar to the famous birthday problem, in which one would expect matching numbers among the $x$ a little more than half the time. However, the likelihood of matching birthdays among $23$ is reasonably robust to real-life situations in which some months are more likely to generate actual human birthdays than others. 

Thus a failure to get one or more matches about half the time would cast doubt on
the randomness of the 'generator', but getting matches nearly half the time time would not be strong evidence that the numbers are generated truly at random.

Classic birthday problem _with equally likely 365 equally likely birthdays._ By simulation in R,
$P(Y = 0) = 0.494 \pm 0.003$ [the _exact_ probability of $0$ matches is $0.4927$ to four places] and $E(Y) = 0.678\pm 0.005.$ 

    set.seed(1234)
    x = 23;  N = 365
    y = replicate(10^5, x-length(unique(sample(1:N,x,rep=T))))
    mean(y==0);  mean(y)
    [1] 0.49395   # aprx P(No Match)
    [1] 0.67842   # aprx E(Nr Matches)
    2*sd(y==0)/sqrt(10^5)
    [1] 0.003162062
    2*sd(y)/sqrt(10^5)
    [1] 0.005012195


_With days not equally likely_ (roughly 95% and 110% as likely in two halves of the year): $P(Y=0) = 0.491\pm 0.002, E(Y)=0.683\pm 0.003.$

Within the margin of simulation error, results are not significantly different from those for equally likely days.

    set.seed(1235)
    x = 23;  N = 365;  pr = c(rep(95, 180), rep(105, 185))
    y = replicate(10^5, x-length(unique(sample(1:N,x,rep=T,p=pr))))
    mean(y==0);  mean(y)
    [1] 0.49102
    [1] 0.68265
    sd(y==0)/sqrt(10^5)
    [1] 0.001580892
    sd(y)/sqrt(10^5)
    [1] 0.002509512

The birthday problem has been shown with more extensive simulations not to
be especially finicky in case birthdays are not exactly equally likely.

There are lists of problems that are notoriously sensitive to imperfections in random number generators. You can google the 'Die Hard Battery' of
especially finicky simulation problems that have been used to vet pseudorandom number generators.