What exactly is a seed in a random number generator? I tried some usual google search etc. but most of the answers I find are either somewhat ambiguous or language/library specific such as Python or C++ stdlib.h etc. I am looking for a language agnostic, mathematical answer, not the specifics of a library. 
As an example, many say that the seed is a starting point of random number generator and the same seed always produces the same random number. What does it mean? Does it mean the output number is a deterministic function of a specific seed, and the randomness comes from the value of the seed? But if that is the case, then by supplying the seed, are not we, the programmers, creating the randomness instead of letting the machine do it?
Also, what does a starting point mean in this context? Is this a non-rigorous way of saying an element $x\in\mathfrak{X}$ of the domain of a map $f:\mathfrak{X}\rightarrow\mathfrak{Y}$? Or am I getting something wrong?
 A: Most pseudo-random number generators (PRNGs) are build on algorithms involving some kind of recursive method starting from a base value that is determined by an input called the "seed".  The default PRNG in most statistical software (R, Python, Stata, etc.) is the Mersenne Twister algorithm MT19937, which is set out in Matsumoto and Nishimura (1998).  This is a complicated algorithm, so it would be best to read the paper on it if you want to know how it works in detail.  In this particular algorithm, there is a recurrence relation of degree $n$, and your input seed is an initial set of vectors $\mathbf{x}_0, \mathbf{x}_1, ..., \mathbf{x}_{n-1}$.  The algorithm uses a linear recurrence relation that generates:
$$\mathbf{x}_{n+k} = f(\mathbf{x}_k, \mathbf{x}_{k+1}, \mathbf{x}_{k+m}, r, \mathbf{A}),$$
where $1 \leqslant m \leqslant n$ and $r$ and $\mathbf{A}$ are objects that can be specified as parameters in the algorithm.  Since the seed gives the initial set of vectors (and given other fixed parameters for the algorithm), the series of pseudo-random numbers generated by the algorithm is fixed.  If you change the seed then you change the initial vectors, which changes the pseudo-random numbers generated by the algorithm.  This is, of course, the function of the seed.
Now, it is important to note that this is just one example, using the MT19937 algorithm.  There are many PRNGs that can be used in statistical software, and they each involve different recursive methods, and so the seed means a different thing (in technical terms) in each of them.  You can find a library of PRNGs for R in this documentation, which lists the available algorithms and the papers that describe these algorithms.
The purpose of the seed is to allow the user to "lock" the pseudo-random number generator, to allow replicable analysis.  Some analysts like to set the seed using a true random-number generator (TRNG) which uses hardware inputs to generate an initial seed number, and then report this as a locked number.  If the seed is set and reported by the original user then an auditor can repeat the analysis and obtain the same sequence of pseudo-random numbers as the original user.  If the seed is not set then the algorithm will usually use some kind of default seed (e.g., from the system clock), and it will generally not be possible to replicate the randomisation.
A: First, there is no true randomness in today's computer-generated "random numbers." All pseudorandom generators use
deterministic methods. (Possibly, quantum computers will change that.)
The difficult task is to contrive algorithms
that produce output that cannot meaningfully be distinguished from
data coming from a truly random source.
You are right that setting a seed starts you at a particular known
starting point in a long list of pseudorandom numbers. For the generators implemented
in R, Python, and so on, the list is hugely long. Long enough that
not even the largest feasible simulation project will exceed the 'period'
of the generator so that values begin to re-cycle.
In many ordinary applications, people do not set a seed. Then an unpredictable
seed is picked automatically (for example, from the microseconds on the
operating system clock). Pseudorandom generators in general use have been
subjected to batteries of tests, largely consisting of problems that have
proved to be difficult to simulate with earlier unsatisfactory generators.
Usually, the output of a generator consists of values that are not, for
practical purposes, distinguishable from numbers chosen truly at random
form the uniform distribution on $(0,1).$ Then those pseudorandom numbers
are manipulated to match what one would get sampling at random from other
distributions such as binomial, Poisson, normal, exponential, etc.
One test of a generator is to see if its successive pairs in 'observations' simulated as
$\mathsf{Unif}(0,1)$ actually look like they are filling the unit square
at random. (Done twice below.) The slightly marbled look is a result of inherent variability.
It would be very suspicious to get a plot that looked perfectly uniformly
grey. [At some resolutions, there may be a regular moire pattern; please
change the magnification up or down to get rid of that bogus effect if it occurs.]
set.seed(1776);  m = 50000
par(mfrow=c(1,2))
  u = runif(m);  plot(u[1:(m-1)], u[2:m], pch=".")
  u = runif(m);  plot(u[1:(m-1)], u[2:m], pch=".")
par(mfrow=c(1,1))


It is sometimes useful to set a seed. Some such uses are as
follows:

*

*When programming and debugging it is convenient to have predictable output.
So many programmers put a set.seed statement at the start of a program
until writing and debugging are done.


*When teaching about simulation. If I want to show students that
I can simulate rolls of a fair die using the sample function in R, I could
cheat, running many simulations, and picking the one that comes closest to
a target theoretical value. But that would give an unrealistic impression of how simulation really works.
If I set a seed at the start, the simulation
will get the same result every time. Students can proofread their copy of
my program to make sure it gives the intended results. Then they can run
their own simulations, either with their own seeds or by letting the program
pick its own starting place.
For example, the probability of getting the total 10 when rolling two fair dice is $$3/36 = 1/12 =  0.08333333.$$ With a million 2-dice experiments I should get about two or three-place accuracy. The 95% margin of simulation error is about $$2\sqrt{(1/12)(11/12)/10^6} =  0.00055.$$
    set.seed(703);  m = 10^6
    s = replicate( m, sum(sample(1:6, 2, rep=T)) )
    mean(s == 10)
    [1] 0.083456         # aprx 1/12 = 0.0833
    2*sd(s == 10)/sqrt(m)
    [1] 0.0005531408     # aprx 95% marg of sim err.



*When sharing statistical analyses that involve simulation.
Nowadays many statistical analyses involve some simulation, for example a permutation test or a Gibbs sampler. By showing the seed, you enable
people who read the analysis to replicate the results exactly, if they wish.


*When writing academic articles involving randomization. Academic articles usually go through multiple rounds of peer review. A plot may use, e.g., randomly jittered points to reduce overplotting. If the analyses need to be slightly changed in response to reviewer comments, it is good if a particular unrelated jittering does not change between review rounds, which might be disconcerting to particularly nitpicky reviewers, so you set a seed before jittering.
A: TL;DR;
A seed usually enables you to reproduce the sequence of random numbers. In that sense they are not true random numbers but "pseudo random numbers", hence a PNR Generator (PNRG). These are a real help in real life!
A bit more detail:
Virtually all "random" number generators implemented in computer languages are pseudo random number generators. This is because given a starting value (===> the seed) they will always provide the same sequence of pseudo random results. A good generator will produce a sequence that can not be distinguished - in statistical terms - from a true random sequence (throw a true die, true coin, etc).
In many simulation cases you want to have a true "random" experience. However, you also want to be able to reproduce your results. Why? Well, at least regulators are interested in that peculiar thing.
There's a lot to dive in to. People even do analysis into the "best" random seed. In my opinion this invalidates their model as they can't handle "true" random behavior - or their PRNG is not fit for their implementation. Most of the time they just don't do enough simulations - but they take time.
Now imagine a "true" RNG. One could implement this based on a kind of randomness in the machine. If you only take a random seed (e.g. time now) you create kind of a random starting point but the randomness of the sequence still depends on the algorithm to determine the next numbers. This is more important than the starting point in most cases as the distribution of results determines the actual "result". If your sequence should be truly random, how would you implement this? Clock ticks of a computer can be said to be deterministic and otherwise probably will show a lot of auto-correlation. So what can you do?
 The best bet so-far is to implement a solid PNRG.
Quantum Computing? I'm not sure that will fix it.
