0
$\begingroup$

Given the first or next steps of NN training, where weights were initially randomly sampled from some distribution, is there a reliable way to determine if some optimization was applied to the weights?

Or in other words, I want to have a way to check if some training has been done, and weighs are not randomly generated.

My thoughts were to check if the distribution of the weights will change in time, but the information I found so far indicates that if weights were, for example, sampled from the normal distribution, the distribution of weights may keep normal even after a bunch of training epochs.

I've thought about tracking the improvements of predictions in time, but to my understanding, there can be theoretical situations when optimization will not give a quick improvement of the accuracy, so in that case, there will be some performed training, and just checking the accuracy will be not enough.

$\endgroup$
  • 1
    $\begingroup$ Is there any reason you're not just looking at the difference between the initial weights W0 and the weights you want to check, W1? I expect the l2 norm of the difference would be close to zero only when no optimization was done (which I guess is what you're trying to debug). $\endgroup$ – galoosh33 Jan 27 '19 at 15:53
  • $\begingroup$ @galoosh33 what I want to achieve is somewhat specific - I want to check the 3rd-party's computations, and there is no expected trust for that 3rd-party, so I want to find a way to check that some meaningful computations have been done, and the result weights are not completely random. $\endgroup$ – coldmind Jan 27 '19 at 16:01
  • 1
    $\begingroup$ If you make the strong assumptions that (1) the initial weights come from a known distribution and (2) the optimized weights do not come from that distribution then this reduces to a statistical test $\endgroup$ – Sycorax Jan 27 '19 at 16:44
1
$\begingroup$

Yes, please see "verifiable computing from homeomorphic encryption" which allows for a client to send job to a worker, the worker to complete the job (== apply gradient descent) and send the results back, and for the client to verify that the computation was done correctly. As an additional side-benefit, the worker never has access to the weights or data used during training. Unfortunately I don't understand this field well enough to explain how exactly it works.

Of course this comes with the caveat that I don't believe there is any computationally feasible implementation of homeomorphic encryption so far, so it is only possible in theory.


Alternatively, consider requesting the worker to send back weights from checkpoint iterations $0$, $k$, $2k$, $\ldots$ then, when training is complete at iteration $T$, select $m$ unique random checkpoints and train to the next checkpoint, verifying that starting from the weights from iteration $tk$ you really will end up with the weights from iteration $(t+1)k$ after $k$ iterations. To do this the worker probably also needs to send you the random seed used for constructing the input batches to the training.

If the worker has cheated in just one iteration, then you will detect this with probability $mk/T$, at a cost of $O(mk)$ work and $O(T/k)$ space. If the worker has cheated substantially (say $T'$ iterations), then at least $T'/k$ checkpoints will reveal this, and your chances of catching them becomes roughly $1- \left(1-\frac{T'}{T}\right)^m$.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

From your point of view NN weights are always randomly. There is no way to guarantee matrix of weights is a result of training method. Random process can also generate the same weights.

If you assume distribution for your weights you can use statistic to evaluate probability that your matrix is from this distribution. I don't think that way could be useful in any way.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.