0
$\begingroup$

I've started playing with BERT encoder through the huggingface module. enter image description here

I passed it a normal unmasked sentence and got the following results: enter image description here

However, when I try to manually apply the softmax and decode the output: enter image description here

I get back a bunch of unexpected tensor(1012) instead of my original sentence. BERT is an autoencoder, no?

Shouldn't it be giving me back the original sentence with fairly high probability since none of the input words was [MASK]? Can anyone explain to me what is going on?

Improve this question
$\endgroup$
3
  • $\begingroup$ Questions solely about how software works are off-topic here, but you may have a real statistical question buried here. You may want to edit your question to clarify the underlying statistical issue. You may find that when you understand the statistical concepts involved, the software-specific elements are self-evident or at least easy to get from the documentation. $\endgroup$
    – kjetil b halvorsen ♦
    Commented Jan 11 at 15:36
  • $\begingroup$ @kjetilbhalvorsen Sorry, but I have trouble with zero-shot learning. I need at least a sample size of one to understand what is expected of me. Could you please point out what is specifically wrong with my question or edit it for me so that I can mimic the formatting later on? Thanks. $\endgroup$
    – AlanSTACK
    Commented Jan 11 at 18:30
  • 1
    $\begingroup$ First, could you replace the screenshots in the post with actual text, please_ $\endgroup$
    – kjetil b halvorsen ♦
    Commented Jan 11 at 19:51

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.