Comment on In heat

serenissi@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

• _stranger_@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

  Because you’re not getting an answer to a question, you’re getting characters selected to appear like they statistically belong together given the context.

  source

  • howrar@lemmy.ca ⁨2⁩ ⁨weeks⁩ ago

    A sentence saying she had her ovaries removed and that she is fertile don’t statistically belong together, so you’re not even getting that.

    source

    • JcbAzPx@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

      You think that because you understand the meaning of words. LLM AI doesn’t. It uses math and math doesn’t care that it’s contradictory, it cares that the words individually usually came next in it’s training data.

      source

      • howrar@lemmy.ca ⁨2⁩ ⁨weeks⁩ ago

        It has nothing to do with the meaning. If your training set consists of a bunch of strings consisting of A’s and B’s together and another subset consisting of C’s and D’s together (i.e. [AB]+ and [CD]+ in regex) and the LLM outputs “ABBABBBDA”, then that’s statistically unlikely because D’s don’t appear with A’s and B’s. I have no idea what the meaning of these sequences are, nor do I need to know to see that it’s statistically unlikely.

        In the context of language and LLMs, “statistically likely” roughly means that some human somewhere out there is more likely to have written this than the alternatives because that’s where the training data comes from. The LLM doesn’t need to understand the meaning. It just needs to be able to compute probabilities, and the probability of this excerpt should be low because the probability that a human would’ve written this is low.

        source

        • -> View More Comments
      • _stranger_@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

        It’s not even words, it “thinks” in “word parts” called tokens.

        source
• Cordyceps@sopuli.xyz ⁨2⁩ ⁨weeks⁩ ago

  And the text even ends with a mention of her being in early menopause…

  source