Two or more LLMs are running at once, one does the generation that you want and the other watches the output closely to see if it triggers one of their no-nos to do the censorship retroactively. If this sound inefficient, and absolutely terrible user interface design, you’re right.
Comment on time of day
jaybone@lemmy.zip 20 hours agoI wonder how they are implementing their censorship safeguards. Why would you let it even get that far?
tankfox@bookwyr.me 9 hours ago
hayvan@piefed.world 11 hours ago
Typically safeguards are also another, smaller model that only detects the forbidden stuff. That’s why Claude Mythos would reject when you ask it to find bugs in your code, but you could “jailbreak” it by asking it by asking it to fix the bugs instead.
FishFace@piefed.social 12 hours ago
They try to train the model to not answer “wrong”. But as we know, LLMs are not completely reliable and they can ignore their training. So to be sure, you need something that looks at the actual output and decides whether it’s ok, either with a dumb text search like old school filters, or perhaps with a second (likely smaller) language model.
The output needs to have been written for those post filters to act on it, so if it shows the output in real time, you see it!
Forester@pawb.social 20 hours ago
The " thoughts " must exist in order to be analyzed
bollvirtuoso@lemmy.world 19 hours ago
So, we’re rewarding them hiding their thoughts? Great.
bort@sopuli.xyz 13 hours ago
yes exactly. that’s like the central problem with AI alignment.