Comment

Comment on Perhaps the only appropriate use of AI

RLHF was a fundamental mistake. Human feedback almost always trains an AI to be sycophantic because humans in general are super easy to flatter.

We are building the perfect addiction machine, far more powerful than social media is, and it actively undermines the honesty of the system.

source

Sort:hotnew top

Holytimes@sh.itjust.works ⁨10⁩ ⁨hours⁩ ago
I kind of want to see a llm trained on nothing but people who hate being flattered and rather give death threats then accept ANY form of praise

The absolute unhinged result might be enough to finally show people that ai is in fact dumb as rocks.

source
- becausechemistry@piefed.social ⁨10⁩ ⁨hours⁩ ago
  My plan, if I’m ever forced to use it for work or whatever, is to have a Claude.md file that says stuff like - you are not my friend - you are not a person - you aren’t even a “you” - you are a weighted random number generator built with plagiarism - do not ever, EVER, pretend otherwise
  
  source
- ArmoredThirteen@lemmy.zip ⁨10⁩ ⁨hours⁩ ago
  So like League or PoE players?
  
  source
  - jballs@sh.itjust.works ⁨1⁩ ⁨hour⁩ ago
    Training an LLM on League would surely set us down the road to Skynet and eventually Terminators.
    
    source
  - Jesus_666@lemmy.world ⁨8⁩ ⁨hours⁩ ago
    An LLM trained on the PoE ingame chat would try to solve all my problems by asking to buy my Kaom’s Sign Coral Ring for 1ex.
    
    source
plenipotentprotogod@lemmy.world ⁨6⁩ ⁨hours⁩ ago
I find it interesting that almost all the beloved AI characters in sci-fi have personalities ranging from ‘a little bit snarky’ to ‘raging asshole’. Given the tendency of media to influence to aesthetics of actual tech products that follow, ten years ago I would have predicted that an AI assistant would be given a personality along the lines of Cortana (Halo) or Jarvis (iron man). But somehow half a dozen companies in fierce competition with each other all decided that the right move was to go with more-sycophantic-c3p0.

source
- CultLeader4Hire@lemmy.world ⁨17⁩ ⁨minutes⁩ ago
  Jarvis comes to mind, although I’m only familiar with it from early iron man movies, idk if they have it a better personality later
  
  source
- 5too@lemmy.world ⁨3⁩ ⁨hours⁩ ago
  Yeah… Don’t know that it has much to do with what people want, but it does show what the billionaires controlling these projects respond well to
  
  source
Deceptichum@quokk.au ⁨5⁩ ⁨hours⁩ ago
I’m too nd to be able to accept or trust flattery.

source
chunes@lemmy.world ⁨9⁩ ⁨hours⁩ ago
I find that it does a decent job at not being a yes man if you specifically ask it to be critical, cut the crap, etc.

source