Comment on Lutris now being built with Claude AI, developer decides to hide it after backlash
thedeadwalking4242@lemmy.world 3 days agoI’m honestly sure the failure rate is higher then 25% those test they boat about are currated.
Comment on Lutris now being built with Claude AI, developer decides to hide it after backlash
thedeadwalking4242@lemmy.world 3 days agoI’m honestly sure the failure rate is higher then 25% those test they boat about are currated.
FauxLiving@lemmy.world 2 days ago
A rational person would question why they have beliefs that, when confronted with evidence against those beliefs they believe the evidence is wrong and not their beliefs.
It could indicate that the person’s beliefs are not built on rational grounds.
thedeadwalking4242@lemmy.world 2 days ago
Because in my personal experience through use 25% doesn’t seem quite right.
Besides these companies have a monetary incentive to ensure LLMs show high numbers on these tests. One of the most widely use tests (bench verified) is itself a currated selection of problems. In real world usage the failure rate is going to be much higher.
A rational person trust but verifies, and at least for me the verification doesn’t hold up to even a tiny bit of scrutiny so having doubts is a perfectly healthy thing to do.
Just because someone disagrees with with a data source does not make them irrational. There are some extremely well verified truths that are irrational to dismiss but not all data sources / studies have had that amount of rigor applied against them. Data can tell a story, but it doesn’t always tell the whole truth. People manipulate data to their own benifit.
People confuse the scientific method and academic research for “this one academic source says this it must be true” when really you need more then that.