Comment on AI Is A Money Trap
Powderhorn@beehaw.org 13 hours agoI don’t think it’s going to come down to these absurd datacentres. We’re only a few years off from platform-agnostic local inference at mass-market prices. Could I get a 5090? Yes. Legally? No.
I have to think that most people won’t want to do local training.
It’s like Gentoo Linux. Yeah, you can compile everything with the exact optimal set of options for your kit, but at huge inefficiency when most use cases might be mostly served by two or three pre built options.
If you’re just running pre-made models, plenty of them will run on a 6900XT or whatever.
I don’t expect anyone other than … I don’t even know what the current term is … geeks? batshit billionaires? to be doing training.
I’m very much of the belief that our next big leap in LLMs is local processing. Once my interactions stay on my device, I’ll jump in.
Feyd@programming.dev 3 hours ago
What makes you confident in that? What will change?
Powderhorn@beehaw.org 58 minutes ago
There are already large local models. It’s a question of having the hardware, which has historically gotten more powerful with each generation. I don’t think it’s going to be phones for quite some time, but on desktop, absolutely.
Feyd@programming.dev 39 minutes ago
For business use, laptops without powerful graphics cards have been the norm for quite some time. Do you see businesses deciding to change to desktops to accommodate the power for local models? I think it’s pretty optimistic to think that laptops are going to be that powerful in the next 5 years. The advancement in chip capability has dramatically slowed, and to put them in laptops they’d need to be incredibly more power efficient as well.
Powderhorn@beehaw.org 16 minutes ago
For the security tradeoff of sensitive data not heading to the cloud for processing? Not all businesses, but many would definitely see value in it. We’re also discussing this as though the options are binary … models could also be hosted on company servers that employees VPN into.