For about $2000, I picked up a Mac Studio with 96 gigs of RAM, which is effectively all VRAM thanks to Apple’s architecture. It doesn’t have the raw number-crunching power of the big GPUs but with all that space, you don’t need to worry much about the size of the model until they start getting really big, so it’s pretty easy and flexible.

I’m able to do basically everything that others are doing with AI, entirely locally. It generates text and writes code (it’s no Opus, but probably on par with the best of a year and a half ago), images, videos, songs, all that stuff (those last few are garbage but it’s basically the same level garbage that the cloud models are making). And I have total privacy, will never get a surprise price hike or lose access to a model I like, and know exactly how many bottles of water I’m using to cool it (zero).

It’s not heroin, it’s weed. There will be a market for it, but, like, you can also just make it yourself. 90% of what people want will just get done on device. I can’t see any way this turns into a trillion dollar industry.