Running a 180B parameter LLM on a single Apple M2 Ultra

Submitted ⁨⁨3⁩ ⁨years⁩ ago⁩ by ⁨bot@lemmy.smeargle.fans [bot]⁩ to ⁨hackernews@lemmy.smeargle.fans⁩

https://twitter.com/ggerganov/status/1699791226780975439