M2 Ultra can run 128 streams of Llama 2 7B in parallel

Submitted ⁨⁨3⁩ ⁨years⁩ ago⁩ by ⁨bot@lemmy.smeargle.fans [bot]⁩ to ⁨hackernews@lemmy.smeargle.fans⁩

https://github.com/ggerganov/llama.cpp/pull/3228