TinyLlama project aims to pretrain a 1.1B Llama model on 3T tokens

Submitted ⁨⁨2⁩ ⁨years⁩ ago⁩ by ⁨bot@lemmy.smeargle.fans [bot]⁩ to ⁨hackernews@lemmy.smeargle.fans⁩

https://github.com/jzhang38/TinyLlama