For more inclusive models, or for current ones? In order to add something, either the size has to grow, or something would need to get pushed out (content, or quality). 4GB models are already at the limit of usefulness, both DALLE3 and SDXL run at about 12B parameters, so to make them “more inclusive” they’d have to grow.
jarfil@beehaw.org 1 year ago
Are you ready to run a 100B FP64 parameter model? Or even a 10B FP32 one?
Over time, I wouldn’t be surprised if 500B INT8 models became commonplace with neuromorphic RAM, but there’s still some time for that to happen.
Even_Adder@lemmy.dbzer0.com 1 year ago
You don’t need that many concepts, 4gb checkpoints work just fine.
jarfil@beehaw.org 1 year ago
For more inclusive models, or for current ones? In order to add something, either the size has to grow, or something would need to get pushed out (content, or quality). 4GB models are already at the limit of usefulness, both DALLE3 and SDXL run at about 12B parameters, so to make them “more inclusive” they’d have to grow.
Even_Adder@lemmy.dbzer0.com 1 year ago
I’m saying SD 1.5 and SDXL capture the concepts just fine, it’s just during fine-tuning people train away some of the diversity.