Submitted 4 days ago by ryujin470@fedia.io to technology@beehaw.org
If GPT (decoder-only transformer) models are text predictors, then why keyboard apps on PCs/phones don't have GPTs as text predictor options? They can be more accurate than the widely used N-gram models.
They are, and have been since before GPT existed
Yeah, also true. Autocorrect on Android was already like a micro GPT
They’re not text predictors; they’re text transformers. So, more for translating a text from French to English, or providing you a summary of what a large input text is trying to say.
They can do prediction, but not well, and not without a lot of extra computation, as they’re really trying to summarize everything you’ve said, including the bit you haven’t written yet.
Have you heard about futo keyboard? It embeds a small LLM model for this very task. It also has an option to retrain based on user data.
TachyonTele@piefed.social 4 days ago
Probably power. It takes a lot to power the text predictors. Adding it to every single phone would probably mean a huge uptick in resource usage.
I’m just guessing though.
iloveDigit@piefed.social 4 days ago
You’re absolutely correct. Battery and RAM usage too hard to manage for now, phone makers will probably wait until the tech gets further before experimenting with this publicly