If so-called AI is basically just Large Language Models, how come predictive text on my phone is bollock-useless?

Submitted ⁨⁨1⁩ ⁨year⁩ ago⁩ by ⁨Mr_Blott@lemmy.world⁩ to ⁨[deleted]⁩

Like if I type “I have two appl…” for example, often it will suggest “apple” singular instead of plural. Just a small example, but it is really bad at predicting which variant of a word should come after the previous

source

Comments

Sort:hotnew top

0x4E4F@sh.itjust.works ⁨1⁩ ⁨year⁩ ago
Phones don’t use LLM for predictive text. The algos are a lot less complex on phones.

source
- Knusper@feddit.de ⁨1⁩ ⁨year⁩ ago
  I guess, the real question is: Could we be using (simplistic) LLMs on a phone for predictive text?
  
  There’s some LLMs that can be run offline and which maybe wouldn’t use enormous amounts of battery. But I don’t know how good the quality of those is…
  
  source
  - bassomitron@lemmy.world ⁨1⁩ ⁨year⁩ ago
    The kind of local/offline LLMs that would work on your phone would not be very good quality. There’s been amazing progress of quantization of LLMs to get them working on weaker GPUs with lower VRAM and CPUs, so maybe it’ll occur, but I’m not an expert.
    
    I also don’t foresee them linking it up to a cloud-based LLM as that’d be a shit load of queries and extremely expensive.
    
    source
  - SpooksMcDoots@mander.xyz ⁨1⁩ ⁨year⁩ ago
    Openhermes 2.5 Mistral 7b competes with LLMs that require 10x the resources. You could try it out on your phone.
    
    source
  - Mr_Blott@lemmy.world ⁨1⁩ ⁨year⁩ ago
    That was my next question, thanks!
    
    Didn’t think of battery use, makes sense
    
    source
  - Munkisquisher@lemmy.nz ⁨1⁩ ⁨year⁩ ago
    A pre trained model isn’t going to learn how you type the more you use it. Though with Microsoft owning SwiftKey, I imagine they will try it soon
    
    source
    -> View More Comments
  - neptune@dmv.social ⁨1⁩ ⁨year⁩ ago
    I think apple has pitched this for a future iPhone, yes.
    
    source
    -> View More Comments
  - 0x4E4F@sh.itjust.works ⁨1⁩ ⁨year⁩ ago
    I guess… why not… but the db is probably huge, like in the hundreds of GB (maybe even TB… who knows), can’t run that offline.
    
    source
- Arthur@literature.cafe ⁨1⁩ ⁨year⁩ ago
  iOS 17 uses a small gpt-2 based model for predictive text.
  
  source
  - 0x4E4F@sh.itjust.works ⁨1⁩ ⁨year⁩ ago
    Hm, that’s interesting 👍.
    
    source
- Kichae@lemmy.ca ⁨1⁩ ⁨year⁩ ago
  The algorithms are the same. The models are different, being trained on a smaller data set.
  
  source
  - FooBarrington@lemmy.world ⁨1⁩ ⁨year⁩ ago
    No, the algorithms are not the same. Phones don’t use transformer models for text prediction, they use Markov chain-based approaches. Where do you get these ideas?
    
    source
  - 0x4E4F@sh.itjust.works ⁨1⁩ ⁨year⁩ ago
    Perhaps, I’m not a dev, especially not an iOS or Android one.
    
    source
Endorkend@kbin.social ⁨1⁩ ⁨year⁩ ago
This is an engine.

And this is an engine.

source
- FilthyHookerSpit@lemmy.world ⁨1⁩ ⁨year⁩ ago
  Succint
  
  source
squaresinger@feddit.de ⁨1⁩ ⁨year⁩ ago
Now guess how it feels to type German with predictive text. Most of our words can have more than a dozen different word endings depending on time and how the word is used. And that’s not taking into account that we use compound words, which word prediction pretty much cannot predict and often doesn’t even know. So spell check will mark a legal compound word as misspelled, because it doesn’t understand the concept of compound words and doesn’t know this specific word combination.

To show what I mean, the term “Danube steam boat captain’s hat” becomes “DonauDampfSchiffKapitänsMütze” (I added capital letters which shouldn’t be there to show where the next word in the compound word begins).

While this is an extreme example, it’s pretty common for compound words to consist of 4-5 words.

source
- agressivelyPassive@feddit.de ⁨1⁩ ⁨year⁩ ago
  And for some reason, some cases seem to be missing completely on my Android default keyboard. “untersuchst”, just like a bunch of second person cases for slightly unusual words is non existent.
  
  source
  - squaresinger@feddit.de ⁨1⁩ ⁨year⁩ ago
    Yeah, noticed that too. This is really annoying.
    
    source
  - Bigmouse@lemmy.world ⁨1⁩ ⁨year⁩ ago
    My favourite: ‘geröntgt’ which is the second participle of ‘röntgen’ to x-ray someone. Never heard it pronounced correctly by a native speaker.
    
    source
- max@feddit.nl ⁨1⁩ ⁨year⁩ ago
  Dutch also has the issue with the compound words. Autocorrect will often put a space in there, which is grammatically incorrect (and ugly). I feel like it’s at a point now where the incorrect space usage has become mainstream and might change the language rules. Oh well.
  
  source
sir_reginald@lemmy.world ⁨1⁩ ⁨year⁩ ago
LLMs are orders of magnitude more sophisticated and expensive to run. But don’t worry, I’m sure not so far in the future will see smaller LLMs being run on device to be used as autocorrect.

source
- pacoboyd@lemm.ee ⁨1⁩ ⁨year⁩ ago
  It would have to be pretty specific and small to work on a phone and I think a side effect would be everyone’s conversations start to sound a lot more homogeneous.
  
  source
  - neptune@dmv.social ⁨1⁩ ⁨year⁩ ago
    Even give years ago, Google had a keyboard that skimmed your emails and texts to start a bank of words you use to supplement it’s dictionary for autocorrect. Like if you are a chemist and send an email that includes the word “tetrahydrafuran” every couple month, it would be nice for your phone keyboard to just have it in the dictionary.
    
    source
    -> View More Comments
  - sir_reginald@lemmy.world ⁨1⁩ ⁨year⁩ ago
    you’re not wrong. Google just announced Gemini Nano that will run directly on the Pixel 8. Of course, it’s the first of it’s kind and will probably be slow and it’s not used as autocorrect yet. But just give it one year or two and it will probably be more common.
    
    source
- Mr_Blott@lemmy.world ⁨1⁩ ⁨year⁩ ago
  Can we have Scottish ones that know what a bawbag is, and when to put an “e” on the end of “shit”?
  
  Thanks!
  
  source
  - OpenStars@kbin.social ⁨1⁩ ⁨year⁩ ago
    Think of it from the LLM's perspective - in the general pool you have common English, you have less common variations such as this, and then you have whatever the heck people like Kid Rock are doing...
    
    Bawitdaba, da bang, da dang diggy diggy
    Diggy, said the boogie, said up jump the boogie
    
    source
asterfield@lemmy.world ⁨1⁩ ⁨year⁩ ago
LLMs like chatgpt take a wild amount of resources to run.

If you want something as smart as gpt3 and you want it to run at typing speeds, you’ll need a gaming PC running it.

People just recently managed to run gpt3 strength models at all on ordinary laptop hardware (slowly).

There is currently no way to run something gpt4 strength on ordinary consumer hardware (I’m just guessing but I think it takes a few hundred gb of VRAM to run)

source
CyanFen@lemmy.one ⁨1⁩ ⁨year⁩ ago
Because they’re using different tech. That’s like asking why do phone calls sound bad compared to voip calls. They’re just using different tech.

source
- Candelestine@lemmy.world ⁨1⁩ ⁨year⁩ ago
  Why can’t my riding lawnmower keep up with a Ferrari? They’re both vehicles.
  
  source
  - Knusper@feddit.de ⁨1⁩ ⁨year⁩ ago
    You’re in the No Stupid Questions community. Think about rule 7 in particular.
    
    source
    -> View More Comments
bioemerl@kbin.social ⁨1⁩ ⁨year⁩ ago
If humans are just brains why are we smarter than dogs who also have brains?

source
- Mr_Blott@lemmy.world ⁨1⁩ ⁨year⁩ ago
  It’s “no stupid questions”, so “no cunty answers” thanks
  
  source
  - bioemerl@kbin.social ⁨1⁩ ⁨year⁩ ago
    Well fuck you too pal, I thought it was a good analogy.
    
    source
    -> View More Comments
doublejay1999@lemmy.world ⁨1⁩ ⁨year⁩ ago
What the duck are you talking about?

source
Hanabie@sh.itjust.works ⁨1⁩ ⁨year⁩ ago
You’re comparing apples to oranges.

source