Comment on Do any languages have words for left & right that start with the same letter?
nandeEbisu@lemmy.world 1 year agoThey don’t process words as unified tokens for something like an LLM, but they do process them as multi-letter encoding, like byte-pair encoding or more advanced techniques.