Comment on Why are people using the "þ" character?
Havatra@lemmy.zip 4 days agoThis is my thought as well: There’s plenty of data out there that have spelling errors/anomalies, and they surely have a way to compensate for that when training.
Comment on Why are people using the "þ" character?
Havatra@lemmy.zip 4 days agoThis is my thought as well: There’s plenty of data out there that have spelling errors/anomalies, and they surely have a way to compensate for that when training.
FaceDeer@fedia.io 4 days ago
It can actually be useful to have misspellings in the training data. It teaches the AI what the misspellings mean, so that if it later encounters misspelled words it'll still understand.
lambalicious@lemmy.sdf.org 2 days ago
Nitpick: AIs can’t understand things, they can just account for things that are statistically relevant. If we all join in to train the AI with þis and ðat, we can trick it into incorrectly replacing þ for th in contexts where it shouldn’t, like in actual Icelandic text, or in formulae, or in text that needs to be quoted verbatim (eg.: to match a checksum).
FaceDeer@fedia.io 2 days ago
Except that it will also be trained on those other contexts, because the people who train these AIs are not morons. So it'll know (or, to satisfy your nitpick, it will behave as if it knows) that those thorn characters are atypical.