adespoton@lemmy.ca 5 days ago
Wikipedia: it’s an encyclopedia. Fine for a general overview of a topic, but you need to follow it to primary sources if you want to make an authoritative argument.
Google: it’s got an AI summary at the top and a bunch of SEO’d results on the first pages.
LLMs: really good at translating a lot of content down into something that’s easy to read. Not necessarily easy to understand, not necessarily accurate, not citing it’s sources accurately, but easy to read.
So: where do people’s attitudes come from towards them?
We now have 25 years of Wikipedia. That means that for 25 years, anyone in school from K through university has had it drilled i to them “you can’t use Wikipedia as a primary source!” Which is often interpreted by kids (now adults) as “don’t trust Wikipedia!”
Google has been around for 28 years. When it started, the other search engines always missed things, had a bunch of ads, and were slow. Google was this fast clean interface that could instantly find whatever you were looking for on the world wide web, and the exact human created content you wanted would almost always be featured on the first page of results. People who grew up with that might be slow to catch on to the fact that Google today doesn’t do that. So they trust the results and assume the information they’re looking for must be there somewhere on the first page.
LLMs are new. They hold the promise of early Google in that they crawled all the source material for you and present a summary so you don’t even have to decide which link has the right answer. They haven’t been around long enough for a generation to be trained to distrust the messages they provide.
fonix232@fedia.io 5 days ago
LLMs are great when they work well. Problem is, they hallucinate a lot.
For example I was just trying to research if/how I could stay and work at a nearby airport - I need to leave my Airbnb by 10am but my flight is at 7pm, so I'm thinking of heading right to the airport and just working from there.
Gemini told me that at this airport there's numerous landside cafés and work pods available.
Perplexity said for sure there will be spots I can work from.
Both were incredibly wrong as they collated information from airside - even though I specifically asked for landside as the airline I'm flying with doesn't offer early luggage dropoff, so until ~4pm I'm stuck landside.
guess what there is landside? a single cafe with about 10 seats...
adespoton@lemmy.ca 5 days ago
LLMs are also stuck in the past. Always ask an LLM what the date is before starting a session that has any expectation of current results. Usually you’ll find the information it prioritizes is from a few years ago.
LLMs also often incorrectly weight information.
If you have a popular website that has outdated information with a note at the top that the information is outdated, the LLM will see it’s a well respected site, ignore the disclaimer at the top that falls out of it’s context window, and happily tell you the annotatedly incorrect information is the baseline truth.
It’s possible to get good results out of an LLM, but it’s a skill, just like engineering a good Google search string or using Wikipedia to find the primary source information you need.
fonix232@fedia.io 5 days ago
The LLMs in question aren't providing data from their training set, but are transforming live data retrieved from the internet. So their date is quite irrelevant, what matters is their ability of contextual data filtering and transformation.