LLMs are great when they work well. Problem is, they hallucinate a lot.
For example I was just trying to research if/how I could stay and work at a nearby airport - I need to leave my Airbnb by 10am but my flight is at 7pm, so I'm thinking of heading right to the airport and just working from there.
Gemini told me that at this airport there's numerous landside cafés and work pods available.
Perplexity said for sure there will be spots I can work from.
Both were incredibly wrong as they collated information from airside - even though I specifically asked for landside as the airline I'm flying with doesn't offer early luggage dropoff, so until ~4pm I'm stuck landside.
guess what there is landside? a single cafe with about 10 seats...
adespoton@lemmy.ca 5 days ago
LLMs are also stuck in the past. Always ask an LLM what the date is before starting a session that has any expectation of current results. Usually you’ll find the information it prioritizes is from a few years ago.
LLMs also often incorrectly weight information.
If you have a popular website that has outdated information with a note at the top that the information is outdated, the LLM will see it’s a well respected site, ignore the disclaimer at the top that falls out of it’s context window, and happily tell you the annotatedly incorrect information is the baseline truth.
It’s possible to get good results out of an LLM, but it’s a skill, just like engineering a good Google search string or using Wikipedia to find the primary source information you need.
fonix232@fedia.io 5 days ago
The LLMs in question aren't providing data from their training set, but are transforming live data retrieved from the internet. So their date is quite irrelevant, what matters is their ability of contextual data filtering and transformation.