Given the error rate of LLMs, it seems more like they wasted $258 and a week that could have been spent on a human review.
Comment on AI Seeks Out Racist Language in Property Deeds for Termination
t3rmit3@beehaw.org 2 months ago
Santa Clara County alone has 24 million property records, but the study team focused mostly on 5.2 million records from the period 1902 to 1980. The artificial intelligence model completed its review of those records in six days for $258, according to the Stanford study. A manual review would have taken five years at a cost of more than $1.4 million, the study estimated.
This is an awesome use of an LLM. Talk about the cost savings of automation, especially when the alternative was the reviews just not getting done.
knightly@pawb.social 2 months ago
OmnipotentEntity@beehaw.org 2 months ago
LLMs are bad for the uses they’ve been recently pushed for, yes. But this is legitimately a very good use of them. This is natural language processing, within a narrow scope with a specific intention. This is exactly what it can be good at. Even if does have a high false negative rate, that’s still thousands and thousands of true positive cases that were addressed quickly and cheaply, and that a human auditor no longer needs to touch.
t3rmit3@beehaw.org 2 months ago
Apart from just a general dislike of LLMs, what specifically do you believe would make this particular use prone to errors?
knightly@pawb.social 2 months ago
The use of LLMs instead of someone that can actually understand context.
t3rmit3@beehaw.org 2 months ago
I think you may have misunderstood the purpose of this tool.
It doesn’t read the deeds, make a decision, and submit them for termination all on its own. It reads them, identifies racial covenants based on patterns of language (which is exactly what LLMs are very good at), and then flags them for a human to review.
This tool is not replacing jobs, because the whole point is that these reviews were never going to get the budget and manpower to be done manually, and instead would have simply remained on the books.
I get being disdainful or even angry about LLMs in our unregulated-capitalism anti-worker hellhole because of the way that most companies are using them, but tools aren’t themselves good or bad, they’re just tools. And using a tool to identify racial covenants in legal documents that otherwise would go un-remediated, seems like a pretty good use to me.
GetOffMyLan@programming.dev 2 months ago
One of LLMs main strengths over traditional text analysis tools is the ability to “understand” context.
They are bad at generating factual responses. They are amazing at analysing text.
dan@upvote.au 2 months ago
The article didn’t say if it was an LLM or not.
howrar@lemmy.ca 2 months ago
Considering that it’s a language task, LLMs exist, and the cost, it’s a reasonable assumption. It’d be pretty silly to analyse a bag of words when you have tools you can use with minimal work with much better results. Even sillier to spend over $200 for something that can be run on a decade old machine in a few hours.
Killer_Tree@beehaw.org 2 months ago
Specialized LLMs trained for specific tasks can be immensely beneficial! I’m glad to see some of that happening instead of “Company XYZ is now needlessly adding AI to it’s products because buzzwords!”