Apart from just a general dislike of LLMs, what specifically do you believe would make this particular use prone to errors?
Comment on AI Seeks Out Racist Language in Property Deeds for Termination
knightly@pawb.social 2 days agoGiven the error rate of LLMs, it seems more like they wasted $258 and a week that could have been spent on a human review.
t3rmit3@beehaw.org 2 days ago
knightly@pawb.social 2 days ago
The use of LLMs instead of someone that can actually understand context.
t3rmit3@beehaw.org 2 days ago
I think you may have misunderstood the purpose of this tool.
It doesn’t read the deeds, make a decision, and submit them for termination all on its own. It reads them, identifies racial covenants based on patterns of language (which is exactly what LLMs are very good at), and then flags them for a human to review.
This tool is not replacing jobs, because the whole point is that these reviews were never going to get the budget and manpower to be done manually, and instead would have simply remained on the books.
I get being disdainful or even angry about LLMs in our unregulated-capitalism anti-worker hellhole because of the way that most companies are using them, but tools aren’t themselves good or bad, they’re just tools. And using a tool to identify racial covenants in legal documents that otherwise would go un-remediated, seems like a pretty good use to me.
knightly@pawb.social 2 days ago
So, what? They’re going to pay a human to OK the output and the whole lot of them never even gets seen?
Say 12 minutes per covenant, that’s 1 million work hours that humans could get paid for. Pay them $50 an hour and it’s $50 million. That’s nothing, less than 36 hours worth of the $12.5 Billion in weapons shipments we’ve sent to Israel in the last year. We could pay for projects like this with the rounding errors on the budget for blowing up foreign kids, and the people we pay to do it could afford to put their kids through college.
Instead, we get a project to train a robotic bigotry filter for real estate legalese and 50 more cruise missiles from the savings.
GetOffMyLan@programming.dev 2 days ago
One of LLMs main strengths over traditional text analysis tools is the ability to “understand” context.
They are bad at generating factual responses. They are amazing at analysing text.
knightly@pawb.social 2 days ago
LLMs neither understand nor analyze text. They are statistical models of the text they were trained on. A map of language.
And, like any map, they should not be confused for the territory they represent.
If you admit that they have issues with facts, why would you assume that the randomly generated facts their “analysis” produces must be accurate?
OmnipotentEntity@beehaw.org 2 days ago
LLMs are bad for the uses they’ve been recently pushed for, yes. But this is legitimately a very good use of them. This is natural language processing, within a narrow scope with a specific intention. This is exactly what it can be good at. Even if does have a high false negative rate, that’s still thousands and thousands of true positive cases that were addressed quickly and cheaply, and that a human auditor no longer needs to touch.