There may be exceptions but everything I’ve seen from AI programming is next level trash. It’s like copy pasting from Stack Overflow without the thousand comments all around it saying DO NOT DO THIS!
When ChatGPT was just released to the general public I wanted to try it out. I had it write a script to handle some simple parsing of network log files. I was having some intermittent issue with my home network I couldn’t figure out, so I had logged a lot of data and was hoping to figure out the issue. But I needed to filter out all the routine stuff that would be just noise in the background. I could have written it myself in about an hour, but figured hey maybe ChatGPT can help me bang it out in a couple of minutes.
The code it wrote looked at a glance to be very good and I was impressed. However as I read it, it turned out to be total nonsense. It was using variables and declaring them after. Halfway the script it seemed to have switched to a completely different approach leaving some sort of weird hybrid between the two. At one point it had just inserted pseudo code instead of actual functional code. Every attempt to get it to fix it’s issues just made it worse. In the end I just wrote the script myself.
I’ve seen examples from other people who attempted to use it and it’s just bad. It’s like having a junior programmer high on weed writing your code, checking it and fixing it takes more time than just writing the code itself.
Then there’s the issue of copyright, a lot of the training data wasn’t licensed and stuff like Github Copilot want to add your data to it’s training set if you want to use it. That’s not OK on many levels and not even possible for people working on corporate codebases.
A lot of programmers work on big code bases, with things like best practices and code standards. Not only does the AI not know the codebase and thus wouldn’t know how to do a lot of stuff in that codebase, it also doesn’t know about the best practices and code standards. So for those kinds of situations it isn’t useful.
I feel like people ask it to do some first year student programming tutorial tasks and the result looks somewhat like what one would expect and conclude the thing can actually write code. It really can’t in reality and probably shouldn’t even if it could.
Kichae@lemmy.ca 8 months ago
Literally the worst possible usage. They’re syntax generators, not search engines, and not knowledge fonts.
Creesch@beehaw.org 8 months ago
I don’t know how to say this in a less direct way. If this is your take then you probably should look to get slightly more informed about what LLMs can do. Specifically, what they can do if you combine them with with some code to fill the gaps.
Things LLMs can do quite well:
These are all the building blocks for searching on the internet. If you are talking about local documents and such retrieval augmented generation (RAG) can be pretty damn useful.
Dark_Arc@social.packetloss.gg 8 months ago
That’s not entirely fair either though… They can incorrectly summarize, omit important information, or just make stuff up.
Creesch@beehaw.org 8 months ago
True, though that isn’t all that different from people doing knee jerk responses on the internet…
I am not claiming they are perfect, but for the steps I described a human aware of the limitations is perfectly able to validate the outcome. While still having saved a bunch of time and effort on doing an initial search pass.
All I am saying is that it is fine to be critical of LLM and AI claims in general as there is a lot of hype going on. But some people seem to lean towards the “they just suck, period” extreme end of the spectrum. Which is no longer being critical but just being a reverse fanboy/girl/person.
mikegioia@lemmy.ml 8 months ago
Ask it to search or list all of Bach’s children and you can see the wheels completely fall off. It was inventing people with actual fake birth and death dates and also gave me some of Handel’s children!
Creesch@beehaw.org 8 months ago
What do you mean by “it”? The chatGPT interface? Could be, but then you are also missing the point I am making.
After all, chatGPT is just one of the possible implementations of LLMs and indeed not perfect in how they implemented some things. In fact,
What I am saying is that it is possible to implement search functionality around LLMs that is both capable and can be reviewed by a human user to make sure it is not fucking up.
Let me demonstrate. I am doing some steps that you would normally automate with conventional code:
I asked chatGPT
It then responded with.
The following step I did manually, but is something you would normally have automated. I put the suggested query in google, I quickly grabbed the first 5 links and then put the following in chatGPT.
It then proceeded to give me the following answer
Going over the search results myself seems to confirm this list.
millie@beehaw.org 8 months ago
Gpt is fantastic at search. Like, check its work but it’ll check hundreds of pages of results way faster than you can.