I remember I was using it and I was so frustrated with a piece of code because it was undocumented and so I decided to ask chat gpt. It said to use this python method that I didn’t know existed and I was so happy because it solved my exact problem. Of course it turned out that that method didn’t exist at all and I had just completely made it up. Which great probability says that there should be a method that does this, but they’re obviously isn’t. So why would you recommend it?
I end up having to play twenty questions with chatgpt. For example, I’ve been asking it for code examples for ffmpeg mpeg4 encoding with C++.
It will happily spit out completely non-working code, where the core part - feeding image frames to the encoder - works, but it doesn’t initialise or tidy up the encoding afterwards.
Until I say, “hey this code doesn’t seem to work and creates corrupted files”, and then it’s like, “oh yeah you also need to do a bunch of other stuff, just like this”. Repeat as it slowly adds more and more pieces until finally you end up with something that actually works.
Or it will happily dream up function names or mix python and C functions, or will refer to older APIs even when I’ve specifically said “use API version x.y” and so on and so forth.
If I didn’t know enough about the subject already, I’d never be able to tease out the answer. So in a sense it’s a mostly useful reference, but it can’t be relied on to actually and consistently provide a result because it’s all statistics and fuzzy text generation behind the scenes, not actual knowledge.
scrubbles@poptalk.scrubbles.tech 5 months ago
Megaman_EXE@beehaw.org 5 months ago
I’ve been tasked with some notion testing and research this week. I tried asking chat gpt some simple questions, and it kept feeding me this exact scenario over and over. I guess notion doesn’t like any kind of recursion. Or at least not without extensions being added.
lvxferre@mander.xyz 5 months ago
I find interesting that you’re using the bot for something else (coding aid) than me (translation aid), and yet the conclusion is the same - you need previous knowledge of what you’re asking to sort out the bullshit being generated by a system that doesn’t really know anything.
dgriffith@aussie.zone 5 months ago
To be honest, I was surprised it had any idea about FFMPEG. The biggest problem is that it sounds so authoritative.
If it said, “hey I don’t know a huge amount about X” then you could work with that. But it will blithely say “no problem” and spit out 6 pages of non working code that you then have to debug further, and if you don’t know the terms in the area you’re working in you end up blundering around trying to find the right trigger word to get what you want.