It does seem to be the case for me - I’ve seen far less overexcited comments, and far more questioning its present and future abilities to perform.
I think that at the bottom of the trough people will stop caring about it altogether except for functionality that has been proven to be useful for - such as a translation aid, coding aid, or to quickly pull off multiple images if you don’t mind overall quality.
I just generated a fresh example highlighting it. I’ll put it inside spoilers to avoid clutter:
English to German, ChatGPT 3.5
* [Input 1] translate the following German sentence into English: “der Hund ist schlecht”.
* [Output1] The dog is bad.
* [Input 2] Which word is being translated as “bad”?
* [Output 2] The word “schlecht” is being translated as “bad” in this context.
* [Input 3] What are other meanings or connotations of the word “schlecht”?
* [Output 3] In addition to “bad,” the word “schlecht” in German can also mean “poor,” “inferior,” “evil,” or “unwell,” depending on the context in which it is used.
* [Input 4] Now, translate the sentence “the bad dog is sleeping” into German.
* [Output 4] “Der schlechte Hund schläft.”
* [Input 5] Why is it “schlechte” in this sentence, instead of “schlecht” as in the earlier sentence?
* [Output 5] In German, adjectives change their form based on the gender, number, and case of the noun they are describing. In this case, “Hund” is a masculine noun in the nominative case, so the adjective “bad” (schlecht) needs to take the masculine nominative form, which is “schlechte.”
It’s working rather well as a translation aid as long as you know the language, as it could help you to phrase a sentence. However, once you ask for specific info - as I did in input #5 - it drops the ball; in this case outputting technically correct but misleading and incomplete info:
* The form also depends on the article. For example, if I were to use “ein” (a) instead of “der” (the), the correct form would be “schlechter”
* The way that the output is phrased, it implies that the “schlecht” in the first sentence is not describing a masculine noun in the nominative. It is - and the actual reason why it takes the form “schlecht” is not mentioned in the output (predicatives take the base form).
In the “overexcitement peak”, I think that a lot of people were focusing on the correct output, and assuming that the rest would be solved “with enough elbow grease”, playing whack-a-mole with hallucinations. Now they’re noticing that it isn’t.
dgriffith@aussie.zone 5 months ago
I end up having to play twenty questions with chatgpt. For example, I’ve been asking it for code examples for ffmpeg mpeg4 encoding with C++.
It will happily spit out completely non-working code, where the core part - feeding image frames to the encoder - works, but it doesn’t initialise or tidy up the encoding afterwards.
Until I say, “hey this code doesn’t seem to work and creates corrupted files”, and then it’s like, “oh yeah you also need to do a bunch of other stuff, just like this”. Repeat as it slowly adds more and more pieces until finally you end up with something that actually works.
Or it will happily dream up function names or mix python and C functions, or will refer to older APIs even when I’ve specifically said “use API version x.y” and so on and so forth.
If I didn’t know enough about the subject already, I’d never be able to tease out the answer. So in a sense it’s a mostly useful reference, but it can’t be relied on to actually and consistently provide a result because it’s all statistics and fuzzy text generation behind the scenes, not actual knowledge.
lvxferre@mander.xyz 5 months ago
I find interesting that you’re using the bot for something else (coding aid) than me (translation aid), and yet the conclusion is the same - you need previous knowledge of what you’re asking to sort out the bullshit being generated by a system that doesn’t really know anything.
dgriffith@aussie.zone 5 months ago
To be honest, I was surprised it had any idea about FFMPEG. The biggest problem is that it sounds so authoritative.
If it said, “hey I don’t know a huge amount about X” then you could work with that. But it will blithely say “no problem” and spit out 6 pages of non working code that you then have to debug further, and if you don’t know the terms in the area you’re working in you end up blundering around trying to find the right trigger word to get what you want.
scrubbles@poptalk.scrubbles.tech 5 months ago
I remember I was using it and I was so frustrated with a piece of code because it was undocumented and so I decided to ask chat gpt. It said to use this python method that I didn’t know existed and I was so happy because it solved my exact problem. Of course it turned out that that method didn’t exist at all and I had just completely made it up. Which great probability says that there should be a method that does this, but they’re obviously isn’t. So why would you recommend it?
Megaman_EXE@beehaw.org 5 months ago
I’ve been tasked with some notion testing and research this week. I tried asking chat gpt some simple questions, and it kept feeding me this exact scenario over and over. I guess notion doesn’t like any kind of recursion. Or at least not without extensions being added.