In the following, Opus bombed hard by ignoring the "when" component, replying with "MemoryStream"; where ChatGPT (I think correctly) said "no":
> In C#, is there some kind of class in the standard library which implements Stream but which lets me precisely control when and what the Read call returns?
---
In the following, Opus bombed hard by inventing `Task.WaitUntilCanceled`, which simply doesn't exist; ChatGPT said "no", which actually isn't true (I could `.ContinueWith` to set a `TaskCancelationSource`, or there's probably a way to do it with an await in a try-catch and a subsequent check for the task's status) but does at least immediately make me think about how to do it rather than going through a loop of trying a wrong answer.
> In C#, can I wait for a Task to become cancelled?
---
In the following exchange, Opus and ChatGPT both bombed (the correct answer turns out to be "this is undefined behaviour under the POSIX standard, and .NET guarantees nothing under those conditions"), but Opus got into a terrible mess whereas ChatGPT did not:
> In .NET, what happens when you read from stdin from a process which has its stdin closed? For example, when it was started with { ./bin/Debug/net7.0/app; } <&-
(both engines reply "the call immediately returns with EOF" or similar)
> I am observing instead the call to Console.Read() hangs. Riddle me that!
ChatGPT replies with basically "I can't explain this" and gives a list of common I/O problems related to file handles; Opus replies with word salad and recommends checking whether stdin has been redirected (which is simply a bad answer: that check has all the false positives in the world).
---
> In Neovim, how might I be able to detect whether the user has opened Neovim by invoking Ctrl+X Ctrl+E from the terminal? Normally I have CHADtree open automatically in Neovim, but when the user has just invoked $EDITOR to edit a command line, I don't want that.
Claude invents `if v:progname != '-e'`; ChatGPT (I think correctly) says "you can't do that, try setting env vars in your shell to detect this condition instead"
Now I come to think of it, maybe the problem is that I only ask these engines questions whose answers are "what you ask is impossible", and ChatGPT copes well with that condition but Opus does not.
---
In the following, Opus bombed hard by ignoring the "when" component, replying with "MemoryStream"; where ChatGPT (I think correctly) said "no":
> In C#, is there some kind of class in the standard library which implements Stream but which lets me precisely control when and what the Read call returns?
---
In the following, Opus bombed hard by inventing `Task.WaitUntilCanceled`, which simply doesn't exist; ChatGPT said "no", which actually isn't true (I could `.ContinueWith` to set a `TaskCancelationSource`, or there's probably a way to do it with an await in a try-catch and a subsequent check for the task's status) but does at least immediately make me think about how to do it rather than going through a loop of trying a wrong answer.
> In C#, can I wait for a Task to become cancelled?
---
In the following exchange, Opus and ChatGPT both bombed (the correct answer turns out to be "this is undefined behaviour under the POSIX standard, and .NET guarantees nothing under those conditions"), but Opus got into a terrible mess whereas ChatGPT did not:
> In .NET, what happens when you read from stdin from a process which has its stdin closed? For example, when it was started with { ./bin/Debug/net7.0/app; } <&-
(both engines reply "the call immediately returns with EOF" or similar)
> I am observing instead the call to Console.Read() hangs. Riddle me that!
ChatGPT replies with basically "I can't explain this" and gives a list of common I/O problems related to file handles; Opus replies with word salad and recommends checking whether stdin has been redirected (which is simply a bad answer: that check has all the false positives in the world).
---
> In Neovim, how might I be able to detect whether the user has opened Neovim by invoking Ctrl+X Ctrl+E from the terminal? Normally I have CHADtree open automatically in Neovim, but when the user has just invoked $EDITOR to edit a command line, I don't want that.
Claude invents `if v:progname != '-e'`; ChatGPT (I think correctly) says "you can't do that, try setting env vars in your shell to detect this condition instead"