Tiberium 5 days ago

There's a HUGE difference that you are not mentioning: there are "gpt-4o" and "chatgpt-4o-latest" on the API. The former is the stable version (there are a few snapshot but the newest snapshot has been there for a while), and the latter is the fine-tuned version that they often update on ChatGPT. All those benchmarks were done for the API stable version of GPT-4o, since that's what businesses rely on, not on "chatgpt-4o-latest".

1
yberreby 5 days ago

Good point, but how does that relate to, or explain, the decision not to release 4.1 in ChatGPT? If they have a nice post-training pipeline to make 4o "nicer" to talk to, why not use it to fine-tune the base 4.1 into e.g. chatgpt-4.1-latest?

Tiberium 5 days ago

Because chatgpt-4o-latest already has all of those improvements, the largest point of this release (IMO) is to offer developers a stable snapshot of something that compares to modern 4o latest. Altman said that they'd offer a stable snapshot of chatgpt 4o latest on the API, he perhaps did really mean GPT 4.1.

yberreby 5 days ago

> Because chatgpt-4o-latest already has all of those improvements

Does it, though? They said that "many" have already been incorporated. I simply don't buy their vague statements there. These are different models. They may share some training/post-training recipe improvements, but they are still different.