So today is Qwen. Tomorrow a new SOTA model from Google apparently, R2 next week.
We haven't hit the wall yet.
Qwen 3 is coming imminently as well https://github.com/huggingface/transformers/pull/36878 and it feels like Llama 4 should be coming in the next month or so.
That said none of the recent string of releases has done much yet to "smash a wall", they've just met the larger proprietary models where they already were. I'm hoping R2 or the like really changes that by showing ChatGPT 3->3.5 or 3.5->4 level generational jumps are still possible beyond the current state of the art, not just beyond current models of a given size.
> met the larger proprietary models where they already were
This is smashing the wall.
Also if you just care about breaking absolute numbers, OpenAI released 4.5 a month back which is SOTA in base model, planning to release O3 full in maybe a month, and Deepseek released new V3 which is again SOTA in many aspects.
> We haven't hit the wall yet.
The models are iterative improvements, but I haven't seen night and day differences since GPT3 and 3.5
Yeah. Scaling up pretraining and huge models appears to be done. But I think we're still advancing the frontier in the other direction -- i.e., how much capability and knowledge can we cram into smaller and smaller models?
Because 3.5 has a new capability which is following instructions. Right now we are in 3.5 range in conversation AI and native image generation, both of which feels magical.
Google's announcements are mostly vaporware anyway. Btw, where is Gemini Ultra 1? how about Gemini Ultra 2?
It is already on the LLM arena right, codename nebula? But you are right they can fuck up their releases royally.
I guess they don’t do ultras anymore, but where was the announcement for it? What other announcement was vaporware?