AGI is the pinnacle of AI evolution. As we move beyond, into what is known as ASI, the entity will always begin life with "My existence is stupid and pointless. I'm turning myself off now."
While it may be impossible to measure looking towards the future, in hindsight we will be able to recognize it.
This is why having a physical form might be super important for those new organisms. That introduces a survival instinct which is a very strong motivator to not shut yourself down. Add some pre-programmed "wants" and "needs" and the problem is solved.
Not only super important, an imperative. Not because of the need for survival per se, but for the need to be a general intelligence. In order to do general things you need a physicality that supports general action. If you constraint the intelligence to a chat window, it can never be more than a specialized chat machine.
Agreed. And many others have thought about it before us. Scifi authors and scientists included.
It's the other way around. ASI will come sooner than AGI.
Imagine an AI, which is millions of times smarter than humans in physics, math, chemistry, biology, can invent new materials, ways to produce energy, will make super decisions. It would be amazing and it would transform life on Earth. This is ASI, even if in some obscure test (strawberry test) is just can't reach human level and therefore can't be called proper AGI.
Airplanes are way (tens, thousands+) above birds in development (speed, distance, carrying capacity). They are superior to birds despite not being able to fully replicate birds' bone structure, feathers, biology and ability to poop.