Yes that's true, but no need to hold your crypto there as a permanent storage. Once your fiat is exchanged to crypto, immediately transfer the crypto to your private wallet.
This just trades the unsolved exchange hacking problem for the unsolved lost/stolen keys problem.
That problem is trivially solved by backups.
Backups don't solve seed phrase phishing for example.
As opposed to the bank's ...? Or your other account's ..., what exactly, passwords? Phising is everywhere. How many times have you heard the elderly have their money stolen, both online and in real life? It happened to my grandma. The mailman is bringing her own pension as cash, and guess what, he has scammed my grandma for years! The food delivery guy who has been delivering lunch for my grandma, guess what he did? He scammed my grandma out of her money! We are talking about cash, right now, and no phising involved, just good old "lying".
Hence why cryptocurrency would never replace regular banks for regular people. The situation with scams and thefts has only gotten worse. Not your keys, not your coin.
I definitely cannot imagine my grandma making use of crypto, or PayPal, or her bank's online site. :)
Theft or loss has always been a problem since life evolved on Earth.
I don't think anyone claimed that crypto was un-losable or un-stealable. It's not magic.