You missed a very important detail, the language runtime.
While Haskell's runtime is designed for Haskell needs, Clojure has to be happy with whatever JVM designers considered relevant for Java the language, the same on the other platforms targeted by Clojure.
This is yet another example of a platform being designed for a language, and being a guest language on a platform.
I don't think this is a limitation of the JVM. When I've used Clojure's STM implementation it's been perfectly serviceable (barring the composability issues I mentioned). Likewise when I've used the various STM libraries in Scala. Eta (basically a Haskell implementation on the JVM that unfortunately stalled in development) also had a fine STM implementation.
It's more of a combination of API and language decisions rather than the underlying JVM.