There's a reason the "con" in "con man" stands for "confidence". Turns out the illusion of confidence is the easiest way to hack the human brain.
Except that's not what "confidence man" means. It means that you gain their confidence/trust, then betray it.
A con man often uses the illusion of confidence to gain trust, though that's not the only way. The reverse also works: gain their trust by seeming unconfident and incapable, and thus easily taken advantage of.