Humans do the same. Everything from medical studies to doctor trainings treat the straight white man as the "default human" and this obviously leads to all sorts of issues. Caroline Criado-Perez has an entire chapter about this in her book about systemic bias Invisible Women, with a scary number of examples and real world consequences.

It's no surprise that AI training sets reflect this also. People have been warning against it [0] specifically for at least 5 years.

0: https://www.pnas.org/doi/10.1073/pnas.1919012117

Edit: I've never had a comment so heavily downvoted so quickly. I know it's not the done thing to complain but HN really feels more and more like a boys club sometimes. Could anyone explain what they find so contentieus about what I've said?