Equality of outcome could even eventually lead to an objectively worse outcome for society as a whole when on a larger time scale due to holding back brilliant minds.

Those who were clearly brilliant and may have been entirely capable of pushing societal, technological, medical etc. advances forward in a larger time scale are held back, stifled, or even in cases of things like affirmative action (which I believe should exist, but only on the economic level, not on the basis of race or identity) have been denied of opportunity to go on and do great things.