No, things need to communicate the required information for the task at hand.
Limiting information can be harmful in many cases, and make things more difficult rather than simpler. And consistency can similarly limit usability.
What if I want to select one of 10 colors, with limited screen space? Isn't a select with color swatches perfect? Or even better, color swatches with color names beside? Why do you think that should be limited to plaintext only? That's anti-user.
Good example. There are dozens, maybe hundreds of different ways people have invented color pickers. Every app does it a little differently, meaning you have to stop and think about it every time. I'll take a simple select with the color name. If you want a swatch, put that off to the side and update it with an event handler on the select.
That's objectively worse, there's no way to know what "blue" means in such a context, there are many blues.
"I want things to be worse" is not a compelling argument. Letting developers describe the needs of their applications with a consistent grammar is not unnecessary invention, complexity, or friction.
The grammars of a technology, the set of building blocks with which developers are empowered to create, should balance flexibility with expressive intent. CSS <select> strikes this balance nicely. To decry its inclusion says more about the naysayer than the feature.
Names don't fully communicate color. There are hundreds of reds. Hundreds of greens.
I don't want to have to click each green to see the swatch. Just put them in a list with swatches. That is obviously the easier UX. Nobody has to stop and think about anything.
Windows had a standard color picker dialog that you may remember from Windows 98-Viata mspaint.
Jack of all trades, master of none. The mspaint-specific palette toolbar was much more intuitive.