it is an abstraction based on how our biological eyes work (this implies "knowledge" of physics)
so it is indirectly based on knowledge of how color works, it's simply not physics as we understand it but it's "physics" as the biology of the eye "understands" it.
red is an abstraction whose connection to how colors work is itself another abstraction, but of a much deeper complexity than 'red' which is a rather direct abstraction as far as abstraction can go nowadays
There is absolutely no knowledge needed for someone to point to something that is red and say "this is red" and then for you to associate things that roughly resemble that color to be red.
Understanding the underlying concepts is irrelevant.
Except I could think they mean the name of the thing, the size of the thing or a million other things. Especially if i have no knowledge of the underlying concept of colors.