CV systems absolutely do struggle with this...
Yes, and that's considered a bug to fix, and can be. CV systems are widely applied to find camouflaged and obscured objects! Literally every automated wildlife camera is proof that this is a silly point.
Call it whatever you want, lack of contrast is a physical constraint on your system and can be trivially leveraged. There's been an arms race between optical camouflage and detectors for the entire history of eyes.
It's not necessarily a game ender, but it's not something you can just wave away either.