Slight street sign modifications can completely fool humans too. Luckily:
1) most people aren't assholes so don't want to cause traffic accidents; and
2) we have laws and police organizations to track down and punish assholes who do
Except that in a world where self-driving vehicles are relying on visual recognition of signage, it would become obvious PRETTY QUICKLY if the modifications to a STOP sign made it look like a 65mph speedlimit, and it would be able to be treated exactly as severely as if someone had covered the stop sign with a 65mph speedlimit sign.
We're not talking about modifications causing a sign to accidentally be mistaken for something else, we are talking about deliberate modifications to road signs that cause vehicles to misinterpret them.
If you modified a sign so that most users still perceived it as a stop sign but colorblind people misread it as a speed limit you'd be doing exactly the same kind of thing.
I mean, it's not like there's some free-speech right to graffiti on road signage in the first place, let alone to modify it so that some road users will misunderstand the sign's meaning. If you interfere with a roadsign in order to deliberately confuse road users, you are a criminal.
