Miguel has a say.
I agree totally, but probably for different reasons. I was raised by a professional photographer. Whenever I see those vests, I think the person has 20 rolls of film and a few lenses on them.
Like Miguel says, 99% of the population has no idea what those are and the ones that do are more than likely the good guys. And without any stories of people being picked off for wearing those vests, much like the belief in ‘If you open carry, the criminals will just shoot you first!’, it’s more an excuse to pooh-pooh something some people don’t like more than anything based in fact.