Most people would never be able to tell a fake from a real watch, but a watch enthusiast can spot a fake almost immediately.
The interesting thing I find is that there are two groups of people we are talking about here: the average Joe who couldn't give a crap if you are wearing a $5 digital watch or a $500,000 Patek (90% of people), and wouldn't even notice you had something on your wrist at all. Then you have the watch enthusiasts who do look at people's watches and sometimes even compliment you on what you're wearing.
Now, observing the fact that the people that would compliment you on your watch or notice that it even exists is going to be someone who knows what they are looking at, wearing a fake means that for anyone who looks at it even once already knows it's a fake and will pass judgement appropriately.
It's why I've never really understood why someone would wear a fake. The people you are trying to impress are the ones that know it's not real and will think less of you as a result, meaning you were worse off than just wearing nothing at all!