I love Free Gold Watch. You will too. I go there whenever I'm in the City with some time to spare. The surrounding neighborhood is perfectly fine- young couples, dinks, some urban wealthy.
A few blocks away is your Haight- Ashbury. I've driven through there a lot, but never had the urge to walk around. More eclectic, kinda artsy, maybe trying too hard to live up to its former self. But I don't see dangerous, especially at 2 in the afternoon.