It's very different for men though, sad but true. Even as a well-known dad at my kids' child care, I've still seen funny looks from other parents and even the staff, when I'm helping my kids in the bathroom there, especially if other kids are in there.
Like crabby says, you can understand it (and even though there've been cases of women abusers, I'd say they're vastly outnumbered, so it's almost 'better safe than sorry'), but doesn't make it any easier to be looked at like that, and it's one of the main reasons there are so few men in those kind of jobs, because it's almost an assumption in some circles hat they're some sort of deviant trying to get access to kids, but it's well known how valuable good male role models can be for little kids, and more male early educators could really help that.
Reminds me of that ridiculous case recently where that poor geek dad (could so easily have been me!) took a photo of himself next to a Darth Vader cut-out at a shopping centre and some mum plastered it all over Facebook and accused him of being a pedo after kids, when he was just taking a nerdy photo for his kids. That's a kinda different issue (the danger of social media!) but still shows how knee-jerk some people can be about men near kids.
You often hear in the media how many males won't help a fallen/lost child in public for fear of being accused of something untoward (even some women feel that way of course, though usually fearing some other accusation than paedophilia).
It's hard... is it an overreaction or sensible caution?