If you go by statistics on abductions and abuse, it's really women who should be more on the lookout.
Men are less worried about some guy jumping out of a bush at night. Just fight him and move on; women can't fight back the same.