There's a handy addon for Firefox called Privacy Settings that can take care of that. Explicitly adds and option to have the referers be not sent, and a quick way of re-enabling it, in case it breaks a website. Because of course that happens too.