Not totally sure, but based on other folks' comments a number of external sources of noise are possible, e.g. from other conversations or from hitting a weird spot in the training data.