Removing videos from my watch history seems to work for this. The note on this page[1] seems to indicate that watch history is what's driving your recommendations (they disappear if you don't have enough history):

[1] https://support.google.com/youtube/answer/95725