I cannot think of anything worse than someone trying to talk to me while I'm at the gym. I'm very regimented, I've got specific exercises (sets and reps) and I'm watching my rest intervals to the second and mentally preparing for the next set. Just thinking about someone interrupting me between sets is giving me anxiety.

I guess it comes down to how you operate at the gym, is it a class, lifting, cardio? For me it would be really frustrating.