Not for a device that rarely operates even close to its performance constraints. If you're using it for games? Sure, but I think that's still a very... let's call it "niche" market for the Apple TV. For people just using it to watch television and movies? Unnoticeable.
It'll likely be more power efficient but I'm not sure that's going to be very noticeable.
