> To get a real comparison they should have written a third system using their new architecture and the old langauge, but that would of course be hard to justify outside of academic research.

Good point. You need a control group to make sure you're measuring the thing you think you're measuring.