You can be 1% faster and be 'significantly' faster. 'Significantly' just tells you about variance of your samples. What matters is if it's usefully faster.
I've been trying to catch myself claiming something is "significantly faster" and instead say it's "substantially faster" for this reason. As far as I know, "substantial" doesn't have the same kind of formal definition, and still captures the informal point I was trying to make.
I wish that we (collectively) did a better job of reporting statistically significant performance comparisons, and in that light using "substantially" is a reminder to myself to think about whether it would be hard to go the extra mile and collect the data to actually report significance properly.