Are you asking are systematic errors "priced-in"/"automatically represented" or are they hidden inside the sigma calculation?
Systematic errors can easily remain hidden. The faster-than-light neutrino had 6-sigma confidence[0], but 4 other labs couldn't reproduce the results. In the end it was attributed to fiber optic timing errors.
So if you don't know you have a system error, then you can very easily get great confidence in fundamentally flawed results.
No. As written in another comment, imagine trying to determine whether two brands of cake mixes have the same density by weighing them. If you always weigh one of the brands with a glass bowl, but the other one with a steel bowl, you'll get enormously high units of sigma, but in reality you've only proven that steel is heavier than glass.
