No, refactoring shouldn't change public interfaces. The very definition of refactoring is rewriting code without changing interfaces.
> Things are factored differently.
internally
> In computer programming and software design, code refactoring is the process of restructuring existing computer code—changing the factoring—without changing its external behavior.
If you're developing a library, then refactoring shouldn't change public interfaces. If you're developing an application and you own all the code paths to the code, then refactoring could change public interfaces, as the external behavior here would be the UI.
If you literally agree with the "refactoring shouldn't change public interfaces" then we need a new word for "code improvement which doesn't change external behavior, which can mean UI", which is the more commonly needed term.
And then perhaps we could agree that "code improvement often changes public interfaces" and how this relates to unit tests.
No, refactoring shouldn't change public interfaces. The very definition of refactoring is rewriting code without changing interfaces.
> Things are factored differently.
internally
> In computer programming and software design, code refactoring is the process of restructuring existing computer code—changing the factoring—without changing its external behavior.
https://en.wikipedia.org/wiki/Code_refactoring
You got the definition of refactoring wrong, please get it right, it's important. If you are breaking a public API, you are not refactoring anything.
Any piece of code meant to be private shouldn't be unit tested at all, only the behavior of a public interface.
Now internally you might call a third party lib, but that third party lib is then a separate "unit" itself.
I don't like the term "unit" because it's yet another word that is easily misunderstood and lost its original meaning with time.
unit testing should really mean "public interface behavioural integrity testing" or something like that.