But could you use GPL software to produce what you delivered (rather than as part of it)? If not were Linux, Emacs and use of Android phones for any work banned?

If they were banned because of the GPL the company was doing its own stupid rules completely unrelated to the restrictions imposed by the GPL. If they weren't then it shows the difference between the GPL and the non-commercial use term of this particular MS license.