I suppose Swift is like a CLR language (e.g. C#) in that sense, then? The language itself will run in many places, but most of the library bindings anyone might care about, or want to use the language to get access to, are for platform-specific libraries.
For what it's worth, C# is very usable on both Linux and macos. Anything outside of UI is pretty much fully cross platform at this point (there are even some cross platform ui attempts, but they're not anywhere near as mature as wpf, xaml, winforms, etc)
These days there’s been a lot of movement towards cross platform C#, with .NET core etc. Whilst there isn’t any desktop application support (other than Xamarin), there’s a good ecosystem for ASP.NET.
