"So if you spawn 200 processes of Swift compilers"
I don't think that implies those processes all were spawned at the same time (or even that Apple's compilation process did the spawning. Given the effort they spent on their build processes, I do not rule out it is their tooling that does the spawning)
The thing is: all these processes are independent of each other. Uber likely has tons of files that are imported into many, many files. Each compilation process that imports such a file has to individually parse it.
I can see two ways around this: reusing intermediate results between processes and using fewer processes.
The first is like precompiled headers in C/C++. The second could mean having a queue of to be compiled files and a fixed number of compilation processes that pick up files from it.
Risk in both cases is that unintended state may leak between compilations. For example, compilation flags may be different for files f and g. Does that mean g can't use preprocessed module m generated while compiling f? Because of that, I would go for something like precompiled headers, because it makes it easier to reason about what information flows between compilations of f and g.
I don't know whether that would be sufficient, though. The biggest concern I have w.r.t. Swift is that all its cool features, combined, make it essentially impossible to build a fast compiler (Yes, languages such as C# have most of them and aren't that slow, but adding protocols and overflow detection (which you may want to optimize away a lot, even in unoptimized builds) may just tip the balance)
I don't think that implies those processes all were spawned at the same time (or even that Apple's compilation process did the spawning. Given the effort they spent on their build processes, I do not rule out it is their tooling that does the spawning)
The thing is: all these processes are independent of each other. Uber likely has tons of files that are imported into many, many files. Each compilation process that imports such a file has to individually parse it.
I can see two ways around this: reusing intermediate results between processes and using fewer processes.
The first is like precompiled headers in C/C++. The second could mean having a queue of to be compiled files and a fixed number of compilation processes that pick up files from it.
Risk in both cases is that unintended state may leak between compilations. For example, compilation flags may be different for files f and g. Does that mean g can't use preprocessed module m generated while compiling f? Because of that, I would go for something like precompiled headers, because it makes it easier to reason about what information flows between compilations of f and g.
I don't know whether that would be sufficient, though. The biggest concern I have w.r.t. Swift is that all its cool features, combined, make it essentially impossible to build a fast compiler (Yes, languages such as C# have most of them and aren't that slow, but adding protocols and overflow detection (which you may want to optimize away a lot, even in unoptimized builds) may just tip the balance)