> warp is currently able to preprocess many source files, one after the other, in a single command. [...] warp is set up/torn down only once for the entire set of files, rather than once for each source file.
I'd like to learn more about this. I spend a fair amount of time building on HPC systems. Frustratingly, compiling on a $100M computer is typically 50x slower than compiling on a laptop due to the atrocious metadata performance of the shared file system. Configuration is even worse because there is typically little or no parallelism. Moving my source tree to a fast local disk barely helps so long as system and library headers continue to reside on the slow filesystem. A compiler system that transparently caches file accesses across an entire project build would save computational scientists an enormous amount of time building on these systems.
This is why I asked Walter about use of asynchronous idioms in Warp. It ought to help even if there is no parallelism involved. It is perhaps a TODO. Walter would be able to elaborate.
So your build speed is limited by metadata file accesses to network storage? Your network/storage must be really bad then. How about mirroring all the needed headers locally before building? You could establish that as a makefile rule.
I'd like to learn more about this. I spend a fair amount of time building on HPC systems. Frustratingly, compiling on a $100M computer is typically 50x slower than compiling on a laptop due to the atrocious metadata performance of the shared file system. Configuration is even worse because there is typically little or no parallelism. Moving my source tree to a fast local disk barely helps so long as system and library headers continue to reside on the slow filesystem. A compiler system that transparently caches file accesses across an entire project build would save computational scientists an enormous amount of time building on these systems.