This is very neat! Last year I had to essentially do this on GCP and relied on a very similar implementation. Everyone was surprised to see JS being used for data processing but it worked wonderfully.
One thing I want to ask is the retries, how do you handle that currently? I ran into multiple cases where functions would fail for transient reasons.
Functions need to be idempotent, so you have to assume they will be retried. Faast.js will proactively do retries in some cases where it thinks a function is slow, to reduce tail latency.
If a function fails to execute for transient reasons and exceeds the retry maximum (a config setting you can change), then it will reject the return value promise. You can catch that and handle with another attempt, or report an error, or just ignore it and report less accurate or complete results.
One thing I want to ask is the retries, how do you handle that currently? I ran into multiple cases where functions would fail for transient reasons.