Which is obviously very similar to how transducers already work in Clojure, but they still lack some of concurrency options of reducers. Getting all this on a smart, distributed runtime seems very promising.
There is a library called tesser [1] (by Jepsen/Riemann author) that behaves like parallel transducers with more Clojure "native" syntax. With transducers, you have to use "comp," and with tesser, you use "->>" as you'd use with lazy functions.
Sadly, tesser is not advertised as it should; I find it much more flexible than transducers. E.g. you could parallelize tesser code over Spark/Hadoop cluster.
As I understand it Tesser doesn't support ordering which makes it tricky for many types of logic. I'd like to be able to control exactly which parts of a pipeline are parallelisable without too much ceremony, but I can't say I've used Rama in anger or that it makes this ergonomic.
> Getting all this on a smart, distributed runtime seems very promising.
Hopefully it is.
This CPS article is the first of the Rama blog posts where it seemed like there might be something there. The earlier posts - "I built Twitter scale Twitter in 10Kloc" - were never really all that convincing. The thing they claimed to have built was too ambitious a claim.
Oh I think there’s a lot of good stuff baked in there. The big idea downstream is that you have incrementally calculated, indexed data structures to query all the results of this fancy CPS logic. It’s all slightly esoteric even coming from a Clojure background but it ticks every box I want from a modern data platform, short of speaking SQL.
