Powderkeg: teaching Clojure to Spark

While there’s no shortage of libraries that let one interact with Spark using Clojure they treat the problem as providing a more fluid experience of using Java interfaces. In doing so they overlook the REPL experience (by relying on AOT) and create code duplication (having their own fn, map, reduce, filter, mapcat…). But there is another way: ignore most of the Java/Scala interfaces and consider Spark artifacts (RDDs and streams) as a transducing context. Through doing this we have achieved a true REPL experience, a small API and overall made interactions with Spark more Clojuresque.
Length: 35:26
Views 721 Likes: 14
Recorded on 2016-12-01 at Clojure/ConJ
Look for other videos at Clojure/ConJ.
Tweet this video