BNDL is a library for map-reduce based distributed processing akin to Apache Spark but is implemented in python (with a bit of cython).

Master branch build status:

BNDL can be installed through pip:

pip install bndl

The main commands to use BNDL are bndl-compute-shell to open an interactive shell hooked up to BNDL workers and bndl-compute-workers to start workers seperately (e.g. throughout the cluster).

Obtain a compute context in python scripts by importing ctx from bndl.compute.run:

from bndl.compute import ctx
print(ctx.range(1000).map(str).map(len).stats())