(1) Single-pass Parallel SCAN: it solely relies on the CombBLAS framework. In this approach, structural similarity computations are done with a matrix-matrix multiplication.
(2) Two-pass Parallel SCAN: the matrix-matrix multiplication operation provided in the CombBLAS framework consumes a lot of memory space and it easily results in failure due to memory error. To avoid the operation, this two-pass algorithm was developed. Structural similarity computation is moved into the first step and computed values are materialized as edge edges. This allows to evaluate the algorithm with varying parameter settings. Instead of performing matrix-matrix multiplicatiion, this computes similarity values by intersecting matrix rows. The first part of this approach internally uses roaring bitmap library to represent graph partitions in memory.
A Python version of CombBLAS is also available and it is called KDT (Knowledge Discovery Toolbox). This is a Python wrapper that allows data scientists to easily use the features of CombBLAS. The parallel SCAN algorithm was also implemented in Python over KDT.
- OriginalSCAN: a custom implementation of the original SCAN algorithm in Python
- SinglePass: single-pass parallel SCAN algorithm
- SCAN-CombBLAS: C++ implementation over CombBLAS
- SCAN-KDT: Python implementation over KDT
- TwoPass: two-pass parallel SCAN algorithm
- SCAN-CombBLAS: C++ implementation over CombBLAS
- SCAN-KDT: Python implementation over KDT
- MPI (Open MPI, Intel MPI, MPICH)
- CombBLAS 1.6.0
- KDT 0.3