The ubiquitous MPI environment in HPC cluster + Work Stealing Pattern + Distributed Termination Detection = Efficient and Scalable Parallel Solution.

pcircle contains a suite of file system tools that we are developing at OLCF to take advantage of highly scalable parallel file system such as Lustre and GPFS. Early tests show very promising scaling properties. However, it is still in active development, please use it at your own risk. For bug report and feedbacks, please post it here at https://github.com/olcf/pcircle/issues.

pcircle is largely a Python implementation, but we don't rule out possible and judicious integration of C/C++ code for performance gains. Currently, it has the following dependencies:

• Python 2.7.x - it is not Python3 compatible, yet.
• mpi4py - wraps the MPI library
• xattr - wraps the libattr library
• cffi - python interface to libffi
• lru-dict - wrap a small C-based LRU cache

xattr dependency requires cffi, which depends on libffi, which is notoriously difficult to install right.

• On Mac, you might have to manually do:

    brew install pkg-config libffi
    PKG_CONFIG_PATH=/usr/local/opt/libffi/lib/pkgconfig pip install cffi
  

• On Redhat:

    sudo yum install openmpi-devel
    sudo yum install libffi-devel
  

• Then, the rest of setup is pretty much Python standard:

    python setup.py install
  

If you have setuptools and virtualenv packages, then

    make deploy

It grab all dependencies and produce you a container, which isolate you from the default installation:

    source ~/app-pcircle/bin/activate
    fcp -h

For this command to work, you do need a working libffi as well.

• Parallel Data Copy: fcp.8
• Parallel Checksumming: fsum.8
• Parallel Profiler: fprof.8

• Feiyi Wang | Oak Ridge National Laboratory | fwang2@ornl.gov