Linux/MacOS: Windows:

cBench is a cross-platform client connected with the open Collective Knowledge portal to help researchers and practitioners reproduce SOTA ML and systems research and share results on live SOTA scoreobards: demo.

cBench is a part of the Collective Knowledge project (CK) and uses portable CK solutions describing how to download, build, benchmark and optimize applications across different hardware, software, models and data sets.

Install prerequisites:

sudo apt update
sudo apt install git wget libz-dev curl cmake
sudo apt install gcc g++ autoconf autogen libtool
sudo apt install libfreetype6-dev
sudo apt install python3.7-dev
sudo apt install -y libsm6 libxext6 libxrender-dev

Install cbrain:

python3 -m pip install cbrain

Initialize the CK solution for MLPerf:

cb init demo-obj-detection-coco-tf-cpu-benchmark-linux-portable-workflows

Participate in crowd-benchmarking:

cb benchmark demo-obj-detection-coco-tf-cpu-benchmark-linux-portable-workflows

See results on a public SOTA dashboard.

You can also use the stable Docker image to participate in crowd-benchmarking:

sudo docker run ctuning/cbench-obj-detection-coco-tf-cpu-benchmark-linux-portable-workflows /bin/bash -c "cb benchmark demo-obj-detection-coco-tf-cpu-benchmark-linux-portable-workflows"

You can also check all dependencies for this solution.

• Online docs for the Collective Knowledge technology

• This is an ongoing project - don't hesitate to contact us if you have any feedback and suggestions!

We would like to thank all CK partners for fruitful discussions and feedback!

Copyright 2020 cTuning foundation