• Description
• Installation
• Usage
• Improvements
• Support
• License

Adaptation and extension of . The present code delivers neural networks optimized according to performance error, working memory (~RAM), model size (~Flash) and latency.

The configuration space (or search space) of the network, that is, the components that build it up are elligible by the user: you can build up networks with CNNs, RNNs or whatever structure you like. However, you will have to build the network builder: the function that will specify the concrete network from a search space point.

Build with PyTorch, GPyTorch and Ax.

The result obtained after running a SpArSeMoD process is the group of best networks (Pareto frontier networks):

1. Clone this repository
2. Install conda environment: conda env create -n sparse -f sparse.yml
   1. Alternatively, install packages directly from pip
3. run pip install .

The folder examples/ contains several separate functioning examples. Each folder is a different example and contains all the examples to run a sessions of SparseMod.

To adapt to your specific problem, try to imitate one of those examples. The components needed are:

1. Create a Space Search function which returns a SearchSpace object, as in /examples/cnn_cifar_example/cnn.py.
2. Create a network builder, as in examples/cnn_cifar_example/cnn.py, which needs to accept the following parameters: parametrization, classes=10, input_shape=None
3. Build you dataset and data pipeline as in examples/cnn_cifar_example/load_data.py.
4. Build the main call function as in examples/cnn_cifar_example/main.py.
5. Complete config.cfg, with all the necessary parameters .
6. Run python main.py
7. Wait until the process completes. You will have to check the results to see which has been the best network.

The process described is for a CNN based network and the CIFAR 10 Binary.

For a more and complete guide for SpArSeMoD usage, you can checkout the Documentation

1. The models will be placed in the folder specified in the configuration as root, and there, a models folder will be created, where all the models will be stored. Only pareto models are saved.
2. Results for the evaluations will be placed in the specified folder in the configuration. It consists of a csv where results are saved as dataframe, a json for the experiment and a txt for the time taken for the whole experiment.
3. In the file examples/result_inspection.ipynb, an example of analysing the results is exemplified.
4. In case yo want to recover the exact structure of your chosen netwrok, you will ahve to rerun SpArse to reload the experiment and inspect the network. An example of such process is found in examples/cnn_cost_example/read_experiment.py

• Add tests for

  • Model building, training and testing
    • Solve problem of image being smaller than needed
  • Quantization and pruning
  • Sobol stage
  • Gaussian process based stage
  • Morphisms

• Complete documentation for

  • Tutorial
  • General description and concepts
  • Load it in Readthe Docs
  • install documentation

• Include tests in

  • Travis CI
  • Makefile

• Complete download_datafor tutorial

• REad the docs problems

• Travis Hooks

The next points represent current WIP points considered as weak or defective points of the framework.

• Check batch size for GP works
• Add notebook for selecting best network and inspecting results
• When loading the data it should be loaded only up to an index, since we dont want to reload morphed solutions results

This code has a main point of smell:

• The json serializer saves almost all that is inside. Experiments end up weighing a lot, like MB-> GB. It woul be needed to extract all the structures outside the experiment to avoid this problem. This would be the worst problem in the code.

Necessary improvements for the framework:

1. Add raytune distribution
2. Add tolerance checker as stopper

If you use SpArSeMod in your work, please cite us as follows:

@inproceedings{Borrego-Carazo2020,
address = {Miami, Florida},
author = {Borrego-Carazo, Juan and Castells-Rufas, David and Carrabina, Jordi and Biempica, Ernesto},
booktitle = {19TH IEEE INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND APPLICATIONS (ICMLA)},
publisher = {IEEE},
title = {{Extending SpArSe: Automatic Gesture Recognition Architectures for Embedded Devices}},
year = {2020}
}

If you are having issues, please let us know. We have a mailing list located at: juan.borrego@uab.cat

The project is licensed under the GNU GLP 3.0 License

This code has been developed by a joint collaboration of