Skip to content

BenJamesbabala/SparseRecurrentNetwork

 
 

Repository files navigation

SparseRecurrentNetwork

SparseRecurrentNetwork is an experimental framework for developing and testing deep, recurrent neural networks for sequence prediction. It relies on Tensor Factorization based optimization:

Unsupervised Feature Tensor Creation

recog1 001

Optimizing by Tensor Factorization

recog2 001

The experimental architecture uses further among others

  • sparse autoencoders
  • a custom cell unit containing feedforward, recurrent and feedback connections with a custom update logic
  • gradient descent with momentum and adaptive gradient updates
  • dropout and inhibition
  • audio and text input preprocessers

It is written in Python using Google's TensorFlow to provide easy, simple developing-testing cycles (parallelization and cluster deployment is currently WIP as well as development in Scala/Spark). It is part of ReCog Technologies

Simplified Architecture Visualizations

Network architecture

alt tag

Forward pass computation

alt tag

Backpropagation

alt tag

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 98.4%
  • HTML 1.6%