Echo State Networks are easy to train recurrent neural networks - In some sense, they show how far you can get doing nothing but good weight initialisation. Here is a relatively simple, self-contained and readable implementation of such networks. It exists because I found the Matlab code in the linked article to be none of these things.
The code is released under the WTFPL.