Echo State Networks are easy to train recurrent neural networks - In some sense, they show how far you can get doing nothing but good weight initialisation. Here is a relatively simple, self-contained and readable implementation of such networks. It exists because I found the Matlab code in the linked article to be none of these things.

• learning to be a tunable frequency generator
• learning a Mackey-Glass system