Creating the world's first autonomous micro-robot!
The goal of this project is to investigate different Reinforcement Learning (RL) architectures and techniques against scenarios that have relevance to the field of micro robotics.
This project is specifically concerned with the control of magnetic micro and nano-robots. These robots consist of magnetic material that are controlled by surrounding the workspace with energized coils of wire to generate a magnetic field that acts on the robot. This project makes the simplification of assuming that the robot only moves in 2 dimensions.
A magnetic micro-robot must sort particles to the left and right sides of the workspace depending on each particle's class.
- Solved with DDPG and Experience Replay.
In progress.
Double Duelling DQN [1] features two estimators: the action advantage function and the satte value function.
Deep Deterministic Policy Gradient (DDPG) [2] uses a model-free, actor-critic algorithm that can successfully learn control policies operating over a continuous action space.
Experience Replay stores past agent experiences and randomly samples from them to perform network updates. First introduced in [3].
In progress. (improving efficiency)
In progress.
[1] Wang, Ziyu, et al. "Dueling network architectures for deep reinforcement learning." arXiv preprint arXiv:1511.06581 (2015).
[2] Lillicrap, Timothy P., et al. "Continuous control with deep reinforcement learning." arXiv preprint arXiv:1509.02971 (2015).
[3] Lin, Long-Ji. Reinforcement learning for robots using neural networks. No. CMU-CS-93-103. Carnegie-Mellon Univ Pittsburgh PA School of Computer Science, 1993.
[4] Diller, Eric, and Metin Sitti. "Micro-scale mobile robotics." Foundations and Trends® in Robotics 2.3 (2013): 143-259.
http://cs.stanford.edu/people/karpathy/reinforcejs/waterworld.html
http://pemami4911.github.io/blog/2016/08/21/ddpg-rl.html
https://jaromiru.com/2016/10/03/lets-make-a-dqn-implementation/