Learning Symbolic Representations from Unsupervised Interactions with the Environment
In this project, you are going to implement and build upon our recently proposed symbol learning method, namely DeepSym [1], and make new experiments on new domains to discover the benefits and shortcomings of the method. If you are interested in building autonomous agents, reinforcement learning*, and deep learning, you might like this project. One possible domain is Minecraft due to its open-endedness. Using Malmo [2] or MineRL [3], you will write a piece of software which will collect data and train the network. You will possibly try a lot of variations of the architecture, and see what kind of symbols will emerge! This by itself will be a lot of contribution which we can possibly transform into a conference paper.
In DeepSym, an agent interacts with an environment, collects state-action-effect tuples which will be used as training data for a deep encoder-decoder network with binary units (which will be our symbols). This architecture follows a neurosymbolic approach which is gaining popularity in recent years. The motivation is to use neural networks to process low-level sensorimotor input, and logical inference for high-level planning. The ultimate aim is to build a life-long learning system in which an agent learns increasingly complex abstractions all by interacting with the environment without any reset, as humans do. You can check the related workshop that we organized last summer at one of the top robotics conferences, RSS 2021 [4].
Requirements:
- Basic knowledge of machine learning
- PyTorch or Tensorflow (or maybe Julia if you are very enthusiastic about it)
[1] Ahmetoglu, A., Seker, M.Y., Piater, J., Oztop, E. and Ugur, E., 2020. Deepsym: Deep symbol generation and rule learning from unsupervised continuous robot interaction for planning. arXiv preprint arXiv:2012.02532.
[2] https://www.microsoft.com/en-us/research/project/project-malmo/
[4] https://dnr-rob.github.io/
* Though, this is not reinforcement learning.