# Overview
[Deoxys](https://github.com/UT-Austin-RPL/deoxys_control) is a modularized, real-time controller library for Franka Emika Panda to facilitate robot learning research. `Deoxys` is developed by [Yifeng Zhu](https://zhuyifengzju.github.io/) and aims to democratize basic knowledge of robot manipulation to the robot learning community through open-sourcing the controller implementation.
Here are a list of features that we identified as strengths of our
library.
✔ A user-friendly [Python Interface](https://github.com/UT-Austin-RPL/deoxys_control/tree/main/deoxys/deoxys) and [real-time controller implementation](https://github.com/UT-Austin-RPL/deoxys_control/tree/main/deoxys/franka-interface/src/controllers) in C++
✔ Specialized for research on closed-loop [visuomotor skill learning](https://ut-austin-rpl.github.io/deoxys-docs/html/project.html)
✔ Easy [configuration](https://github.com/UT-Austin-RPL/deoxys_control/tree/main/deoxys/config) of robot controllers
✔ Seamless transfer from [robosuite](https://robosuite.ai/) for real-robot control
## Past research
A number of [projects](../project.html) have been boosted by
`Deoxys`. Here is a list of publications based on `Deoxys` so far.
Highlight
VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposals Priors. Yifeng Zhu, Abhishek Joshi, Peter Stone, Yuke Zhu
Learning Generalizable Manipulation Policies with Object-Centric 3D Representations.Yifeng Zhu, Zhenyu Jiang, Peter Stone, Yuke Zhu
MimicPlay: Long-Horizon Imitation Learning by Watching Human Play. Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, Anima Anandkumar
NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities. Ruohan Zhang, Sharon Lee, Minjune Hwang, Ayano Hiranaka, Chen Wang, Wensi Ai, Jin Jie Ryan Tan, Shreya Gupta, Yilun Hao, Gabrael Levine, Ruohan Gao, Anthony Norcia, Li Fei-Fei, Jiajun Wu
2023
Learning Generalizable Manipulation Policies with Object-Centric 3D Representations.Yifeng Zhu, Zhenyu Jiang, Peter Stone, Yuke Zhu
MUTEX: Learning Unified Policies from Multimodal Task Specifications.Rutav Shah, Roberto Martín Martín, Yuke Zhu
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations.Ajay Mandlekar, Soroush Nasiriany*, Bowen Wen*, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, Dieter Fox
Interactive Robot Learning from Verbal Correction.Huihan Liu, Alice Chen, Yuke Zhu, Adith Swaminathan, Andrey Kolobov, Ching-An Cheng
Model-Based Runtime Monitoring with Interactive Imitation Learning.Huihan Liu, Shivin Dass, Roberto Martín-Martín, Yuke Zhu
Doduo: Dense Visual Correspondence from Unsupervised Semantic-Aware Flow.Zhenyu Jiang, Hanwen Jiang, Yuke Zhu
NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities. Ruohan Zhang, Sharon Lee, Minjune Hwang, Ayano Hiranaka, Chen Wang, Wensi Ai, Jin Jie Ryan Tan, Shreya Gupta, Yilun Hao, Gabrael Levine, Ruohan Gao, Anthony Norcia, Li Fei-Fei, Jiajun Wu
MimicPlay: Long-Horizon Imitation Learning by Watching Human Play. Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, Anima Anandkumar
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models. Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Li Fei-Fei
Primitive Skill-based Robot Learning from Human Evaluative Feedback. Ayano Hiranaka, Minjune Hwang, Sharon Lee, Chen Wang, Li Fei-Fei, Jiajun Wu, Ruohan Zhang
Robot Learning on the Job: Human-in-the-Loop Manipulation and Learning During Deployment. Huihan Liu, Soroush Nasiriany, Lance Zhang, Zhiyao Bao, Yuke Zhu (RSS 2023 Best Paper Nominee)
2021-2022
VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposals Priors. Yifeng Zhu, Abhishek Joshi, Peter Stone, Yuke Zhu
Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation. Yifeng Zhu, Peter Stone, Yuke Zhu
Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks. Soroush Nasiriany, Huihan Liu, Yuke Zhu (ICRA 2022 Outstanding Learning Paper)
Learning and Retrieval from Prior Data for Skill-based Imitation Learning. Soroush Nasiriany, Tian Gao, Ajay Mandlekar, Yuke Zhu
Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations. Zhenyu Jiang, Yifeng Zhu, Maxwell Svetlik, Kuan Fang, Yuke Zhu
Ditto: Building Digital Twins of Articulated Objects from Interaction. Zhenyu Jiang, Cheng-Chun Hsu, Yuke Zhu
ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulation. Bokui Shen, Zhenyu Jiang, Christopher Choy, Leonidas J. Guibas, Silvio Savarese, Anima Anandkumar, Yuke Zhu (RSS 2022 Best Paper Nominee)
## Reference
If you find this repo useful for your research, please cite us through
the following work:
```bibtex
@article{zhu2022viola,
title={VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors},
author={Zhu, Yifeng and Joshi, Abhishek and Stone, Peter and Zhu, Yuke},
journal={arXiv preprint arXiv:2210.11339},
doi={10.48550/arXiv.2210.11339},
year={2022}
}
```