SIFToM: Robust Spoken Instruction Following through Theory of Mind
Lance Ying, Xinyi Li, Shivam Aarya, Yizirui Fang, Jason Xinyu Liu, Yifan Yin, Stefanie Tellex, Joshua B. Tenenbaum, Tianmin Shu
arXiv preprint arXiv:2409.10849· 5 citations
Developing embodied AI for domestic robotics
Hey there! I'm Shivam, an undergraduate researcher with a deep passion for robotics and AI.
My research focuses on embodied AI and domestic robotics - specifically, how general-purpose household robots can correctly interpret human commands and reliably complete their tasks. I aim to equip robots with the cognitive ability to build comprehensive world models that are tightly synchronized with the mental models of the humans they assist, understand the physical environments they inhabit, and deeply model the objects they manipulate. Ultimately, I want to enable intuitive cooperation between people and robots so that robots can help everyone live better lives.
To pursue this vision, I began my research journey as a Freshman at Johns Hopkins University in January'24 with Prof. Tianmin Shu's SCAI Lab in collaboration with Prof. Josh Tenenbaum at MIT. I also recently joined the Computational Cognitive Science (CoCoSci) Lab at MIT as a Research Assistant to develop Cog-Gym, a platform to standardize and host hundreds of online human-agent interaction studies. I'm looking forward to continuing my research journey as a PhD student in Fall'26.
Beyond research, I really enjoy building interactive and intuitive systems, which often pulls me into the startup world. Right now, I'm involved with three student-led startups, two of which are being funded by JHU's Spark Accelerator, and the third is supported by the Johns Hopkins Office of Undergraduate Research (HOUR).
If you're into AI, robotics, or just want to chat about cool projects, feel free to reach out!
I believe the future of robotics lies in systems that can seamlessly integrate into our homes, understanding both the physical environment and human intent through multimodal AI.

Lance Ying, Xinyi Li, Shivam Aarya, Yizirui Fang, Jason Xinyu Liu, Yifan Yin, Stefanie Tellex, Joshua B. Tenenbaum, Tianmin Shu
arXiv preprint arXiv:2409.10849· 5 citations
Yifan Yin, Zhengtao Han, Shivam Aarya, Shuhang Xu, Jianxin Wang, Jiawei Peng, Angtian Wang, Alan Yuille, Tianmin Shu
Robotics: Science and Systems (RSS)· 4 citations
Lance Ying, Kunal Jha, Shivam Aarya, Joshua B. Tenenbaum, Antonio Torralba, Tianmin Shu
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)· 19 citations
Developing technology for real-world impact
In development
More details coming soon.
Talks, teaching, and service contributions
Johns Hopkins University
Massachusetts Institute of Technology
Developing Cog-Gym, a platform to standardize and host hundreds of online human–agent interaction studies in Prof. Josh Tenenbaum's research group
Johns Hopkins University