Qualification Videos

RoboCup@Home 2017 SSPL

Human Body Orientation Estimation using Convolutional Neural Network

Pororobot: A deep learning robot that plays video Q&A games

Schedulebot: A home robot learning and acting schedule adaptively via dynamic environments

Team AUPAIR : Face Detection and Recognition

Multimodal Residual Learning for Visual QA

Bayesian Deep Learning for Incremental Teaching a Home Robot

Childbot: A conversational assistant for child care

Multi-focus Attention Network for Efficient Deep Reinforcement Learning

Selected Publications

• Human Body Orientation Estimation using Convolutional Neural Network(link)
  • Jinyoung Choi, Beom-Jin Lee,and Byoung-Tak Zhang
  • published in IROS 2016 Assistance and Service Robotics in a Human Environment Workshop
  • Abstract
• We propose an active viewpoint selecting method based on human body orientation with consideration of the robot's given environment. This makes the robot position itself in front of the person to observe one's behavior
• Multimodal Residual Learning for Visual QA(link)
  • Jin-Hwa Kim, Sang-Woo Lee, Dong-Hyun Kwak, Min-Oh Heo, Jeonghee Kim, Jung-Woo Ha, Byoung-Tak Zhang
  • published in NIPS 2016
  • Abstract
• We propose Cambot which can be instantiated in any platform including robots, desktops and tablet PCs, which have a camera and microphone, engaging natural environmental situations of visual conversation for human interactions.
• Pororobot: A deep learning robot that plays video Q&A games(link)
  • Kyung-Min Kim, Changjun Nan, Jung-Woo Ha, Yujung Heo, and Byoung-Tak Zhang
  • published in AAAI 2015 Fall Symposium on AI for Human-Robot Interaction (AI-HRI 2015), pp. 89-93, 2015.
  • Abstract
• We proposed a memory model called the deep concept hierarchy (DCH) model that enables the progressive abstraction of concept knowledge in multiple levels. Moreover, by employing the model to a mobile robot, it can learn the cartoon video with the child and teach the child what has happened in the video
• Childbot: A conversational assistant for child care(link)
  • Hwiyeol Jo, Woo-Young Kang, Dong-Sig Han, Byoung-Tak Zhang
  • published in International Symposium on Perception, Action, and Cognitive Systems: Beyond AlphaGo (PACS 2016), pp.91-92, 2016.
  • Abstract
• We propose a RNN Encoder-Decoder (seq2seq) model[Sequence to sequence] for Korean conversational robot (Konvbot). We collected about 10,000 dialogue pairs for this scenario from more than 30 people. With the data, we implemented the base conversational model, which is used for collecting more dialogues in real environments.
• Schedulebot: A home robot learning and acting schedule adaptively via dynamic environments(link)
  • Chung-Yeon Lee, Chaeeun Lee, and Byoung-Tak Zhang
  • published in International Symposium on Perception, Action, and Cognitive Systems: Beyond AlphaGo (PACS 2016), pp.69-70, 2016.
  • Abstract
• A Robot needs to understand the environments and ongoing events to operate safely and efficiently in complex real situations such as private homes. To illustrate some of these problems, we built a pseudo-real home environment that family members and a robot can interact with each other based on the morning at home scenario. Predicted events, activities, and robot actions following the results are shown in the video.
• Bayesian Deep Learning for Incremental Teaching a Home Robot(link)
  • Jiseob Kim, Seongho Son,and Byoung-Tak Zhang
  • submitted in AAAI2017
  • Abstract
• Humans are very good at learning new objects. If some objects are unusual or confusing, they ask other people and learn from the answer. We investigate how this scenario can be performed by robots. Specifically, we develop a robot that adapts to dynamically changing home environments for which it actively asks human and learns online. In our setting the robot adapts to the new environment by actively asking for a confusing object and updating its recognition model from the answer in an online manner.
• Multi-focus Attention Network for Efficient Deep Reinforcement Learning(link)
  • Jinyoung Choi, Beom-Jin Lee,and Byoung-Tak Zhang
  • submitted in AAAI2017
  • Abstract
• We propose the Multi-focus Attention Network (MANet) which mimics human ability to spatially abstract the low-level sensory input into multiple entities and attend to them simultaneously. Our model showed significantly improved performances in grid world navigatoin problem and multi-agent combat problem.

Related Works

International Journal Papers

• Whole-body balancing walk controller for position controlled humanoid robots, S.-J. Yi, B.-T. Zhang, D. W. Hong, and D. D. Lee, International Journal of Humanoid Robotics, 13(1):1550047, 2016.(link)
• Consensus analysis and modeling of visual aesthetic perception, T.-S. Park and B.-T. Zhang, IEEE Transactions on Affective Computing, 6(3):272-285, 2015.(link)
• (link)
• Team THOR's entry in the DARPA robotics challenge trials 2013, S.-J. Yi, S.G. McGill, L. Vadakedathu, Q. He, I. Ha, J. Han, H. Song, M. Rouleau, B.-T. Zhang, D. Hong, M. Yim, and D.D. Lee, Journal of Field Robotics, 32(3):315-335, 2015.(link)
• Bayesian evolutionary hypergraph learning for predicting cancer clinical outcomes, S.-J Kim, J.-W. Ha, and B.-T. Zhang, Journal of Biomedical Informatics, 49:101-111, 2014.(link)

International Conference Papers

• DeepSchema: Automatic schema acquisition from wearable sensor data in restaurant situations, E.-S. Kim, K.-W. On, and B.-T. Zhang, International Joint Conference on Artificial Intelligence (IJCAI 2016), pp. 834-840, 2016.(link)
• Dual-memory deep learning architectures for lifelong learning of everyday human behaviors, S.-W. Lee, C.-Y. Lee, D. H. Kwak, J. Kim, J. Kim, and B.-T. Zhang, International Joint Conference on Artificial Intelligence (IJCAI 2016), pp. 1669-1675, 2016.(link)
• Storybot: Story learning from cartoon videos via consecutive event embedding, M.-O. Heo, and B.-T. Zhang, 2016 International Symposium on Perception, Action, and Cognitive Systems: Beyond AlphaGo (PACS 2016), pp.87-88, 2016. (link)
• Cafebot - A conversational cashier robot in cafes, C. Han, K.-W. On, E.-S. Kim, and B.-T. Zhang, 2016 International Symposium on Perception, Action, and Cognitive Systems: Beyond AlphaGo (PACS 2016), pp.83-84, 2016. (link)
• Knowledgebot: Neuroknowledge based complimentary learning model for question answering systems, K.-W. On, E.-S. Kim, and B.-T. Zhang, 2016 International Symposium on Perception, Action, and Cognitive Systems: Beyond AlphaGo (PACS 2016), pp.81-82, 2016.(link)
• Pandabot: Multimodal story learning with dynamic memory construction, Y.-J. Heo, E.-S. Kim,K.-W. On, and B.-T. Zhang, 2016 International Symposium on Perception, Action, and Cognitive Systems: Beyond AlphaGo (PACS 2016), pp.79-80, 2016.(link)
• Jibobot: A personal assistant robot with social motions, E.-S. Kim, J. Kim, D.-H. Kwak,K.-W. On, and B.-T. Zhang, 2016 International Symposium on Perception, Action, and Cognitive Systems: Beyond AlphaGo (PACS 2016), pp.77-78, 2016. (link)
• Glassbot: Personalized wearable agents learning from everyday human behaviors, S.-W. Lee, C.-Y. Lee, D.-H. Kwak, and B.-T. Zhang, 2016 International Symposium on Perception, Action, and Cognitive Systems: Beyond AlphaGo (PACS 2016), pp.75-76, 2016.(link)
• Aupair: A home robot for personal care, B.-J. Lee and B.-T. Zhang, 2016 International Symposium on Perception, Action, and Cognitive Systems: Beyond AlphaGo (PACS 2016), pp.67-68, 2016. (link)
• Modeling situated conversations for a child-care robot using wearable devices, K.-W. On, E.-S. Kim, and B.-T. Zhang, AAAI 2015 Fall Symposium on AI for Human-Robot Interaction (AI-HRI 2015), pp. 103-106, 2015. (link)
• Behavioral pattern modeling of human-human interaction for teaching restaurant service robots, E.-S. Kim, K.-O. On, and B.-T. Zhang, AAAI 2015 Fall Symposium on AI for Human-Robot Interaction (AI-HRI 2015), 2015.(link)
• Schedule Management System for Child-Care Home Robot, D.-H. Kwak and B.-T. Zhang, AAAI 2015 Fall Symposium on AI for Human-Robot Interaction (AI-HRI 2015), 2015. (link)
• Non-parametric Bayesian sum-product networks, S.-W. Lee, C. J. Watkins, B.-T. Zhang, ICML Workshop on Learning Tractable Probabilistic Models, 2014.(link)
• Predictive property of hidden representations in recurrent neural network language models, S. Yoon, S.-W. Lee, and B.-T. Zhang, 2014 NIPS workshop on Modern Machine Learning Methods and Natural Language Processing, 2014. (link)
• Online learning of low dimensional strategies for high-level push recovery in bipedal humanoid robots, S.-J. Yi, B.-T. Zhang, D. Hong, and D. D. Lee, In Proceedings of IEEE International Conference on Robotics and Automation (ICRA 2013), pp. 1649-1655, 2013.(link)
• Estimating multiple evoked emotions from videos, W.H. Choe, H.-S. Chun, J. Noh, S.-D. Lee,and B.-T. Zhang, In Proceedings of Annual Meeting of the Cognitive Science Society (CogSci 2013), pp. 2046-2051, 2013.(link)
• Enhancing human action recognition through spatio-temporal feature learning and semantic rules, K. Ramirez-Amaro, E.-S. Kim, J. Kim, B.-T. Zhang, M. Beetz and G. Cheng, In Proceedings of 2013 IEEE-RAS International Conference on Humanoid Robots (Humanoids 2013), 456-461, 2013.(link)
• Data driven SLAM for a mobile robot with short ranged navigation sensors, J. Lee, S.-H. Ji, and B.-T. Zhang, Proceedings of the 17th International Conference on Mechatronics Technology (ICMT 2013), pp. 377-379, 2013.(link)
• Active stabilization of a humanoid robot for impact motions with unknown reaction forces, S.-J. Yi, B.-T. Zhang, D. Hong, and D. D. Lee, In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2012), pp. 4034-4039, 2012. (link)
• Sparse population code models of word learning in concept drift, B.-T. Zhang, J.-W. Ha, and M. Kang, In Proceedings of Annual Meeting of the Cognitive Science Society (CogSci 2012), pp. 1221-1226, 2012.(link)

Books & Magazines

• B.-T. Zhang and Y.-Y. Choi, Communication: Human, animals, and artificial Intelligence (Korean translation of "Menschen, Tiere und Max: Natuerliche Kommunikation und kuenstliche Intelligenz" by I. Wachsmuth, Springer, 2012), Seoul National University Press, 2014.
• Humans and machines in the evolution of AI in Korea, B.-T. Zhang, AI Magazine, 37(2):108-112, 2016.(link)
• Deep learning-based video analysis techniques, J. Kim, C.-J. Nan, and B.-T. Zhang, Communications of the Korean Institute of Information Scientists and Engineers, 33(9):21-31, 2015. (link)
• Deep hypernetwork models, B.-T. Zhang, Communications of the Korean Institute of Information Scientists and Engineers, 33(8):11-24, 2015.(link)

Korean Journal Papers

• Event cognition-based daily activity prediction using wearable sensors, C.-Y. Lee, D. H. Kwak, B.-J. Lee,and B.-T. Zhang, Journal of KIISE, 43(7):781-785, 2016.(link)
• Social network analysis of TV drama via location knowledge-learned deep hypernetworks, C.-J. Nan, K.-M. Kim, and B.-T. Zhang, KIISE Transactions on Computing Practices, 22(11):619-624, 2016.(link)
• Character-based subtitle generation by learning of multimodal concept hierarchy from cartoon videos, K.M. Kim, J.-W. Ha, B.-J. Lee,and B.-T. Zhang, Journal of KIISE, 42(4):451-458, 2015.(link)
• Smartphone-user interactive based self developing place-time-activity coupled prediction method for daily routine planning system, B.-J. Lee, J. Kim, J.-H. Ryu, M.-O. Heo, J.-S. Kim, and B.-T. Zhang, KIISE Transactions on Computing Practices, 21(2):154-159, 2015.(link)

Korean Conference Papers

• Human activity as a sequence to sequence representation, Patrick M. Emaase, Beom-Jin Lee, Byoung-Tak Zhang, 2016 한국컴퓨터종합학술대회(KCC2016)논문집, pp. 868-870, 2016.06.
• 베이지안 신경망을 활용하는 온라인 이미지 인식을 통한 순차 추정, 손성호, 김지섭, 장병탁, 2016 한국컴퓨터종합학술대회(KCC2016)논문집, pp. 1033-1035, 2016.06.
• ActMap: 가정로봇의 사용자 일상생활 활동기록을 통한 고차 인지 특징점 발견 방법, 이범진, 장병탁, In 2016 한국인지과학학회 연차학술대회, 2016.05.
• Deep convolutional neural network을 이용한 2D 영상에서의 사람 자세, 행동 및 위치 통합 인식 시스템, 김지섭, 김은솔, 윤상웅, 정문식, 최현수, 장병탁, 2015 한국컴퓨터종합학술대회(KCC2015)논문집, pp. 846-848, 2015.06..