New paper accepted: ISMAR 2025

Human-Centric Software Engineering

News:

Humans are a key part of software development, including customers, designers, coders, testers, and end-users. While most current software engineering research and practices are function, data, or process-oriented, human-centric software engineering focuses on the human factors in engineering software systems. At present, there are major issues with misaligned software applications related to human factors, such as accessibility, usability, emotions, personality, age, gender, and culture. We aim to investigate enhanced theory, models, tools, and capability for next-generation human-centric software engineering aiming to achieve significant benefits of greatly improved software quality and user experience, developer productivity, and cost savings. Major research interests include but not limited to:
- Human-centric requirements engineering
- Incorporating human factors into requirements and design e.g., emotions, bias, personality, and culture
- Context-awareness in human-centric software (and systems) engineering
- Impact of human factors on development processes and software teams
- Tools and models for capturing and interpreting user behaviours
- Software applications that demonstrate the practice of human-centric software engineering
- Education of human factors in software engineering for IT students

Please visit Prof. John Grundy's page on human-centric software engineering to get a more general view at: https://sites.google.com/site/johncgrundy/research/human-centric-software-engineering

Currently, one of our on-going projects is "A Virtual Reality Platform to Assist Human-Centric Requirements Engineering". We invented a VR based requirements engineering platform (VR4HcRE) which can assist requirement engineers to create immersive experiences to capture special user requirements during the requirement workshop. The early version of the system can be found at here: https://github.com/Ben5460/ACMVRHCSE.

The platform has some key features such as:

Automated creation of 3D avatars based on user personas
Synchroised movement through camera (full body movement)
Simulate different environments such as indoor, outdoor, light and weather
Simulate special user conditions such as colour blindness
Voice control and voice to text (for automated generation of meeting minutes)
Supports online collaboration of multiple users (requirement engineers and clients)

VR4HcRE Platform

This is an on-going project with more features to be added to support both software engineers and users to take advantage of the immersive envoironment to eliciate, design and validate software requirements

Edge4Real

An edge computing based platform to capture real-time user movement and synchronise the movement of the Avatar

The VR4HcRE platform: showing two participants with remote multiplayer based collaboration (A) multiplayer communication through audio transcription, (B) simulation of human/environmental factors panel, (C) keywords capture from audio transcription and (D) virtual mobile phone prototype.

We are also interested in investigating the following topics:

VR for Software Engineering. Investigate the use of immersive technologies (e.g. VR, AR, XR, and here we use VR for simplicity) and develop supporting tools for solving human factor relevant issues in software engineering. Currently, we focus on requirements engineering especially requirements elicitation. We have developed a prototype VR system which can simulate different environments (indoor/outdoor, weather, light) and user conditions (e.g. colour blindness), and use 3D avatar to enable the interaction between the user and the virtual space, and the software engineer can embody the user perspective via the first-person view (HCSE-CS 2021).
Software Engineering for VR. With the increasing popularity of VR software especially with the push of Metaverse, the demand for developing VR software is rapidly increasing and will soon over the supply of qualified software developers with required skillset on the market. For developing VR software, the developers not only need to be familiar with 3D Game development engine such as Unity, but also various human factors naturally embedded in VR software. Does the current software engineering models and methods suit the need of VR software? What are the technical and non-technical challenges for software engineers in developing VR software? The aim is to find out the research gaps and propose new models and methods to improve the efficiency and quality in developing VR software.
Edge computing and machine learning based methods for supporting human-centric software engineering. For example, the detection of human behaviour and emotions throught multi-source data. We have already developed an Edge Computing based Human Behaviour Recognition System (Edge4Real at ASE2020), an Edge Computing based Multi-source Emotion Recognition Platform (Edge4Emotion at CCGrid 2021). There methods and tools will be built into our VR for software enginering platform.

Papers:

O. Wang, X. Liu, B. Cheng, C. Arora, T. Hoang, VR4HcRE: Virtual Reality Platform for Human-centric Requirements Elicitation, 21st IEEE International Symposium on Mixed and Augmented Reality (ISMAR'22, Posters Track), accepted on 16 August 2022. (CORE A*)
D. Shao, X. Liu, B. Cheng, O. Wang, T. Hoang, Edge4Real: A Cost-Effective Edge Computing based Human Behaviour Recognition System for Human-Centric Software Engineering, 35th IEEE/ACM International Conference on Automated Software Engineering (ASE20 Demo Track), accepted on 04 July 2020. (CORE A*) Video URL: https://youtu.be/dH0oWnWk924
O. Wang, B. Cheng, T. Hoang, C. Arora, X. Liu, Virtual Reality Enabled Human-Centric Requirements Engineering, 2nd International Workshop on Human-Centric Software Engineering and Cyber Security, colocated with 36th IEEE/ACM International Conference on Automated Software Engineering, Melbourne, Australia, 2021.
B. Cheng, O. Wang, D. Shao, C. Arora, T. Hoang and X. Liu, Edge4Emotion: An Edge Computing based Multi-source Emotion Recognition Platform for Human-Centric Software Engineering, 21st IEEE/ACM international Symposium on Cluster, Cloud and Internet Computing (CCGrid 2021 Poster Track), accepted on 27/02/2021. (CORE A) https://youtu.be/TqLjEDbzN90
B. Cheng, C. Arora, X. Liu, T. Hoang, Y. Wang and J. Grundy, Multi-Modal Emotion Recognition for Enhanced Requirements Engineering: A Novel Approach, 2023 IEEE International Requirements Engineering Conference (RE’23 Next), accepted on 31 May 2023. (CORE A)

We have also organised the following venues for people who are interested in the similar areas:

International Workshop on Human-Centric Software Engineering and Cyber Security. https://research.csiro.au/distributed-systems-security/hcse-cs-2021/
JSS Special Issue on Human-Centric Software Engineering – Approaches, Technologies, and Applications: https://www.journals.elsevier.com/journal-of-systems-and-software/call-for-papers/special-issue-on-human-centric-software-engineering-approaches-technologies-and-applications

Given the cross-discipline nature of Human-Centric Software Engineering, we are actively looking for collaborators from both research and industry communities. If you are interested in our work or would like to discuss potentional collaboration opportunities, please feel free to conact us.

Google Sites

Report abuse