QUEX Workshop:
Creation with GenAI
Wednesday, 29 May (UQ, Brisbane & Exeter, UK)

The "Creation with GenAI" workshop, jointly hosted by the University of Queensland (UQ) and the University of Exeter,  aims to explore methods for scaffolding the process of using GenAI (i.e.OpenAI  ChatGPT and DALLE 3, Microsoft CoPilot, Google Gemini, Anthropic Claude, Stable Diffusion and Midjourney), enabling individuals to create artifacts in both familiar and unfamiliar domains.

The workshop seeks to foster collaboration among Human-Centred Computing researchers, GenAI algorithm researchers, and current GenAI practitioners. The workshop is funded by the QUEX Institute Workshop Grant Scheme.

The workshop will take place on Wednesday, 29 May, starting at 2 pm AEST at the UQ St Lucia campus and 9am BST at the Exeter location. It will feature a joint location hybrid panel before transitioning to the Exeter venue. Both locations will host keynote speakers and the event will also be livestreamed. The agenda is structured such that half the events and schedule will come from UQ in the afternoon, transitioning smoothly to Exeter in their morning, thereby fostering a cohesive cross-continental collaboration. A distinctive element of the workshop is a dedicated crossover time, to facilitate a hybrid collaborative discussion among the participants from both universities.

Date: Wednesday, 29 May  2024, 2pm AEST and 9am BST
UQ St Lucia Location:  46-371 - Andrew N. Liveris Building, Collaborative Room
Exeter Location: SWIoT space, Innovation Centre, Streatham Campus

Online: The workshop will be livestreamed from both the Brisbane and UK locations

Registration has now closed - Wed 29 May 12:15pm.

Schedule

Abstracts

• UQ Keynote: Prompted: our changing relationship with artificial intelligence - Prof. Marek Kowalkiewicz
  GenAI tools are reshaping the landscape of creative processes. In his talk, Prof. Marek Kowalkiewicz will delve into the evolving relationship between humans and AI, exploring the complex interplay of prompts, agency, and creativity. While we often perceive ourselves as the ones directing AI through prompts, the reality is more nuanced. The very existence of these tools, with their pre-trained models and inherent biases, prompts us in turn, thus influencing our creative starting points and shaping the trajectory of our ideas. Drawing upon insights from "The Economy of Algorithms: AI and the Rise of the Digital Minions," this talk will examine the shifting power dynamics between humans and algorithms. As AI tools gain increasing agency, blurring the lines between tool and collaborator, we must re-evaluate our role in the creative process. Are we truly the masters of our creations, or are we becoming the 'prompted,' guided by the subtle influence of our digital counterparts?

• Exeter Keynote: Practical AI Tools for Music Makers, Not Content Creators - Tom

• Midjourney: Collaborating with AI for Visual Storytelling, Not Replacing Traditional Art - Felipe Sarria
  In this presentation, I will share my personal journey of creating an illustrated storybook using the AI image generator, Midjourney. As a storyteller with a background in 3D modeling, I lacked confidence in my drawing abilities, which hindered my ability to visualize characters and worlds effectively. The discovery of Midjourney in September 2022 opened up new possibilities for me to express my story visually.

Through this experience, I have gained a deeper understanding of the potential and limitations of AI image generators in the artistic process. While these tools can enhance creativity, streamline communication, and help validate concepts, they should not be seen as a replacement for traditional art forms. Midjourney has served as a collaborator in my storytelling journey, enabling me to create evocative visuals that immerse readers in the worlds I craft. However, it is crucial to recognize that the human touch, the intention, and the soul poured into artwork cannot be replicated by machines.

In this presentation, I will discuss the evolution of Midjourney, its impact on my illustrated storybook project, and the insights I have gained regarding the relationship between AI and traditional art. I will emphasize the importance of embracing AI image generators as collaborators that ignite imagination and empower storytellers, while acknowledging the irreplaceable value of human artistry. Ultimately, I aim to showcase how AI can enhance the creative process without diminishing the significance of traditional art forms.

• Synthetic Monuments and Alternative Futures: Using GenAI in Creative and Speculative Ways - Dr Luke Munn
  This presentation explores GenAI, focusing on how it offers new questions and approaches to research in HASS disciplines. It first showcases work on heritage. We take UNESCO heritage sites and convert official photographs to machinic descriptions, then use these as prompts to generate synthetic sites. Analysing outputs by Stable Diffusion and MidJourney provide insights into how these sites are constructed and maintained as "heritage." It next turns to ongoing work on economic futures. Drawing from Chile's famous early cybernetics experiment, we develop synthetic personas of Allende and Beer, the prime minister and chief architect of the project, using both fine-tuning and RAG-based approaches. Interviewing these "subjects" begins to open up alternative approaches to economic and social organisation, driven by different values and aims. 

• Building Software with GenAI - Dr Aneesha Bakharia:
  As we navigate through a transformative era in artificial intelligence, Large Language Models (LLM) are redefining how software is designed and created. This presentation is designed for individuals without a background in computer science and focuses on the practical use of generative AI technologies to create software. The presentation will focus on prompt engineering - the skill of crafting questions and commands that guide AI to produce desired outcomes. We will demystify this process, making it accessible by using everyday language and relatable examples. Additionally, we will explore domain-specific language, explaining how to customise AI interactions to fit specific needs in various fields, from business to art. This approach empowers those without programming skills to effectively utilise AI tools to build software solutions. The session will also provide insights into the current capabilities of LLM models like OpenAI ChatGPT, Google Gemini and Microsoft Bing Chat, emphasizing how these tools can be leveraged today. We aim to equip participants with the knowledge to harness the power of generative AI, transforming their ideas into functional software applications through simple interactions. This talk will encourage a shift in perspective, showcasing AI not just as a tool for the future but as an accessible technology that is reshaping the present.

• Creativity Gone Wrong: The Misuse of Generative Content Online - Amraj Singh
  The ease and accuracy with which generative technologies have enabled abstract creative content to be produced has exponentially increased over the last several years, as available tools have grown in both variety and complexity. In this talk, we look at the recent history of Generated content online, and how it's existence has become a force multiplier for generating such creative content. Then, we discuss how generative tools have been co-opted to cause real world harm, as well as the potential ramifications of behaviour around this technology, and what could be done to mitigate it. 

• Revolutionizing Education: Empowering the Future with GenAI’s PhysicsTeacherBot - Dr Mustapha Bukar Gana
  This article discusses the development of an innovative virtual teaching assistant called the PhysicsTeacherBot, which aims to revolutionize physics education through artificial intelligence. The primary goal of the PhysicsTeacherBot is to provide students with an interactive and personalized learning experience in physics. By leveraging natural language processing and AI algorithms, the bot tailors explanations to individual learners of varying proficiency levels. It intends to make physics more accessible and engaging by simplifying complex concepts using step-by-step breakdowns, examples, and visual demonstrations. The article examines the process behind designing the PhysicsTeacherBot, outlining how it integrates extensive physics knowledge with capabilities like concept decomposition, adaptive instruction based on a user's needs and style, and personalized feedback. Key features that facilitate interactive learning opportunities and customizable support are also detailed. Insights are provided on how the bot's pedagogical framework employs techniques such as scaffolding and analogies to optimize subject comprehension. User experiences and case studies showcase the PhysicsTeacherBot's potential to positively impact learning outcomes and student engagement in physics. By offering individualized learning pathways and tracking a user's progress, it aims to nurture confidence and curiosity in physics mastery. The implementation of such AI-driven virtual teaching assistants is positioned as integral for revolutionizing education and motivating future scientists. Overall implications involve the technology's adaption to other disciplines through personalized virtual guidance.

• Agents That Think and Feel: Presenting the GPTNPC Architecture for LLM-driven NPCs to Mimic Human-Likeness and Autonomy in a Video Game Context - Dr David Ogunlesi
  Non-player Characters (NPCs) in video games has been ubiquitous for many years. However, the interactions players have with these NPCs have typically been limited to pre-defined dialogue options, limiting theplayer’s immersion within the game world. Recent advances AI and NLP have opened up new possibilities for creating NPCs that are capable of engaging in more naturalistic conversational interactions with players. We explore the use of GPT-powered NPCs in video games in the context of virtual reality and developing NPCs that possess long-term memory, autonomous behaviour, and believable interactions. We propose a novel GPTNPC framework that utilises GPT models, in a modularised approach, to generate coherent and engaging dialogue for NPCs, while also incorporating techniques for thought and emotion modelling to create more immersive characters. To assess the effectiveness of our framework, human trials and a turning test were performed. We demonstrate a mixed level of general believability, showing a rating score of 60%. From this we further discuss ways to better improve and utilise the framework. Ultimately, our project aims to contribute to the growing field of AI-powered game development and create more engaging gaming experiences for players.

• Review of empirical studies that employ generative AI - Dr Avon Huxor
  This presentation will review the literature in which the application of generative AI - for creation purposes - has been empirically studied. These will point to what approaches may, or may not, be valuable in future applications of the technology.

Workshop Organisers:

• Aneesha Bakharia (UQ)

• Avon Huxor (Exeter)

• Xiaoyang Wang (Exeter)

• Guoqiang Zhang (Exeter)

• Man Luo (Exeter)

• Amraj Singh (UQ)