Diversity and Inclusion in AI Symposium: Accepted Abstracts

The use of Artificial Intelligence (AI) in recruitment is rapidly increasing and drastically changing how people apply to jobs and how applications are reviewed. In this paper, we use two field experiments to study how AI recruitment tools can impact gender diversity in the male-dominated technology sector, both overall and separately for labor supply and demand. We find that the use of AI in recruitment changes the gender distribution of potential hires, in some cases more than doubling the fraction of top applicants that are women. This change is generated by better outcomes for women in both supply and demand. On the supply side, we observe that the use of AI reduces the gender gap in application completion rates. Complementary survey evidence suggests that this is driven by female jobseekers believing that there is less bias in recruitment when assessed by AI instead of human evaluators. On the demand side, we find that providing evaluators with applicants’ AI scores closes the gender gap in assessments that otherwise disadvantage female applicants. Finally, we show that the AI tool would have to be substantially biased against women to result in a lower level of gender diversity than found without AI.

This research aims at utilizing artificial intelligence (AI) to enhance the learning of Australian Sign Language (Auslan) for two distinct target groups: parents of deaf children and children of deaf adults. The research intends to harness AI-driven technologies to facilitate effective communication and bridge the gap even in the smallest societal unit Family. By leveraging machine learning algorithms and gamification design, the study seeks to develop tailored educational resources and platforms/software that cater to the unique linguistic needs and preferences of these two groups. The outcome of this research will foster stronger connections within deaf families and even a more inclusive society in Australia. It underscores the significance of AI in revolutionizing Auslan education and promoting cross-generational language acquisition.

Both research literature and industry press offer examples of gender bias in artificial intelligence (AI), with harmful consequences for individuals, communities and society at large. However, the existing multidisciplinary research offers very limited insights into management of gender bias in AI, including prevention and mitigation of its adverse effects. Consequently, there is a need for a better understanding of various contributing factors to gender bias in AI and suitable mitigation strategies. Therefore, the main objectives of this research are to: (i) Investigate and articulate different contributing factors to gender bias in AI, along with possible mitigation approaches. (ii) propose a research-informed conceptual framework of gender bias in AI.
This research is situated in the context of AI recruitment systems because they have been reported as biased towards gender.

This research deployed multiple research methods including interviews with 20 AI experts followed by a two-phase Delphi study with experts, which was used to refine the findings from the interviews. Based on the empirical findings, this research conceptualizes gender bias in AI as a socio- technical phenomenon and proposes a theoretical framework for its management through socio- technical approaches in an organizational context. The theoretical framework identifies and describes both internal factors, which are technology, organizational and human-related as well as external factors such as societal factors – all contributing to gender bias in AI. The theoretical framework also contextualises different mitigating approaches in the light of organizational theory, making the results more useful for socio-technical and organizational researchers and practitioners. Based on the proposed framework, this research also discusses possible management strategies for gender bias in AI, including its prevention and mitigation.

Artificial Intelligence (AI) holds immense potential to revolutionize our society, yet it also presents a formidable challenge in the aspect of diversity and inclusion (D&I). Despite AI's rapid advancement and widespread applicability, the incorporation of D&I considerations is often overlooked during AI's design, development and deployment, resulting in numerous D&I related AI incidents. However, there is a deficit in the understanding on D&I issues in AI systems that often perpetuates bias, unfairness, inequality, and discrimination. Furthermore, to the best of our knowledge, there are no existing tools, guidelines, practices, or strategies that help ascertain whether AI incidents directly or indirectly stem from bias or discrimination related to D&I. Neither do we have means to recognize incidents that might inflict D&I related harm. Recognizing this shortfall, we conducted a comprehensive study to delve into these D&I issues in AI incidents. We conducted an in-depth manual analysis on the two well-known AI incident databases (AIID and AIAAIC) to discern which incidents were primarily rooted in D&I issues and identify the diversity attributes that triggered these incidents. Following this exploratory investigation, we applied our findings to establish a set of criteria that ascertains whether an AI incident is fundamentally based on D&I issues, thus providing a more concrete approach to identifying and possibly mitigating such incidents in future. This study seeks to raise consciousness about the significance of D&I issues in AI systems, utilizing our investigative results as advocacy tools underscoring the crucial role of D&I in AI. In future, we aim to map D&I guidelines with AI incidents, offering a roadmap to operationalize these guidelines effectively and thus reduce the D&I related harm caused by AI systems.

Ethics in AI has become a very important topic of discussion in recent years. Particularly in recent years, we have been hearing a lot about AI-based systems being biased against humans based on their gender, race, ethnicity, language, etc. For example, Amazon’s gender-based recruitment tool was biased against women, the US’s health algorithm was racially biased against black people, etc. Because such biases not only harm people's sentiments but also can lead to economic and opportunity loss, it is crucial to prevent them. Different regulations, strategies, and tools have been introduced by different organizations in recent years to mitigate such AI biases. However, such incidents still persist, and people are still suffering due to the biased AI-based systems.

Therefore, to gain a closer understanding of the causes of such biases so that effective strategies can be developed to mitigate such biases, we are conducting semi-structured interviews with AI practitioners who develop AI-based systems. The 10 interviews aimed to uncover their firsthand experiences of handling biases when developing AI-based systems. We used the Socio-Technical Grounded Theory (STGT) method for data analysis. Our preliminary findings from 10 interviews show that the causes of AI biases include, (i) lack of access to large datasets, (ii) unawareness of biased data, (iii) lack of representative datasets, and (iv) tedious chains of approvals. Among these, the majority of AI practitioners believed that the lack of a representative dataset is the main cause of AI bias. When probing deeper into the reasons for the lack of a representative dataset, most participants believed that no perfect dataset existed in reality, so it is impossible to capture 100% dataset to train an AI model. Our future objective is to conduct interviews with more AI practitioners to gain in-depth insights into the experiences of AI practitioners on AI bias. We aim to uncover valuable insights and investigate potential effective approaches for addressing and reducing AI bias. Attending this symposium will enable me to get feedback on my research from experts, exchange research insights with fellow peers, and expand my network within the community.

Diversity, Equity, and Inclusion (DEI) is an integral part of many ethical frameworks used to regulate AI (Ryan & Stahl, 2020). However, DEI is often criticised for exposing issues but not addressing them, for reinforcing patterns of tokenism and assimilation and for being rooted in the white-dominant culture (Ballard et al., 2020; Davis, 2021a; Dobbin & Kalev, 2016; Hagendorff, 2020; Munn, 2023; Nalliah et al., 2021). In the face of these limitations, there are calls to approach DEI from the perspectives of belonging, dignity and justice (Davis, 2021b; Saxon, 2021). As a way of considering dignity in AI development practice, Ruster & Snow (2021) developed a “Dignity Lens”. The working definition of dignity in the Dignity Lens is that it refers to the inherent value of individuals, not connected to usefulness and equal amongst all humans from birth regardless of identity, ethnicity, religion, ability or any other factor. Taking a cybernetic approach of seeing dignity as an ecosystem and drawing upon Hicks (2013) 10 essential elements of dignity model, the purpose of the Dignity Lens is to provide a way to understand how a particular artefacts (for example, a policy instrument, an algorithm, or a project plan) impacts the Dignity Ecosystem. Felstead (2023) used the Dignity Lens to analyse written submissions into the recent Royal Commission on Robodebt and its impact on welfare recipients. This research, using the Dignity Lens, reveals violation of the recipients' dignity. This presentation will be delivered in a conversational style between the original Dignity Lens creator and the author of the latest Robodebt application case. The presentation will outline the Dignity Lens and dive into its application to Robodebt - unpacking how the tool was used, what insights it revealed and how it may be improved in the future. It is hoped that this conversation regarding an emerging tool applied in practice will spur further conversation regarding dignity in the AI ecosystem of humans, data, processes, systems and governance.

Artificial intelligence is changing the public’s perception of technology. Generative AI, which describes a type of AI able to create novel content — in the form of text, image, audio, or video — represents the most publicly engaging delivery of AI. Its humanistic veneer, most visible in AI chatbots such as OpenAI’s ChatGPT, is remarkably persuasive. Using our conversational skills, we can ask the AI questions, comment on its replies, and further interact with its responses as we would with a human partner. No-one would deny that this form of AI is set to revolutionise our way of sourcing, synthesising, and shaping the very knowledge that we rely on to think, work, and play. And yet, given the revolutionary potential of this new synthetic human form, what do we know about the impact of this technology on all members of society? When we talk of responsible implementation of AI — “for and with society” — who in society are we thinking of, and who have we forgotten? We designed a programme of research aimed at filling this gap. Beginning with a public facing survey, we measured socio-demographic, socio-cultural, and socio-technical identities, shedding light on differences in perceptions, attitudes, and experiences of AI. Further, we measured a range of psychological mechanisms hypothesised to underlie these differences. Identifying these mechanisms allows us to design and deploy AI applications that can be calibrated according to the principles of inclusion, reflexivity, and respect for diversity that lie at the heart of Responsible Innovation. In this talk, we will present preliminary findings from this first wave of data and demonstrate how these insights will contribute to translating RAI principles into RAI practices.

Businesses worldwide have been feeding AI with massive amounts of data to deliver products helpful in influencing decisions on human problems. To increase the chances of AI identifying the best decision options, we assume the availability of enough knowledge to support contextualised decision-making. However, data usually says half-truths rather than everything AI needs to solve human problems. These half-truths are often defined by or become incomplete or even wrong stereotypes over time. Whereas data enables accumulating knowledge at scale, it provides solely a  partial view, which is a major cause of AI failures. Data represent past disconnected actions related to vague, ephemeral and inferred contexts, devoid of clear human values and intentions. When solving people’s problems, social, emotional, and humanistic components exist and cannot be detangled. In this talk, we argue that to solve human problems efficiently, we must go beyond data and listen to human experiences. In a moment when contemporary AI systems are becoming human-competitive at general tasks, the best we can do is to understand and exploit what makes us humans. In this sense, we introduce the experience-augmented Responsible AI framework for exploiting practical representations of human experiences that complement data, bringing diversity and inclusion to AI. Increasing diversity and inclusion is humanity’s most effective way to solve partial-view risks. We diversify views to enhance our capability to frame problems,  identify risks, propose alternative solutions, and consolidate better businesses. An A-game for AI-driven businesses is designing tangible ways to incorporate experiences into the AI innovation framework.

Diversity, Equity and Inclusion (DEI) agendas suffer a common critique: that words are empty without action (see for example Ballard et al., 2020; Davis, 2021; Dobbin & Kalev, 2016; Saxon, 2021). Often referred to as the ‘principles-to-practice gap’ (Morley et al., 2021), there is great need for practical ways to ‘walk the talk’ on values, including commitments to diversity, equity and inclusion, particularly in the context of new technologies (Karim, 2023; Lekhraj, 2023; Trim, 2023). This challenge of putting principles into practice in organisations’ everyday decision-making is one that permeates all types of organisations, but is acute in startups who are operating in ‘move fast and break things’ cultures and who are increasingly called to get onboard with Responsible AI (Eliot, 2022). In this presentation, we discuss two startups who want to ‘be responsible’ and how they are using generative AI to prompt them to stay true to their values through taking aligned actions. The first case study experiments with prompting ChatGPT to “act as a product manager for an early-stage startup who is developing a Minimum Viable Product (MVP)” and give suggestions for how each of their values should be considered in the product development process. The developer co-founder in this case found the process valuable. They used the outputs to create a matrix of considerations and self-rated actions taken to date against the matrix, finding new areas of focus for future actions. The second case study prompts GitHub CoPilot to prompt the developer co-founder to think about the startup’s Responsible AI values during the coding process itself. This example is yet to land in a comfortable resting place, citing difficulties in zooming in and out from values to coding in real-time as a large barrier to implementation. These two case studies highlight the potential value of generative AI in addressing concerns around the demonstration of values in action and points towards the importance of temporal considerations in putting principles into practice. Future work will continue to follow these startups’ use of generative AI in ensuring actions align with values and future iterations on these prototypes for effectiveness in their respective organisational contexts.

The organisational context within which AI is developed often relies on top down, disembodied approaches to AI Ethics. This process risks being disconnected from the everyday workflows of practitioners, lacks representative input from outside of the organisation and frequently leads to what we refer to as the ethical intent to action gap. 

We propose that AI Ethics can and should be diverse and inclusive by design. An active invitation to participate, rather than a top down mandate. AI Ethics can be integrated and embedded into the everyday workflows of the cross functional teams responsible for model development, deployment, monitoring and refinement. It can result from a highly participatory process that goes beyond the organisation's boundaries, including customers, regulators, independent advocacy groups, and others that help meaningfully represent the diverse views, life experiences and value systems of the people that will be directly and indirectly impacted by AI.

We are approaching this challenge at the intersection of language, imagination, and practice that reflects the symbiotic relationship between humanity, technology, and the natural world. This ecological grounding to AI Ethics has culture shifting potential. Building on decades of collective industry experience focused on helping organisations operationalise technology, data and AI ethics, we have recently begun conducting action research to explore the validity of our proposed approach to changing the culture within which AI Ethics Principles are diversely and inclusively established, then operationalised. We are humbled by the opportunity to potentially share some of our work with the research community. We look forward to participating in continued discussion.

Academic research and popular media are rife with examples of unfair algorithms that warrant scepticism (Glikson and Woolley 2020). Seen as particularly problematic are those machines that make decisions about one’s lives and livelihoods, such as whether someone should or shouldn’t be hired (Kochling and Wehner 2020). Consequently, inquiries pertaining to the ethical and responsible deployment of AI tools within the realm of recruitment have become the centre of heightened scholarly attention (Kelan 2023). 

Much of the current research on discrimination in AI-recruitment has focused on how algorithms impact single-issue marginalised job candidates (e.g. gender bias in job ad delivery in Lambrecht and Tucker 2019). However, our identities are rarely one-dimensional, and more often a set of various interlinking characteristics. Thus, the aim of this study is to extend the current debates on diverse and inclusive recruitment to take an intersectional approach, understood as the interlocking systems of power and oppression based on gender, race, sexual orientation, class, disability (Crenshaw 1991), and apply that lens to the case of AI-supported recruitment.

This study delves into the evident divide in trust and perceptions of fairness that runs along diversity dimensions, within the context of AI-supported recruitment. This divide significantly influences not only the initial decision of potential job applicants to submit their applications but also shapes their perceptions of the equitable nature of the entire candidate selection process (Shick and Fischer 2021). Additionally, its ramifications extend to the recruiters and employers, who are compelled to deliberate on whether to place their trust in the automated systems (Lacroux and Martin-Lacroux 2022). Addressing this divide with an intersectional lens is crucial for the future development of AI-supported recruitment so meaningful change would come about.

Built on a comprehensive desktop review as its foundation, this conceptual paper constructs an intersectional action framework that impacts each pillar of the AI ecosystem as outlined by Zowghi et al. As the gravity nor the complexity of intersectionality in understanding job seekers‘ and employers’ experiences with AI-supported recruitment cannot be understated, this study is sure to provide an important contribution to the AI and D&I literature.

AI interviews enhance efficiency and quality in decision-making and the procedural aspects of interviews; however, often they are perceived as overly simplistic and at times unfair (e.g., Gonzalez et al., 2022; Koch‐Bayram, Kaibel, Biemann, & Triana, 2023; Köchling, Riazy, Wehner, & Simbeck, 2021; Suen, Chen, & Lu, 2019). Previous studies have predominantly focused on the comparison of perceived fairness between AI and traditional interviews (e.g., Mirowska & Mesnet, 2022; Zhang & Yencha, 2022), neglecting to explore perceived fairness specific to AI interviews. This study suggests that perceived fairness in the AI interview process is bound to augmentation, the intricate balance between AI interview and human involvement (Raisch & Krakowski, 2021), especially for marginalised communities. We later validate the proposed research through an empirical study. 

This abstract delves into the important discourse of fairness perceptions, advocating for marginalised individuals within the context of the AI-powered interview process. We emphasise the importance of providing marginalised candidates with the option to interact with both AI and human interviewers, creating a more empathetic and personalised interview experience. Mind perception theory, widely applied in previous AI studies, suggests that experience -but not agency- is perceived as fundamental to human perception, and notably lacking in AI tools (e.g., Bigman & Gray, 2018; Shariff, Bonnefon, & Rahwan, 2017). Thus, augmentation holds the potential to address this absence of agency. In fact, we posit that it serves as a pivotal boundary condition of achieving a perceived fair AI interview, especially for marginalised applicants.

The research also highlights the need for a comprehensive understanding of fairness that transcends statistical parity founded solely on unbiased data, extending to encompass contextual nuances. For that reason, we propose proactive approaches, such as collecting inclusive datasets, conducting regular algorithmic audits, adopting transparent decision-making processes, and considering local context to further foster a fairer and more inclusive interview experience for marginalised job candidates. In conclusion, this study underscores the responsibility of AI-powered recruitment processes to prioritise fairness, especially for historically marginalised individuals. By combining augmentation and technical precision, stakeholders can contribute to an inclusive job market that values a diverse range of talents.

In this paper I reflect on responsible AI research undertaken with local and Indigenous communities in remote and regional Australia to reflect on present and future scientific requirements needed to combine rigorous scientific standards with ethical data practices, so we are all better equipped to make informed decisions and take actions in our mission to safeguard and sustain our unique environment.

The rise of global migration flows in the past decades has not only resulted to people who are disconnected from their home networks and government, but also who are trying to fit in within their new locations. One of the biggest struggles immigrants and people with difficult pasts have in new countries is what we commonly refer to as ‘the proof of identity’ – which entails proof of character and trustworthiness. This not only makes it difficult for new people to access opportunities and other social amenities like jobs, housing, credit and other services without local references or experience – but also takes a toll on their time and finances as they try to acquire piles of documents to prove their character. Inferring personality from online profiles offers an opportunity to bridge that gap that separates people, as it has become a fundamental part of the way human beings interact now and, in the years, to come. We conducted a data driven investigation into the correlations between online content-based personality inference and psychometric survey personality measurements to determine whether online content can be a good approximator of more traditional psychometric surveys. Using the International Personality Item Pool (IPIP) 50-item Big 5/OCEAN model survey, we found a statistically significant (p < 0.05) positive correlations of 0.34 (34%) in Conscientiousness scores between the two methods, and 0.36 (36%) in Extraversion scores. Both these traits have already been linked to employability. We also found strongly predictive ordinary least squares regression models linking online-content inferred personality analysis to psychometric survey personality analysis. In this presentation, we argue that online content does in fact reflect ‘real-life’ personality traits. We further argue that an individual’s online presence and data is an asset that should be available for them to own and leverage in support of their real-life goals. As such, our findings have positive implications not only in the fields of recruitment, employability but also increasing access to other social amenities like those seeking for housing, credit, and other community services for both individuals and service providers.

The increasing popularity of large language model (LLM)-based chatbots in natural language processing tasks, particularly in the customer service domain, has brought attention to their embedded biases and insensitivity towards users of diverse backgrounds, cultures, and perspectives. These shortcomings often arise from biases present in the training data. This study aims to address this significant gap by enhancing the inclusivity of LLM-based chatbots using a novel approach that leverages simulated personas and scenarios to inform prompt engineering. The methodology involves three main steps: (1) creating personas that represent a diverse range of backgrounds, cultures, languages, and perspectives, (2) developing scenarios involving the personas where inclusivity and sensitivity are particularly important, and (3) simulating conversations between the constructed personas and LLM-based chatbots to identify limitations and biases. The study employs two customer service-oriented chatbots: one designed to deliver biased responses, and another equipped with a self-assessment module to produce more sensitive and less biased responses. We present our insights derived from the analysis of simulated conversations with the two chatbots to inform the design of LLM-based chatbots that are supportive of inclusion and diversity. Ultimately, this project assesses the suitability of using LLM-based simulations for the inclusive design of chatbots in the customer service domain. Furthermore, this study will contribute to a dataset of simulated conversations that can be leveraged by the broader HCI community to support the inclusion and diversity in LLM-based chatbot design.

Gender perception in AI is flawed with understanding based in binaries and in facial structure, rather than in gender expression. This form of system is predicated on a gender binary and on a strictly biological view of gender. This project, from my PhD practice based work, explores gender as a 3D color space. Making use of a Multi Label classifier trained in pronouns and largely artificial faces, this classifier assigns users a color instead of a gender. Gender Tapestry explores how our perception of our own gender is unique because it incorporates expression and experience and exploration. The artwork stemming from this practice led research is an evolving and growing mosaic of faces as more people interact with the system and receive a color and it grows in detail and complexity, reflecting our increasing understanding of gender and the beauty of our increasing diverse celebration of gender.

Thoughtful self-reflexive experiments by artists, writers, and hacktivists who build and/or use Artificial Intelligence systems to investigate a multiplicity of themes, including AI itself, do not appear out of the ether. Rather, closer consideration of a project will suggest its tentacular and rhizomatic lineages. Such influences range from artform and genre-specific histories (for example, rule-based musical compositions and conceptual art associated with Fluxus, art manifestos, bricolage, sampling, mash-ups), to localised and/or networked movements mobilising discriminated and marginalised communities in struggles against oppression (for example, the AIDS/HIV activism of ACT UP, public performative gestures by Extinction Rebellion, use of distinctive iconography and symbolism by the Zapatistas). Some artists and writers employing AI explicitly engage with social and political injustices connected to their own communities and lived experiences, whether these be geographically or culturally based, or related to gender identity, sexual orientation, Indigeneity, disability, and so on. Others using AI create speculative fictions, casting hyperstitions to move beyond cyberpunk’s bleak dystopian landscapes. And some conceptually or materially hack into AI’s black boxes to reveal platform limitations and model flaws that can produce differential harmful impacts without sufficient interrogation, transparency, accountability, and regulation.

This presentation applies a Cyberfeminist lens to some creative AI works that exemplify what Critical Art Ensemble call ‘cultural activism.’ That is, art, writing, ‘electronic disturbances’, and other forms of cultural expression that interrogate the status quo, power, and politics. Art that tests boundaries, and produces platforms, expressive modes, and forms of exchange outside dominant systems built on repression, subjugation, exclusion, and dominance. Some compelling algorithmic art has been made by autodidacts who code their own projects. However, programming skills are not a prerequisite to produce something that is formally innovative, intellectually challenging, or affectively engaging. The belly of the beast is stuffed with processed goods, and an examination of its entrails can be an effective form of divination of other possible and kinder futures.

Multimorbidity is ubiquitous in healthcare clinics, in the lives of 80% of older adults, 45-60% of working age adults, but is only mentioned 1.5% of research papers and clinical practice guidelines. If multimorbidity and diversity have been systematically excluded from the research, it makes no logical sense to then “generalise” (impose) those research findings and then wonder why it made things worse — especially for conditions where multimorbidity is the norm. Excluding disabled and neurodivergent patients medically is proportionally equivalent to excluding all black and Latinx and queer consumers commercially in the United States. That’s one in six of the population, and one in three of folks who use healthcare. The chasm between research data and real-world health is obvious to those with lived experience of disability and diversity, who have traditionally been excluded from the data and the decision-making. 

We don’t talk enough about coercion in healthcare. Or ableism and inaccessibility in academia. Or clinical relevance and inequity in medical research. They remain in the system, while the people who see them clearly fall away. By not talking about these issues, they too frequently become normalised, unquestionable, entrenched, reinforced, and reproduced. Inclusion of cognitive diversity and lived experience in health Ai and research can identify and reduce blind spots, rather than amplify them. To enable this, I will share case studies of examples of open scholarship, Kaupapa Māori research methodologies, and universal design principles at all stages of health AI research.

Community-representative staff, students, and leadership, and clinically-representative research samples — aim to include knowledges from diverse cultures and the most severely affected and mobility-limited participants who have been arbitrarily erased or excluded from traditional on-site single-disease research studies. Person-centred design and equity-driven data collection — consider the invisible sample: “who is centred, who is represented, who gets to speak, who is silenced, missing, or erased” from medical literature, clinical studies, population health, research samples, follow-up, and knowledge sharing. Co-design, co-analyse, co-produced research, clinically-representative datasets, investigate specific and systemic causal mechanisms that transcend population inferences. Accessible knowledge generation, and knowledge sharing tools — accessible for creators, not just audiences

Artificial intelligence has transformed many sectors and driven innovation, demonstrating its vast potential. However, AI also presents serious risks of harm due to the lack of diversity and inclusion considerations. While numerous governance solutions exist to manage AI-related risks, these solutions do not explicitly address the unique challenges arising from the absence of diversity and inclusion practices in AI. 

This paper introduces a novel AI governance framework, aiming to mitigate risks rooted in diversity, and inclusion. The proposed framework is based on a systematic review of existing AI governance solutions, encompassing frameworks, models, and tools, which involved an in-depth analysis of 51 studies. This analysis revealed a glaring gap: a lack of attention to diversity and inclusion-based risks within AI systems. Although untested in real-world scenarios due to its ongoing research status, this framework strives to fill three crucial gaps observed in the current AI governance frameworks. Firstly, it explicitly addresses diversity and inclusion-based AI risks, including challenges tied to biased decision-making, transparency, and inclusivity. Secondly, it underscores the importance of human involvement—engaging experts, developers, and the public—to enhance the overall governance process. Thirdly, it emphasises reducing ethical risks encompassing various ethical dimensions of AI. 

In this talk, we will first present the analysis and results from the systematic literature review on AI Governance. This will be followed by introducing the proposed AI Governance framework and its components that address diversity and inclusion considerations in AI.

As AI continues to shape our digital landscapes, it must authentically reflect multifaceted human experiences and cultures and avoid reifying dominant voices at the expense of the marginalised. Much work in this arena has focussed on improving training data but there are other important areas to address as well, notably evaluation processes. Three key aspects of evaluations of AI where exclusion may occur are the testing data, goal setting, and task design. To explore these aspects, I will provide simple examples based on the differences between dog people and cat people, as well as real world examples where marginalised human groups can be impacted.

Sensitivity to Evaluation Data Distribution: The choice of evaluation dataset might be inappropriate, exacerbated by cultural and contextual incongruities. Biases or inadequate representation within these datasets can magnify issues. Imagine assessing a cat image generation model using a dataset of dog images. 

Subjective goal setting: Goal definitions can encode the designer's values, unintentionally excluding certain groups. Success metrics may implicitly assume specific value standards and endorse particular normative assumptions. Consider a 'cat person' setting goals that inadvertently favour cat images over dog images.

Subjective task design: The design of the evaluation task necessitates design choices that can echo the creator’s cultural and personal values, potentially misaligning with other groups. Picture testing a model designed to generate short stories about pets; the designer may decide to write a list of story prompts as part of the evaluation process. If the designer is a fervent dog enthusiast, they may unconsciously lean towards dog-centric prompts undermining the test’s ability to assess the model on stories about cats. This scenario illustrates how subjective task design, influenced by personal values, inadvertently privileges one group (dog enthusiasts) while marginalizing others (cat enthusiasts). 

I will provide practical advice on mitigating these sources of non-inclusion and bias in evaluations of generative AI systems. By cultivating a new generation of evaluation frameworks that amplify the voices of underrepresented individuals, we pave the way for a truly inclusive approach to generative AI.