Journalism and AI

How might we design experiences of journalist-facing AI systems to enable quality journalism?

Today, journalists are immensely challenged by market pressure. More newsrooms are adopting AI technologies (NLP, ML) for efficiency. However, research shows these AI systems might diminish professional journalistic values, such as transparency, accountability and responsibility.

users and technology

USER: Journalists based in the UK who might or might not use AI systems to automate key journalism tasks in near future
TECHNOLOGY: AI systems for journalists

problem

AI technologies to automate journalists' work might compromise important professional values held by journalists

goals

Due to the speculative nature of this study, goals and results are necessarily equivocal and open-ended

To understand how journalistic values might be supported and/or undermined by journalist-facing AI technologies
To draw out design recommendations

research design

Exploratory | Value Sensitive Design | Sample size: 11 | Team size: 6

Value oriented semi-structured interview

As journalism is a profession strongly guided by values, we designed 3-stage semi structured interview focusing on values:

Story-sharing
Value elicitation
Potential AI impact on values

Story-sharing

We asked 11 journalists to pick one of the most important story they've worked on, then to pick 4 or 5 values that they practiced during the news story creation. Here, stories worked as user-generated scenario that contextualise the participants' responses.

Value Cards

We used Value Cards during the interview as a prompt to facilitate value elicitation situated in news story creation

We found:

Despite having diverse backgrounds, participants selected similar values as most important.
Truth, Originality and Public interest were the most mentioned values

AI Cards

Lastly, we asked journalists to tell us how values might be affected by AI systems depicted in AI Cards.

We found:

Journalistic values as a holistic ecosystem; interrelation and tensions between each value
Duality; AI can undermine or support journalistic values.

Greg (pseudonym, right, data journalist) explains a set of professional values that he practiced during the news story creation

Value Cards and AI Cards used together in semi-structured interviews

design Recommendations

Due to the speculative nature of this study, goals, recommendations and results are necessarily equivocal and open-ended

Designers of journalist-facing AI systems should consider the following:

Supporting truth-seeking through AI 'scrutability': Truth was of paramount importance for most journalists and was linked by them to values such as trust, credibility and transparency. Several of the journalists feared automating truth-seeking, citing risks of losing authority or propagating mis/disinformation. This suggests that fully automating tasks such as verification may not be desirable, as the values of truth, accountability and credibility could be significantly challenged.
Supporting impartiality assessment with AI-facilitated sub-editing: AI could act as an ‘external’ subeditor, reviewing stories before publication and assessing if they provide a balanced view. We suggest journalists should be more involved in fine-tuning editorial criteria for flagging non-impartial stories and designers should embrace journalistic value ecologies around impartiality. Responsibility for impartiality should not fall solely on journalists, but also be distributed across designers and data providers, who should adhere to the codes of conduct and ethical standards of journalism
Supporting journalistic accountability: AI designers should be open about the ‘incompleteness’ of data and make explicit what information is used to train models and make decisions. They should also explain the role of AI as sub-editor, letting the user know what sources were used and how to flag a story as unbalanced.
Empowering story idea generation with AI-facilitated discoverability: While some journalists were curious about what ideas a computer might generate, others feared that AI could constrain their thinking by encouraging them to reuse story arguments or that AI would lack the ‘human eye’ necessary for identifying story opportunities. AI systems should engage with the paradoxical nature, where existing stories are used as inspiration. For example, AI could use previous stories to produce visualisations depicting the connections (or disconnections) between existing stories. Moreover, these systems could spark novel ideas by highlighting opposing perspectives or news provenance.

Result

Due to the speculative nature of this study, goals, recommendations and results are necessarily equivocal and open-ended

We drew out design guidelines for designers of journalist-facing AI systems
Design recommendations were fed into the next design iteration
Study was published (acceptance rate: 24%) and we presented at an international conference

Reflection

While participatory design was considered as ideal, semi-structured interview was selected due to difficulty in having time-pressured journalists in one place.
Using AI cards together with the value cards in semi-structured interviews helped ground and contextualise the discussions

Voice of users

“The biggest problem with using AI in journalism is we've got to be able to check it. How do we know it's true? And in other cases it's okay if it works 90% of the time, but that's not good enough for us, it needs to be absolutely correct, you need to be confident that it's true.” --Michael (data journalist)

“If AI gets so good that it writes in a more engaging way than I can, I wouldn't want it to do that.” --Anthony (culture journalist)

"I think it's really, really, really important that when journalists are scrutinising stories that have come out of AI. [...] Just because it is from AI doesn't mean it's any more. truthful, or trustworthy." --David (data journalist)

team

Myself (Planning, recruiting, moderation/interview guide, moderation/interviewing, prompts design, project management, presenting, research paper writing)
Journalism researcher (Journalism advisory, research paper co-authoring)
AI researchers (AI advisory, research paper co-authoring)
HCI researchers (HCI advisory, research paper co-authoring)

Link to Publications

Page updated

Google Sites

Report abuse