Background

In the current context of threats to European heritage, the OTHER (Operational Threats in HEritage Recovery) project, in a culture-driven innovation ecosystem, is crucial to understand, address impacts and guide development, anticipating consequences and equipping Cultural and Creative Industries (CCIs) with skills for new challenges.

The OTHER project is an initiative between technology (science) and humanities (society), which aims to involve citizens and academics, in promoting innovation and the development of technologies framed within cultural diversity. 

But to do this, we need to understand cultural differences and believe in cultural preservation, but also to foster a European innovation ecosystem driven by culture. It is crucial to understand and address the impacts and guide development, anticipate consequences and equip Cultural and Creative Industries (CCIs) with the skills for new challenges.

We present a set of innovative solutions to integrate artificial intelligence (AI) together with cultural expertise, which will be used to design a human-centric AI tool to address challenges in an ethical, sustainable, trustworthy, and culturally sensitive way.

Intuition, imagination and creativity, which are difficult for AI to replicate, coupled with design expertise, can defend cultural values, develop an operating system for heritage support, and unique insights to create human-centric AI tools, that address this challenge

to enhance the cultural experience of European citizens.

The OTHER project is a Digital Humanities project with an interdisciplinary area that brings together methods from human sciences, such as History, Sociology, Arts, etc., using an approach in digital and information technologies.

The project explores the use of technology to develop the study of humanities topics, but also how to increase the quality of automatic responses and constructive criticism.

In a global vision, it is about implementing AI-assisted text analysis processes, image recognition and collaborative annotation among researchers, but also the operationalization of citizen science where the public will collaborate in the collection and production of cultural data.

It involves areas such as Digital History, Art and Interactive Visualization, but especially what can be also called Digital Public Humanities, that is, a project with an interdisciplinary field that uses available digital technologies to involve citizens with research carried out by academic researchers, an excellent and exemplary opportunity to demonstrate relevant processes of accessibility to the study of humanities beyond the University.

In addition to the available tools that the OTHER project's technological partners may use: programming languages, databases, GIS tools, data visualization software, artificial intelligence and machine learning, and collaborative platforms, the ultimate goal is to preserve, analyse and disseminate heritage through digital means, expand access and public participation in the production of knowledge in the humanities and, through artificial intelligence, enable the model to develop automatic interpretation capabilities for cultural data.

The OTHER project is a citizen science project in the digital humanities, because it aims to involve the public in activities such as cultural data collection, recognizing the social and educational impact that public participation can offer to academic research.

Contact with the public will be carried out via an intuitive mobile platform (APP OTHER) for collecting digitized cultural materials (photographs) optionally combined with text, which defines one of the research areas.

A second area of research is linked to the processing of contributions via voluntary APPs, by researchers and with the use of artificial intelligence. It is the most complex area and requires collaborative work between digital and human sciences to develop the capacity to automate the quality process. 

In this context, the OTHER project adds a new level of participation, with the introduction of crowdsourcing, that is, the involvement of the educational and social sector (schools and local associations) in the valorisation of information collected with school and oral memory knowledge.

The specific use of experts in the interpretative process to develop machine learning will be important the more transnational it is. The geographical and cultural dispersion of the participants will be an added value in the annotation, validation and contextualization of images, but also for the validation of the representativeness of the data entered into the system, in what can be typified as representative of the types of data, that is, regarding the positivity, ambiguity, or hatred, of the data transmitted in relation to a theme proposed to citizens.

The OTHER project has the aim and ambition to promote critical visual literacy and to perfect a collaborative method of cultural teaching between machine and humans, in the improvement of AI models.

The overall objective of the OTHER project is to create a working model supported by Artificial Intelligence, for the processing of large amounts of data, but especially a critical process (quality algorithm) with the ability to automatically recognize images and value them through a definition of clear criteria “Classes”, or precise labels such as: “positive theme” (e.g.: historical artifacts, sacred art, or other legitimate symbolic objects); “negative/ambiguous” (e.g.: images, messages that are difficult to automatically classify, or that differ from the topic under analysis), and; finally “hate message” (e.g.: negative political symbols, or offensive attitudes, that is, culturally reprehensible).

Today, it is extremely important to empower European citizens with a support tool (APP) that facilitates fact-checking, and that from a machine learning perspective simultaneously allows validated information to be made available to citizens and to a Cultural Heritage Cloud (ECCCH) type database with positive/validated information on topics analysed in the project.

As is common knowledge, advanced AI models like “CLIP” or “Vision Transformers” have difficulty understanding complex historical contexts, visual ironies, or implicit meanings. It is in view of this fragility that the OTHER project intends to investigate, through case studies to be defined by the consortium, whether a capacity to reduce the complexity of historical analysis and errors in duplicate automatic classifications will be developed.

Working with transnational multidisciplinary teams, where they promote the Gender Equality Plan and Ethics Issues, opens up the range of cultural, social and scientific areas of expertise as broad as History, AI Technology and IT Programming. This opportunity will allow the integration of historical and language debugging concepts that will transversally value descriptions, and metadata.

The use of a screening interface by human experts with explanatory visualizations of the data (photographs and text) for comparison with the results of the AI pre-filtering will be the method to calibrate the algorithm, which will make the “negative/ambiguous” class, the review class par excellence of the model under development.

Throughout the research, all the work is measurable and verifiable. The validation of the results of the automatic filtering by human expert researchers considers it is realistically achievable to reach a goal of 80% of positive classifications, for the introduced case studies at the end of the project. 

Thus demonstrate the feasibility of planning targeted teaching processes on AI algorithms, using information randomly provided by volunteers and validated by AI, that is, obtaining a systematized process of reducing “ambiguities”. A strong proposition for using machine learning (ML), a branch of artificial intelligence (AI) that focuses on enabling computers to learn, from data without being explicitly programmed.

The OTHER project is based on process transparency and, as such, is open to independent audit processes, from the AAP OTHER to the systematic review of the final algorithm.

One of the vectors of the process' success is the option of transferring data via the internet, through a simple but secure mobile IT program (APP). This APP will only be used by volunteer citizens who wish to collaborate in the project, that is, within the method currently called citizen science, in which responsibly identified people contribute to the collection of useful information, usually from geographically dispersed points.

It is very important to understand the voluntary participation of citizens in this process, not only due to the diversity of contributions, within the themes to be addressed throughout the project, but also because the transnational geographic dispersion itself will allow for a socioculturally differentiated data collection, something impossible to achieve through traditional direct interview processes, which would imply irrational costs.

The AI ​​filtering system will evolve throughout the project, with automatic debugging and data tracking analysis, first essentially comparative with human decision-making capacity, then progressively less human and more automatic.

The data received is classified by type and, in addition to all being saved in a log (auditable), the AI filter and human experts in the various stages will separate the results into distinct areas (positive, ambiguous, hate), with only positive information being transmitted to the ECCCH. The “non-positive” data (ambiguous and hateful) are of particular importance for a future understanding of the illiteracy of volunteers and their lack of social integration.

With the process of recognizing physical heritage (natural, scientific, industrial, artistic), we are opening up an extensive catalog, but in particular the possibility of transmitting data (photographs, georeferenced and comments), to a public database (Cultural Heritage Cloud) accessible to everyone, but with a quality filter (validation) based on artificial intelligence.

What constitutes an innovative level, within what can be considered “Digital Humanities”, a strengthening of the use of Artificial Intelligence module, with the purpose of overcoming human limitations in processing large amounts of data, over long periods of time.

For the development of digital humanities, the use of AI allows for process automation, definition of decision-making and problem-solving, filtering by algorithms, speed and efficiency (productivity), but above all innovation.

The development and progressive increase in the positive analysis capacity of the AI algorithm will, in this sense, be a contribution of the project to the work of cultural professionals and creative companies, because they will be able to capture online knowledge, for new perspectives and opportunities for cultural and creative services, for example, knowing trends and changing the focus on regional cultural hubs.

Thus, the OTHER project is innovative because it does not start from an economic perspective (marketing), but from a cultural perspective to design the functionalities of the APP OTHER interface (human/machine), with the ambition of motivating young people in an attractive way to collaborate in the project and in the University's effort to use Artificial Intelligence as a useful tool in the service of knowledge.

It is possible to achieve results, yes. The OTHER project will enhance educational processes, encourage adoption of AI tools and, above all, dispel doubts about the alleged negative impacts of AI in the cultural field.

It is based on a team of researchers with multidisciplinary knowledge and contributes to the consolidation of the existing cooperation between the Naval Research Center, the Marine Museum and the History Center of the University of Lisbon.