Machine Learning

Machine Learning, AI and creative industries:

ML applications are found in many different industries, from online marketing to financial services. Across all these industries, people are using AI technologies to achieve specific goals that help make routine tasks easier or foster innovation.

The creative industries are no exception. Practitioners are taking ML technologies and using them to push the boundaries of what we previously thought possible. In this way, ML is set to revolutionise creative media. New forms of AI can make great differences to the way we work in fashion, fine art, broadcasting, journalism, music and film.

What are these technologies?

As we covered previously, machine learning is the specific use of computers to learn from input data. When a specific bit of software is made to improve depending on the data given to it, it is called a machine learning algorithm. These machine learning algorithms are at the heart of the systems we are talking about here. When a machine learning system has been given data and has learned from it, we can then say that it displays signs of artificial intelligence.

Over the last decade, there has been a sharp increase in the amount of software frameworks and tools available to build machine learning systems. Many feature open source software, meaning the original source code is made freely available and can be redistributed and modified.

These include the following:

Weka

Open source machine learning software used for teaching, research and industrial applications.

TensorFlow

Open source platform for machine learning.

Scikit Learn

Open source Python machine learning toolkit for data analysis.

PyTorch

Open source machine learning framework for research prototyping.

For creative applications, two of the most important tools available are:

MIMIC

Open source collaborative project for creating musical applications using machine learning, machine listening and artificial intelligence.

The Wekinator

Rebecca Fiebrink’s interactive machine learning software for creative applications. We’ll learn more about Rebecca’s work later in the course.

There are many others, some designed for specific tasks, whilst others are more general purpose libraries that can be used to integrate machine learning algorithms into your application. These include:

RapidLib

Javascript and C++ library for interactive machine learning.

Theano

Python library focused on optimisation and evaluation of mathematical expressions using multi-dimensional arrays.

Keras

High-level neural network application programming interface (API) written in Python.

Google ML Kit

Machine learning development framework focused on mobile platforms.

CUDA

Parallel computing API allowing developers to use graphics cards for general purpose computing.

What are the challenges?

The main challenges facing the creative practitioner of integrating these technologies into their workflow are the following

Learning how to code:

If you want to work with deep learning neural networks, you will need to learn how to code.

Learning the individual syntax of each framework:

Each of the frameworks mentioned above have their own way of doing things. To learn how to use a framework, you will need to learn commands specific to that framework.

The machine learning workflow:

There is a general workflow that each machine learning project must follow. The workflow will change slightly depending on whether you are doing deep learning or interactive machine learning. The general sequence of events is as follows:

• Task identification: assess the problem you are trying to solve, and decide whether it is a classification problem or a regression problem.

• Data collection: collect the data that you want to analyse.

• Algorithm choice: choose the best algorithm that you think will give you the best results.

• Train model: give the model examples from the data set and train.

• Evaluation of results: run the trained model on data not in the training set and see if the model processes the new data as expected. If the model does not run in the way you would like, then you go back and adjust the data or the algorithm.

Alongside these challenges, there are also questions about how we should go about using these new systems. It’s all very well and good to have access to these technologies, but we also need to have a reason to use them. They need to be applied to creative processes in such a way that they are beneficial to either the workflow itself or the output. In other words, creative practitioners need to be able to use these technologies to enhance the work they do.

Machine learning systems are also posing existential problems for artists and the art-buying public. As we will see later this week, AI and machine learning systems are forcing us to re-evaluate what we think of as art and what we know about the role of the artist.

Previously we reflected on what we mean when we talk about AI. We also considered what AI could become in the future. In this step we are going to look at some current AI applications that are being used in the industry.

What do we mean by intelligence?

When we talk about artificial intelligence, what level of intelligence are we talking about? The systems we work with today are not alive or aware in any sense of the word. These systems can appear intelligent to the end user, but they are mainly statistical models that give us insight into patterns and trends in time. When a system identifies a pattern, it can then apply that pattern to new input data to create original output. HAL, the sentient computer system from the film 2001: A Space Odyssey, is still what it was when it appeared in 1968: science fiction.

At some point in the future, HAL may make the transition from science fiction to reality. However, the revolution that is happening around us, including in the creative industries, is not in any way because of self-aware AI systems. What we’re now seeing is an increased automation of tasks and processes that, up until now, have been very difficult to implement using computers.

These implementations include:

·         natural language processing (NLP)

·         speech recognition

·         image recognition and classification

·         gesture recognition

·         recommender systems.

Let’s look at each in turn.

Natural Language Processing

Natural language processing (NLP) has a significant history. As with many tasks associated with AI, we are only starting to see its full potential in recent years. NLP is an umbrella term. It describes the ability of a piece of software to analyse and process human speech. It then enables the computer to use the speech as data or commands to perform a task. You can see everyday examples of NLP in action in text completion features in your email, website chat boxes that answer your queries while shopping, and programmes such as Google Translate.

Speech recognition:

Speech recognition is a subset of NLP and stretches all the way back to the development of an automatic digit recognition machine, Audrey, at Bell Labs in 1952. This machine was a gigantic computer that could recognise phonemes, the basic units of human speech. For a more in depth look at the history of speech technology, see the link to the BBC article The Machines That Learned To Listen below. Speech recognition systems listen for vocal commands and process the information to complete (some) requests. This technology has now progressed to the point that we have products such as Google’s Assistant and Amazon’s Alexa in many homes, all providing information for the user based on voice input.

Image recognition and classification

Whilst the field of AI research is many years old, recent developments in ‘deep learning’ have provided a boost for some really amazing work over the last decade. Deep learning is a term used in AI to describe the process of using massive amounts of data as input to a large neural network. This input can then be used to identify patterns in the data.

A neural network is a computer algorithm that uses small units to process bits of information. By adding these small units together, we can identify features in the data that are difficult to describe manually. When these networks are trained on massive amounts of photos, the network will eventually be able to describe the features of the content of the photos. Once the content features are identified, some networks can then produce new images from this information.

You can see examples of this deep learning recognition and classification in systems used to power self-driving cars, as well as more creative applications such as in Google’s Deep Dream generator which allows you to see what a neural network sees with startling results, and Magenta, a software library that is tailored towards creating music and image applications powered by machine learning algorithms. You can find out more about both in the links below this article.

Gesture recognition

A gesture recognition system has the ability to interpret the user’s body movement to perform some task. The user’s movement is captured using a sensor. This movement is then used as input into a machine learning algorithm. The system then produces some output depending on the gesture. You can now change the channel on your TV by simply swiping your hand through the air, and control your laptop by moving your fingers in a certain way. Both are thanks to gesture recognition.

Recommender systems

Anyone who has used the internet will be familiar with targeted advertising. It’s used by companies to promote products they think you will be interested in based on your previous online activity. Thanks to recommender systems, it’s now possible to predict the kinds of things that people might like to do based on the things they have done in the past. They also make it possible to predict how those things they have done might relate to other things.

Recommender systems are deeply embedded in everyday life. From your YouTube watch list to your favourite playlist on apps like Spotify, they can also influence the type of media you may be interested in, so they are incredibly powerful tools.

So far we have learned about the different types of AI that are being developed. These include:

·         voice recognition

·         recommender systems

·         natural language processing.

Over to you

Using the terms introduced in the previous step, can you search online for any applications that are using this technology? Choose one type of AI technology, and look for one or two products or services which rely on this implementation.

How effectively do you think these technologies have been implemented?

Are there any challenges with how they are currently used? If so, how might they be improved?

Think critically and creatively, and share your discoveries with other learners.

Creative AI: creative issues

As we’ve seen in the previous steps, it’s hard to deny the impact of AI on everyday life. But it is also found at the heart of the creative industries, with artists using AI and machine learning techniques to explore deeper issues.

The proliferation of new forms of technology and the creative use of AI poses many additional challenges. For example, harnessing and manipulating algorithms to create new works opens up many ethical questions about authorship. Where does technology end and creativity begin?

The issue here is that one person may write the code that creates an algorithm. This algorithm can then be used by someone else to produce material that becomes an artistic work. Who is the author here? Furthermore, with the algorithm actually doing the work of creating the material, can any human rightfully claim authorship? Should the machine be considered the real author?

upsample and downsample:

This post covers how to upsample and downsample data and the possible pitfalls of this process. Before we cover the technical details let us first explain what we mean by upsample and downsample and why we may need to use it.

Both these techniques relate to the rate at which data is sampled, known as the sampling rate.

We will see below that the sample rate must be carefully chosen, dependent on the content of the signal and the analysis to be performed. Imagine we have some data which has already been sampled, from a particular test or experiment and we now wish to re-analyse the data maybe using different techniques or to look for a different characteristic and we don’t have the opportunity to repeat the test. In this situation we can look at resampling techniques such as upsampling and downsampling.

Downsampling, which is also sometimes called decimation, reduces the sampling rate.

Upsampling, or interpolation, increases the sampling rate. Before using these techniques you will need to be aware of the following.

1. Up-sample Minority Class

Up-sampling is the process of randomly duplicating observations from the minority class in order to reinforce its signal.

There are several heuristics for doing so, but the most common way is to simply resample with replacement.

First, we'll import the resampling module from Scikit-Learn:

Module for resampling Python

from sklearn utilities import resample

Next, we'll create a new DataFrame with an up-sampled minority class. Here are the steps:

• First, we'll separate observations from each class into different DataFrames.

• Next, we'll resample the minority class with replacement, setting the number of samples to match that of the majority class.

• Finally, we'll combine the up-sampled minority class DataFrame with the original majority class DataFrame.

For complete practical understanding on Machine learning please click on My Research where you can find a project.