Machine Learning Gets Real

Basic artificial intelligence (AI) is no longer the domain of science fiction, as evidenced by the buzz surrounding machine learning, neural networks, and data mining techniques. The support infrastructure and readily available software make it easier to practically employ these powerful techniques in everyday problems. This article gives an overview of how the new AI capabilities can be used in products today.

How machine learning works

In many ways, machine learning is not all that different from traditional statistical techniques used to make sense of raw data used for over a century. Both take some observations from the world and use that data to make predictions of what that information represents. These can be classification problems, such as disease diagnosis, and determining whether a piece of email is spam, or a regression problem, such as trying to come up with a number that represents the odds of something occurring, like the chance someone will default on a loan.

There are a number of techniques used in in modern machine learning, but they typically follow the same basic pattern: We have a number of factors that when combined, can be used to predict some result. We collect a number of examples, called a training set, each of which contains the factors and importantly, the actual outcome that we want to predict in future examples (when that outcome is not yet known).

The flu or just a cold?

Suppose we want to identify whether someone has a cold or the flu from the symptoms he or she might present to their doctor. We first identify a number of factors that are likely to indicate both conditions; sniffles, sneezing, fever, etc.. Then for a large group of people that we have already diagnosed as having one of the other conditions, we collect information about all those factors and put them into a spreadsheet like this: