Food Preference Prediction Through On-Screen Ordering

The initial problem statement for this project was very broad: Help KLM become the most customer-centric, innovative, and efficient network carrier using AI. There were several routes we could have gone down, but from our user research, competitive analysis, and product requirements, we were able to zone in to one specific area of the passenger experience - food.

Let me start from the beginning...

Fig. 1: Possible routes we could have taken

First, we set out to understand the high-level constraints of a project that seeks to implement AI to improve the customer journey. SET factors were a good way to start.

International, cultural differences can play a huge role in determining people’s behavioral patterns, however, because of the data these models may be fed, they can be highly biased to the point that they start ethnically profiling subjects, i.e. may promote immoral ideas like racism, which poses social problems. Separation of classes in-flight should also be considered, as seats of higher-class are a sizable percentage of revenue for airlines. Artificial Intelligence has a huge economic role to play in the airline industry as well. Because of the fierce competition in the industry, most if not all airlines, function on low profit margins. Machine learning and automation are excellent ways to improve the efficiency of many processes, which can in turn increase the margin of profit. The technology to make this possible is out there: micromobility options, biometrics, data collection, internet connection, automation, machine learning libraries, etc., but it seems like the biggest concern is on whether a concept can be scaled up to be highly profitable while also improving the overall customer experience.

Fig. 2: SET Factors

To position ourselves, I developed a stakeholder map to gain a better understanding of who we might have to talk to about the industry and potential ideas. Initially, we had several stakeholders positioned depending on the amount of power they had and how interested they might be in getting involved. Because we were focusing on the Customer Journey, it also made sense to create a higher resolution illustration of the customer base, where the axes are how frequently this group flies and how big the group is.

Fig. 3: Stakeholder Map v1

Fig. 4: Customer Base v1

To humanize the initial requirements posed by KLM’s problem statement, we scheduled a visit to SFO airport. There, we interviewed several passengers ready to go through the TSA process. The following empathy map highlights the quotes of interviewees at the airport.

Fig. 5: Empathy Map

This trip was also a great chance for our team to get to know each other better, which was a big part of us working better together.

Because of the open-ended nature of this project, team management and organization was critical to make meetings efficient and deadlines achievable. I proposed the use of Google tools such as Google calendar and Google drive. We developed meeting notes for our biweekly meetings and a Gantt Chart to help keep us on track (even though it was difficult to keep it up since we were unsure what the assignments entailed in the future).

Fig. 6: SFO Airport Trip

Fig. 7: Google Drive organization

Fig. 8: Google Calendar invite

As a manager, I distributed these responsibilities to each member to give everyone the chance to contribute to the team. From there, we could keep moving forward more effectively. Upon researching the airline industry, identifying the pain points of passengers through interviews, and honing our understanding of the stakeholders and customers involved, we met together to identify many product opportunity gaps (POGs).

Fig. 9: POG Generation

The POG generation resulted in many POGs, which we then categorized into groups that characterize different segments of the customer journey. To choose the best category to focus on, we developed a weighted matrix - factors and scores are listed in the weighted matrix below.

Fig. 10: Weighted Matrix

As a team, we decided to ignore Purchasing, since it was solely an app-based project. We chose to focus on the in-flight experience, specifically how the basic needs, services, and comfort could be improved. In addition, we updated our stakeholder map and customer base.

Fig. 11: Stakeholder Map v2

Fig. 12: Customer Base v2

To get a better idea of what ideas were feasible within the in-flight design space, we developed a competitive analysis position diagram. Although many ideas in the operations side could be implemented and yield quality results, our team was to focus on the Customer Support aspect of creating a new product. Furthermore, each product can be mapped unto resource and time optimization (although looking back, this may be confusing because time can be seen as a resource. In this case, resources include concepts like food and money, where as time is just time).

Fig. 12: Competitive Analysis

Fig. 13: Airplane food meme

As it is in our generation, memes are a great way to communicate ideas. Here is one that I found very relevant to the in-flight experience. Subconsciously, it may have directed us to the choice we made on our final concept. For some key products, each of us developed a Value Opportunity Analysis (VOA). I developed a template in which each teammate could populate the metrics of a product.

Fig. 14: VOA template

I specifically did a VOA on a product that is not on the market - using biometrics to take a customer from street curb to the airplane directly. The idea is that a customer staying at a hotel can call an autonomous car to take them to the airport. In the car, the customer can scan their fingerprint to tell the car what the destination is. If it’s the airport, the fingerprint could serve as a passport. The TSA process could be done in the car, if the car has the equipment and tools to scan luggage. Without having to walk around the airport or become disoriented in it, the customer being taken straight to the airplane doors will make the customer journey much faster, and therefore get rid of a pain point many passengers have.

Fig. 15: VOA for biometrics

Through a competitive analysis and VOA charts, we then had a much better idea of what problems people find are worth solving. We developed product requirements that flowed from the VOAs as well.

Fig. 16: Product Requirements

Fig. 17: Concept Generation sketches

With these in mind, we individually came up with 10 concepts that could tackle not just the in-flight pain points but also the customer journey in general, since at this point, we did not have enough evidence to say we should ignore all the other aspects of the customer journey. In this context, the team participated in a concept generation session where we wrote ideas on a sheet of printer paper and after 5 mins discussed all together. Then, each paper was passed on to another teammate, who built up on the idea.

We ended up having about ~66 concepts, which we then reduced by removing ideas we all found unfeasible. We then had plenty of discussion around ideas that a few of us found unfeasible. Ideas we all agreed seemed good were prioritized. To categorize all the ideas, each one was color-coded. In the end, we found that many ideas that the team was on the fence about could be applied to the groups created from the 10 core concepts. Some ideas were related to the cabin as a whole, others were physical hardware, and most were supply chain concepts that could boost efficiency or the customer experience.

Fig. 18: Elimination of POGs

Fig. 19: Discussion of POGs

Fig. 20: Categorization of POGs

Fig. 21: More Categorization of POGs

As a team of two, we developed a prototyping canvas for a potential hardware solution in-flight - a capsule seat made to help induce sleep or a sense of exploration while flying.

Fig. 22: Prototyping Canvas for In-flight Seat Capsule

From earlier, it wasn’t quite clear whether the in-flight experience should be our primary focus. Furthermore, if our focus had been the in-flight experience, we wouldn’t know what element we should tackle. The team developed a survey to help us understand what pain points were the most significant, and more importantly, how willing people were willing to pay for upgrades.

Fig. 23: Survey Questions

The results from 18 responses are illustrated in the figure below. We found that the top 5 most important elements were in-flight, which helped us deliberate on our choice to focus on the in-flight experience.

Fig. 24: Survey data visualization

These data can be better understood with the following diagram.

Fig. 25: In-flight elements positioned on importance level, willingness to buy, and AI applicability

Although seat comfort ranked among the most important and willingness to pay, our team had a strong factor to take into account: our problem statement states that the concept we develop should be AI-based, meaning that the in-flight element should somehow be improved with data collection. Seat comfort is a hardware problem, and although it is fathomable to add weight sensors in seats to help passengers postures, sensors would have to be added to each seat, which increases the weight of the plane significantly. The diagram above then had 3 dimensions, the last one considering how applicable AI seems to be to the in-flight element. We chose to focus on increasing the food quality/experience in-flight since WiFi is out of our scope. Entertainment could be part of this product, but food quality ranks higher in willingness to pay. Therefore, we chose food quality as the in-flight element we focused on.

Our final concept then focused mainly on improving the passenger experience surrounding food. What could be a scalable product that increases customer satisfaction with AI while also being profitable to KLM? The concept we ended up with was this - allow customers to choose their food and drink options on-screen.

Fundamentally, rather than creating a machine learning model, which is mathematically rigorous, the most difficult problem with Machine Learning is collecting data, that is, digitizing it with sensors and/or user inputs. The following flow diagram describes the customer and company interaction with this on-screen interface.

Fig. 26: Food Preference Prediction model

This concept is very simple, but I believe it has a lot of thought behind it. This solution provides a way to collect data from passengers with already existing hardware - the thing that needs to be built in the software infrastructure. It provides an obvious way to digitize data, which is essential to build a machine learning model quickly. It also takes out the problem of forgetting to order a meal 24hrs in advance; a passenger can simply order once they board or as soon as the food cart makes its way over. With a prediction system, airline chefs can provide more options, as the number of data points collected per fleet test will increase the accuracy of what meal to bring in each flight and how many of each to stock in the plane. Given passenger input on-screen, passengers can be nudged to rate their food after they’ve finished it. These reviews can be fed back into the ML model to predict future likeability of specific menu items. This product gives passengers the ability to choose, on-demand, from more options than they had before. Free will (or the illusion thereof) leads to the sense of independence and power, which emotionally impacts the way each person feels during the flight. Although it may seem like a small effect in the customer journey, it’s the details that count a lot. For future work, assuming this work on fleet tests, a load sensor could be added to the cart to measure how much food waste each tray holds. With RFID tags, each tray could be mapped to each passenger. Then food waste could be mapped to each passenger and their reviews of the food. This could potentially predict future passenger food and drink preferences.

To end the semester, I redid some familiar figures to put in our showcase poster.

Fig. 27: Stakeholder Map v3

Fig. 28: Customer Base v3

Fig. 29: Competitive Analysis v2

I originally made a poster template for our project showcase.

Fig. 30: Original Poster Template

Link to our final poster.

However, we realized that a better way to describe not only our product but also our process, we could create a timeline of how our product opportunity gap changed over time as more user research was done.

Fig. 31: New timeline poster concept

For the showcase, we also developed a survey to correlate user input data to potential preferences people had.

Fig. 32: Survey question for preference prediction

To create the ML model, I had a script saved from a Kaggle competition, which we used to attempt prediction of food and drink preferences based on the user inputs.

# other importsimport numpy as np # linear algebraimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)import matplotlib.pyplot as pltimport os # Any results you write to the current directory are saved as output.
# ML importsfrom sklearn.neighbors import KDTreefrom sklearn.linear_model import LogisticRegressionfrom sklearn.neural_network import MLPClassifier
# read the datadata_test = pd.read_csv("train/train.csv")test_temp = pd.read_csv("test/test.csv")
# clean the datadata_test.drop('Description', axis=1).head()
# important importfrom sklearn.ensemble import GradientBoostingClassifier
# dividing datay_train = df["AdoptionSpeed"]x_train = df.drop(columns = ["AdoptionSpeed", "Name", "Description", "PetID", "RescuerID"])y_test = df["AdoptionSpeed"]x_test = df.drop(columns = ["AdoptionSpeed", "Name", "Description", "PetID", "RescuerID"])
# testing modelGBC = GradientBoostingClassifier(),y_train)score = GBC.score(x_test, y_test)print(score)

Fig. 33: Script for ML using GBC library in Python