Bite detection

The universal eating monitor (UEM) is a table-embedded scale used to measure grams consumed over time while a person eats. It has been used in laboratory settings to test the effects of anorectic drugs and behavior manipulations such as slowing eating, and to study relationships between demographics and body weight. However, its use requires restricted conditions on the foods consumed and behaviors allowed during eating in order to simplify analysis of the scale data. Individual bites can be only measured when the only interaction with the scale is to carefully remove a single bite of food, consume it fully, and wait a minimum amount of time before the next bite. Other interactions are prohibited such as stirring and manipulating foods, retrieving or placing napkins or utensils on the scale, and in general anything that would change the scale weight that was not related to the consumption of an individual bite. This paper describes a new algorithm that can detect and measure the weight consumed of individual bites during unrestricted eating. The algorithm works by identifying periods of time in which the scale weight is stable and then analyzing the surrounding weight changes. The series of preceding and succeeding weight changes is compared against patterns for single food bites, food mass bites and drink bites to determine if a scale interaction is due to a bite or some other activity. The method was tested on 271 subjects, each eating a single meal in a cafeteria setting. A total of 24,101 bites were manually annotated in synchronized videos to establish ground truth as to the true, false and missed detections of bites. Our algorithm correctly detected and weighed approximately 39% of bites with approximately 1 false positive per 10 actual bites. The improvement compared to the UEM is approximately three times the number of true detections and a 90% reduction in the number of false positives. Finally, an analysis of bites that could not be weighed compared to those that could be weighed revealed no statistically significant difference. These results suggest that our algorithm could be used to conduct studies using a table scale outside of laboratory or clinical settings and with unrestricted eating behaviors.

When detecting bites using only a table-embedded scale, three types of bite are detectable through use of the scale. These three bite types:

1) Drink bites occur when a participants picks up their drink, consumes some liquid, and returns the remainder of their drink to the scale.

2) Food mass bites can occur when a participant picks up a food mass (such as a sandwich or a piece of pizza), takes a bite, and returns the remainder of the food mass to the scale.

3) Individual bites can occur when a participant picks up a single bite of food and consumes it fully, never returning any food to the scale.

These three bite types are shown below.

Depiction of a hand-drawn drink bite

Depiction of a hand-drawn food mass bite

Depiction of a hand-drawn single bite

The greatest challenge when detecting bites using a table-embedded scale in an unrestricted eating environment is removing scale interactions which do not result in bites. For an example of the challenging nature of this problem, see the figures below:

This image shows scale data collected during a meal that includes several individually detectable bites. Unstable scale periods are identified by labels A through M. In the example shown, label A indicates the time when the participant picks up a glass. Between labels A and B, the participant takes a drink from their beverage, and at label B, the participant returns the glass to the tray. The stable weight before label A and after label B indicate the weight of drink consumed. At label C, the participant picks up a piece of pizza. Between labels C and D, she takes a bite of pizza, and at label D, she returns it to the tray. Weights before C and after D indicate the weight of the food consumed. The same pattern as shown for labels C and D is shown again at labels E and F. At labels G, L, and M, the participant picks up a piece of salad and consumes it. In these scenarios, the bite weight can be found by taking the difference between the weights before and after the label. At labels H, I, J, and K, the participant is eating salad, but the scale does not have time to stabilize and allow the bite weights to be individually determined.

This image shows an example containing more challenging data. In this example, label A is an unstable region in which the participant picks up a French fry from their tray. This is the only individually detectable bite in the example. At label B, a large piece of pita bread is picked up but not immediately consumed. At labels C and D, bread is dipped in hummus and consumed in multiple bites while never being returned Fig. 8: Example of scale data in which the only individually detectable bite occurs at label A. to the scale. Labels C and D only correspond to the amount of hummus added to the bread when it is dipped, not the weight of the bite including bread and hummus. At label E, the participant picks up a sandwich and takes multiple bites before returning the sandwich to the tray at label F. None of the individual bites from the sandwich can be measured because the sandwich does not make contact with the scale during this time. A drink is picked up at label G, some is consumed, and the glass is returned at label H. This bite weight is not detectable because the scale is not able to stabilize between labels F and G.

Research to create and improve methods for bite detection are on-going.