Student Demo Projects

Below I list selected demo projects completed by my bachelor and master students at TU Graz. Most projects were developed as part of the Mobile Computing Lab and proposed and realized by the students. More projects and updates are on the way ... stay tuned!

COFFECTLY
Idea and app implementation by Theo Gasteiger, 2023

Shot timing is one of the most important variables in the extraction of a great espresso. Extracting an espresso too fast leads to a light body and results in a high acidity and unpleasant taste. On the contrary, a too high extraction time leads to a heavy body and increased amount of bitter tasting notes. The Istituto Nazionale Espresso Italiano, therefore, recommends an extraction time of 25 seconds (+/-) 5 seconds. However, timing each extraction is cumbersome and time consuming. COFFECTLY is a novel iOS application that enables automatic shot timing as well as coffee brand classification, based on acceleration measurements. COFFECTLY exploits the three-axis accelerometer in iOS devices to not only detect the state of the espresso machine (pump on/off) but also to extract information about the coffee, used in the current extraction. With COFFECTLY, we are the first to provide automatic shot timing as well as coffee brand detection solely based on accelerometer measurements.

Classification: Water

Classification: Hardy

Classification: Martella

Pump state detection

Emmission Tracking App
Idea and app implementation by Adrian Opheim, 2023

EmTrack is a smartphone application that enables day-to-day tracking and recording of CO2-emmissions estimated based on the user activity and its duration. The project is limited to only estimating the user’s emission coming from the type of transportation. The app would detect whether the user is walking, biking, driving a car or taking a bus etc. and given the time spent in each transport mode estimate how much CO2 is emitted in the atmosphere. The transportation mode detection is accomplished by recording sensor data on an Android phone and then classifying user activity using a pre-trained LSTM model locally on the phone. The app provides information about the current user activity and the statistics of the part activities over the whole observation period. The app is lightweight and does not use the power-hungry GPS.

Deep Inference on STM32 Discovery Kit
By Peter Prodinger, 2022

In this demonstration, the student utilized the STM32F7 Discovery Kit, equipped with 340 kB of RAM, to investigate the influence of various optimization techniques on the performance of a machine learning model. This involved experimenting with model optimization, quantization, and compiler flags. Both convolutional and fully connected models were examined during the experimentation process. Detailed observations were made regarding the model's accuracy in predicting the first and second most probable classes, as well as instances where the model made errors. To enhance the visual representation of the demo, the large display portion of the platform was utilized. This allowed for the simultaneous display of the input image, which was a subset of the MNIST test set, and the model's performance statistics. This visual representation provided valuable insights into the model's behavior and performance.

Magic Wand Demo on Arduino Nano 33 Sense
By Rahim Entezari and David Mihola, 2021-2022

This demo was developed as a component of the Embedded Machine Learning course. It showcases a deep learning model deployed on an Arduino Nano 33 Sense device with limited resources, specifically 256 kB of RAM. The model's purpose is to accurately identify spells performed by a wizard, utilizing a set of wand motions outlined in the "Beginner's guide to wand motions". The Arduino platform is equipped with an IMU sensor, and the sensor's readings serve as input to the spell recognition model. It is important to note that the proficiency of spell execution may vary among magicians. Designing and implementing a robust magic wand, while taking resource constraints into account is an art.

by Rahim, 2021

by David, 2022

BlindApp: Free path for visually impaired people
Idea and app implementation by Philip Samer, 2022

The app is designed for visually impaired people, to help them move around without pumping into objects. The app uses the main camera of the smartphone to estimate the distance to objects and notify the user if something is in front of him. The phone is placed in the front pocket of the shirt or carried around in a hand (which makes it easier to explore different directions). Headphones can be used to communicate with the user and give him an audio signal in case an obstacle is detected. To estimate the distance to the surrounding objects, the app relies on a recently published method by W. Yin et. al., "Learning to Recover 3D Scene Shape from a Single Image" to recover accurate 3D scene shape.

PlantApp: Garden plant identification and diagnostics
Idea and app implementation by Alexander Palmisano, 2022

Many people have plants in their homes or gardens. Sometimes it can be challenging to identify a plant and determine whether it is healthy or has a disease. PlantApp developed by Alexander Palmisano is able to identify 14 of the most common plants (fruit and vegetable plants) that people typically have in their homes and gardens and diagnose whether a plant has a disease or not. The dataset used to train a model is the Kaggle's New Plant Diseases Dataset. A deep learning model classifies images of leaves and instantly reports the result. The model has been quantized in order to be used on a mobile phone. PlantApp also lets the user quickly scan their plants and upload the data to a cloud service in order to retrain and improve the model.

SpeechMood: Track your expressiveness
Idea and app implementation by Maximilian Nothnagel, 2022

SpeechMood is a mobile app for Android that enables a user to record and instantly process their voice, for instance during a speech or presentation, and be presented with a prediction of the conveyed mood category based on the input audio signal. The motivation originates from the interest in the capabilities of detecting a person’s emotional state based on objectively perceived factors, such as frequencies in speech, pulse rate or other factors. SpeechMood relies on a deep network pre-trained on RAVDESS and Berlin EmoDB datasets. The model works well on test data, but several tricks had to be used to also obtain good performance on live data, as can be observed in the video. The idea of the app and the implementation are by Maximilian Nothnagel.

Activity Recognition with Transfer Learning
by Julian Rudolf and Christopher Hinterer, Jakob Soukup, Peter Prodinger, Dietmar Malli, 2020-2022

In this project, the students developed mobile apps to track users' activities using various methods. In the first step, a K-Nearest-Neighbour (KNN) classifier was trained to solve the problem. Then, the same task was approached by training a deep learning model (DNN), in some realizations pretrained on a well-known WISDM data set. Finally, we used on-device transfer learning (TL) to adapt the model to a new phone location and orientation by re-training it with very little data. In the videos below MCL students demonstrate the performance of their apps. Nice!

by Julian and Christopher, 2022

by Peter, 2020

by Jakob, 2022

by Dietmar, 2020

Quantle: Filler Word Detection and Counting
by Rahim Entezari and Franz Papst, 2020

The ability to speak in public is a skill which is gaining more and more importance. Disfluencies are quite common in everyday speech. They give a speaker time to think about what to say next and help to structure spoken language. Disfluencies usually manifest themselves as utterances of filler words like “uhm”, “well” or “like”. While moderate usage of such fillers helps to sound natural, excessive use of them or long pauses indicate that the speaker lacks confidence. In this project Rahim Entezari and Franz Papst detect and count vocal fillers on a mobile phone with help of a deep neural network. They plan to add the model to Quantle to give speakers a tool to improve their public speaking skills when rehearsing a presentation!

Wireless Drum Sticks
Idea and app implementation by Markus-Philipp Gherman and Fabian Moik, 2019

The project implements wireless drum sticks without surface reference on conventional iPhones. The app works as follows: The stick hit detection without surface reference is based on the measurements provided by the accelerometer sensor integrated into the phone and is determined by a trained long short-term memory (LSTM) network running locally on the phone. Also the location of the hit is determined by the same LSTM and is attributed to one of the six drums on the screen. The LSTM network was trained on a manually gathered data set comprising up to 700 hits per drum location performed by multiple drum players. Detection is almost 100% accurate (ha! professional drummers get better accuracy results than beginners!). The initial location of the phone is used to calibrate to the middle of a virtual screen. Once a hit and its strength are detected, the audio is played by the phone. One can even use two phones simultaneously! Watch awesome videos below played and explained by the students. Professional drummers found the app impressively responsive and usable!

Indoor Localization using Particle Filter
2019

Five teams in SS 2019 implemented indoor localization using particle filter algorithm and performed indoor localization at ITI in the Inffeldgasse 16. It's impressive how many optimizations were suggested by the students to improve their app performance, from optimized computation of wall positions to speed up computation to various ways of resampling particles. Thank you guys for your efforts and enthusiasm!