Automated Product Extraction Dashboard

In association with Monstyr

Problem Overview

Introduction to Monstyr

Monstyr is a lifestyle mobile app that collates and shares a wide range of promotions, deals and bargains from stores around Singapore.

These include the promotional deals from supermarkets and convenience stores, which are frequently published as posters in merchants' stores or websites.

User Persona 1

Kishen

User Persona 2

Ririn

NTUC Promotional Poster from 20 June

Pain Point

As each poster can consist of as many as 30 products, members of Monstyr's data extraction team have to manually copy each product's information and image from the merchant's poster into their database.

There exists no automated solution for this manual, repetitive process.

Current Implementation

Our Solution

At the click of a button, our web app allows the Monstyr Data Extraction Team to upload a merchant's poster and receive the cropped images of each product in the poster.

They can then edit the product images and export them to the Monstyr database.

Product Video

Feature 1:

One-click Image Uploading

At the click of a button, the user can upload a merchant's poster with multiple products.

Feature 2:

Extracting and Cropping Each Product

The poster is uploaded to Google Cloud; a square image of each product is extracted using AutoML Vision. While this happens in the backend, the user is kept informed with the loading screen that displays the current task.

Feature 3:

Editing Images to Preference

The user is greeted with a dashboard of square product images. If images are not cropped satisfactorily, a simple click brings the user to a page where they can shift and resize the product image.

Feature 4:

Exporting Images with a Click

Export all product images with a simple click of the button at the bottom of the page. A ZIP file will be exported, ready to be imported to Monstyr's database.

Design Journey, Early Iterations

We knew early on in our design journey that engineering the machine learning model for extracting each item's image would pose the greatest technical challenge in this project.

These are the results from the early iterations of the image extraction, using AutoML API.

Results from an early ML model.

Testing the ML model on images that were not part of the training dataset.

Software Architecture

Microservices

RESTful API

Database Design

Services Used

Google Cloud Platform was used as the database for the uploaded poster images, as well as the individual product images.
AutoML Vision was used for its Image Classification capabilities, to extract each product image from posters of up to 30 products.
Heroku was used to deploy the web application