Our project will be implemented as a web application that is better for research purposes. We are going to use the FReMP stack (Flask, ReactJS, MongoDB, Python). Accordingly, we are using ReactJS for the front-end development to build interactive UIs, Flask REST APIs for handling CRUD operations, and MongoDB for the database. Since Python can easily handle tasks of data analysis and web scraping, we get the ability to statistically analyze the datasets and display them nicely on the client side using ReactJS. Finally, we will deploy the website with Heroku.
List of new features:
Improve UI of the current version of the website
Search for datasets through names and types
Add a dynamic contact form to submit dataset and send an email, instead of directing straight to email
Download dataset with a single button and store in the database (for future use, implement once we have access to actual datasets)
Include dataset information preview
View similar datasets and their details when looking at a particular dataset
Responsive website (compatible for bigger screens)
Website deployed on Heroku
Include admin system for managing the datasets and contact databases
Possible challenges we will have during implementation:
We need to properly design the schema in MongoDB based on the requirements. Unlike relational databases which normalize data into tables, in MongoDB, the data is modeled in JSON format as we want. We need to carefully design what data to store and how to structure them in order to provide good query performance.
Implementation of our new design for the front end. We try to develop a new user-friendly interface with additional features and implement it using React.
Challenges with Heroku deployment since we have different dependencies and there are often breaks in the process of deployment that need additional fixing.