Projects

ELT (EXTRACT LOAD TRANSFORM

A project to demonstrate some of the main concepts behind data engineering using open-source tools such as Airflow, DBT, great expectations, PostgreSQL, and Redash in order to perform an end-to-end ELT data pipeline.

The main objective of this project is to help the organization obtain critical intelligence based on public and private data they collect and organize.

This is going to be achieved by deploying an end-to-end ELT data pipeline that will extract the required data from several sources of data generation tools, then loading it into a data warehouse (single source of truth) in order to later transform the obtained data that can serve the needs of several people in the organization's staff like data scientists, machine learning engineers, business and data analysts as well as several reporting staff members.

KAFKA CLUSTERS

An ETL data pipeline to collect and extract vocal data, transform and load it to an S3 bucket using Kafka clusters, Airflow, and spark for a text-t0-speech conversion project.

This project recognizes the value of large data sets for speech-to-text and sees the opportunity that there are many text corpora for Amharic and Swahili languages, want to design and build a robust, large-scale, fault-tolerant, highly available Kafka clusters that can be used to post a sentence and receive an audio file.

Producing a tool that can be deployed to process posting and receiving text and audio files from and into a Kafka topic, apply transformation in a distributed manner, and load it into an S3 bucket in a suitable format to train a speech-to-text model would do the required job.

BACKTESTING

Cryptocurrency and stock trading engineering: A scalable back-testing infrastructure and a reliable, large-scale trading data pipeline.

In this project, the main objective is to make it simple for everyone to enter the world of cryptocurrencies and the general stock market trade. It also wants to give investors a reliable source of investment while lowering the risk associated with trading cryptocurrencies.

Although the past performance of any financial market is never a reliable indicator of the future, it is important to run backtests that simulate current and past particular situations as well as their trend over time. Having a clear understanding of the financial system, and stock market trading, and recognizing the complex data engineering systems involved in the crypto and general stock market trading systems are essential.

Time Series Analysis

Time series data set sales prediction.

The aim of this project is to predict the sales six weeks ahead across all the stores of the Rossman Pharmaceutical company using Machine and Deep Learning. The different factors affecting sales are promotions, competitions, school-state holidays, seasonality, and locality.

Building and serving an end-to-end product that delivers this prediction to analysts in the finance team.

The Algorand Blockchain

Web3 is the third generation of the web, or the commonly known and referred to communication network that we all love and call the internet. But Web3 is much more than the commonly known communication network and its uses. Web3 technology is inherently about the user-controlled internet. It is being achieved by a growing stack of decentralized technologies, such as blockchains, smart contracts, oracles, crypto wallets, storage networks, and more.

In this project, the main objective is to build an end-to-end Web3 decentralized application on the Algorand Blockchain that will help its client generate and distribute Non-Fungible Tokens (NFTs) as certificates that will represent the successful completion of a task or project to its customers, and allow its customers that are holding these NFTs to interact with a smart contract to perform pre-defined actions.

Defund by location

The defund by location smart contract is aimed to be used when one party, for example, an employer, agrees to pay another party, for example, an employee, for being present in a certain geographic area for a certain duration. The employee’s phone sends its GPS location to a smart contract at a certain interval. Based on the pre-agreed contract codified in an Ethereum smart contract, a cryptocurrency payment is executed when all the agreed conditions are met. If at any point, the GPS sensor indicates that an employee is outside the range of the agreed GPS area, the contract state will be updated to indicate that it is out of compliance.