Deepak's homepage

Understanding of ML pipeline in data science

Introduction

Laymen explanation

A machine learning pipeline is used to help automate machine learningworkflows. If you line to understand its basics, then this document helps.

Technical explanation

Machine learning pipeline is a means of automating the machine learning workflow by enabling data to be transformed and correlated into a model that can then be analyzed to achieve outputs. This type of ML pipeline makes the process of inputting data into the ML model fully automated.

Examples

Sentiment analysis pipeline

OCR pipeline

Pipeline in Kubernetes

The Kubeflow project is dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable and scalable.

TFX is another such project

Pipeline optimisation

Focussing on most important component helps to improve the outcome(for example, accuracy rate). So, We need to allocate resources judiciously.

Ceiling analysis helps to decide which component needs how much focus. Refer here to understand this.

Pipeline Additional automation features

Data lineage

Reference

https://algorithmia.com/blog/ml-pipeline

https://images.app.goo.gl/Y7XCcuUn1hDRQBBo6

https://www.kubeflow.org

https://nanonets.com/blog/receipt-ocr/

https://rossbulat.medium.com/ceiling-analysis-in-deep-learning-and-software-development-8bc41e59364a

https://www.youtube.com/watch?v=8fx1nSFQqkk

https://coursera.org/share/67590d45c20e11f49b50c4e970b6924d

https://images.app.goo.gl/jV5mGukTTy677a8W6

https://images.app.goo.gl/QxNheV1aRfQnE2H19

Page updated

Google Sites

Report abuse