Reason Why there are many datasets for NLP training

Introduction

Laymen explanation

If you see NLP training, you will find that some algorithm has used a particular data-set and other algorithm has used different data set. Why not the same data-set applicable for all algorithms? Won't it bring the uniformity? This document tries to address it.

Technical explanation

Natural language processing is a significant part of machine learning use cases, but it requires a lot of data and some deftly handled training. Machines are getting better at figuring out our complex human language. Each time someone trains a model to understand us, we are one step closer to integrating our machines more efficiently into our lives. Training data is an important part of this overall process.

Dataset properties which matter

• • Source of data - For example Quora corpus is having Question and Answer type data

• Total data corpse size

  • Input sequence size of ML model

Factors to consider while selecting data-set

• What kind of machine learning you want to build?

  • For example, if you want to build Chat-bot type NLP engine, then you need Quora corpus.

• Model tolerance size for input data

  • BERT model expects maximum 512 tokens long sequence. So, dataset should be pre-processed for taking care of this requirement of model

  • Reformer model allows longer input sequence (million). So, for this, dataset will be processed to have longer input sequence

• How much data is needed to reasonably approximate the unknown underlying mapping function from inputs to outputs?

  • Too little test data will result in an optimistic and high variance estimation of model performance.

• How much data is needed to reasonably estimate the performance of an approximate of the mapping function?

  • Corpus size should be good enough to divide into training and validation data

Where can you get this data?

This link has good list of repository with category

Reference

https://machinelearningmastery.com/impact-of-dataset-size-on-deep-learning-model-skill-and-performance-estimates/

https://medium.com/@ODSC/20-open-datasets-for-natural-language-processing-538fbfaf8e38