Customizable Data Acquisition Tool Based on Scrapy Framework

Contents

1. Clarify the data target you want to obtain (the website URL).

2. Customize data acquisition rules according to your own needs.

3. Customize the storage location of the acquired data.

Target
  AI researchers and engineers who need access to specific datasets.

What you will learn
  Parallel computing technology, Scrapy technology, and database technology.

Features/Appeals
  The tools developed have the following three appealing points:

1. Ability to custom extract multiple data types.

2. Providing real-time status display, which is convenient for users to view the data extraction status.

3. Batch acquisition in a short time.