AutoDL: Automated Tools for Fast Development of Deep Learning Networks and Accelerators

Comprising projects:

• Auto-NBA [ICML 2021, GitHub]:

  • An automated framework that jointly searches for the Networks, Bitwidths, and Accelerators

  • Efficiently localize the optimal design within the proposed huge joint design space for each target dataset and accelerator specification.

  • Auto-NBA generated networks and accelerators consistently outperform state-of-the-art designs in terms of search time, task accuracy, and accelerator efficiency.

• HW-NAS-Bench [ICLR 2021 Spotlight, GitHub]:

  • The first public HW-NAS dataset aiming to democratize HW-NAS research to non-hardware experts, and to facilitate a unified benchmark comprising SOTA NAS search spaces of NAS-Bench-201 and FBNet, to make HW-NAS research more reproducible and accessible.

  • HW-NAS-Bench enhances the above search spaces by providing measured/estimated hardware-cost of all the 46, 875 (NAS-Bench-201) and 1021 (FBNet) architectures on six hardware devices spanning all three categories (i.e., commercial edge devices, FPGA, and ASIC), which are primarily targeted by HW-NAS work.

  • We demonstrate exemplary use cases to show that HW-NAS-Bench allows non-hardware experts to perform HW-NAS by simply querying our pre-measured dataset and verifying that dedicated device-specific HW-NAS can indeed lead to optimal accuracy-cost trade-offs.

• DNN-Chip Predictor [ICASSP 2020, GitHub]:

  • An analytical tool which accurately predicts various performance metrics of DNN accelerators such as energy, throughput, and latency to help facilitate the fast design space exploration and optimization before actual ASIC/FPGA implementation.

  • DNN-Chip Predictor enables fast and effective DNN accelerator development and is validated using different DNN models and accelerator designs (i.e., architectures, dataflows, etc).