NebulOS
Fair, green AI. For real.
Francesco Capuano (1), Matteo Matteotti (1), Marco Alberto Pellicanò (2), Davide Rinaldoni (2), Federica Rosellini (2)
(1): Politecnico di Torino, Turin, Italy; (2): Politecnico di Milano, Milan, Italy
Get in touch! {first.last}@asp-poli.it
Ciao! 🤖
Welcome to the extended abstract page of the Alta Scuola Politecnica 18th cycle NebulOS Project.
We are super excited to share with you the results of the work between July 2022 and September 2023 alongside the tech-startup Nebuly. With them, the team behind this project investigated the applicability of Neural Architecture Search for a social and environmental sustainable adoption of AI in society.
If you are curious to know more about what we have done... keep on reading! This extended abstract serves the scope of guiding you though the motivation behind our work and choices made along the way. You can also check the open-source project we released on GitHub at fracapuano/NebulOS, and check out the live-demo we prepared to share with the world the results of our work!
Or... you can go on and watch our 3-minutes video 👇
As the world witnesses an unprecedented explosion of Artificial Intelligence (AI) and its applications in everyday personal and professional life, many of the fundamentally challenging aspects of this ground-breaking technology start to become evident.
Advanced AI technologies such as conversation-oriented Large Language Models (LLMs) induced a paradigm shift in human-machine interaction, enabling humans to fully harness natural language as their means of control of computing resources. However, technologies such as LLMs have been obtained at costs that are astonishing both from a socio-political and environmental perspective. Peer-reviewed studies estimate the emissions associated with producing conversational LLMs such as ChatGPT to be equivalent to more than half a hundred Milan-NY commercial flights, with LLM instances like LlamaV2 being obtained at an even higher cost.
Among the various reasons that make AI development extremely energy-demanding, it emerges as crucial the nature itself of the process with which modern-AI-based technologies are developed. Modern AI-based technologies all employ computational models that aim at mimicking the human brain, i.e. Artificial Neural Networks (ANN, or DNN for Deep Artificial Neural Networks), uniquely identified by their structure, referred to as architecture. Due to the very limited theoretical understanding of the inner mechanisms of Deep Learning (i.e., the branch of AI making use of DNN), DNNs are mostly considered black-box models (i.e., models where the internal workings are not completely understood, and of which only the inputs and outputs are directly observable). This crucial aspect directly implies that there are generally no all-purpose guidelines about whether to prefer one model configuration over another, which inevitably leads to a trial-and-error approach, usually requiring the exhaustive training of each single candidate architecture among a set of candidates before finding the best one.Â
An overview of a trial and error approach. Here, architectures must undergo a resource-intensive step such as training before being evaluated.
In light of this, automating the procedure of architecture design and possibly tailoring it to be as energy-efficient as possible holds the promise of enabling a new generation of genuinely green AI models.
Moreover, the brittle nature of architecture design has led to a massive centralization of innovation capabilities in a handful of players that leverage a very privileged position in the AI market considering:
The barrier to entry represented by the high costs associated with building an AI-training infrastructure
Their capacity to attract and retain the best talent given favorable working conditions and brand reputation
Counting on computational resources orders of magnitude more potent than those typically available enables big-tech players to win the architecture-design race with a muscular yet practical approach. Should these factors be mitigated, it is reasonable to expect a much broader involvement of under-represented and underprivileged players - such as public research institutes or SME - in the AI scene, effectively leveling up the playing field for researchers and practitioners across diverse institutions.
Neural Architecture Search (NAS) is a Deep Learning sub-field that automates the process of selecting the most suitable architecture design for a particular task.
Rather than relying on human intuition or prior experience, NAS employs search algorithms to explore a vast space of possible architectures and determine the optimal configuration. The process is guided by using the model's performance on a given task as an objective function. Whilst NAS has proven to outperform human designers on multiple occasions, it typically requires the training of every single candidate architecture it evaluates - this extensive resource requirement makes the traditional NAS approach inaccessible or impractical for many researchers and practitioners.
To mitigate this challenge, "Training-Free NAS" (TF-NAS) has been introduced. Instead of fully training each candidate architecture, TF-NAS uses performance estimations which are computed in a fraction of the time, significantly reducing the computational overhead, and making the architecture search process faster and more resource-efficient.
Though sophisticated, popular TF-NAS presents some drawbacks, as it typically lacks insight into energy consumption, to the extent it might even disregard architectures marginally worse in performance but exponentially more energy-efficient. In other words, whilst it indubitably chops the computational toll associated with automated network design, TF-NAS may still opt for architectures unnecessarily too complex and resource-demanding.
An overview of the usual NAS process in which a search space of candidate architectures is explored according to a predefinite search strategy. Then, architectures are evaluated according to a search strategy.
This is why we are proud to introduce a novel approach based on a hardware-aware, training-free NAS method. Unlike traditional training-free NAS techniques that focus solely on performance proxies, our approach uniquely integrates the energy consumption for a specific hardware, producing tailored DNNs designs on a variety of different devices.Â
NebulOS is inherently user-centric, as it allows users to customize the weightage given to energy efficiency, which ensures that the final model aligns with the end-user's unique needs, resources, and preferences. Such a framework makes it possible to democratize the use of advanced AI solutions by extending their applicability to all those scenarios with limited computational resources.Â
This has the potential to empower researchers at regular institutions and SMEs to contribute to the AI race, bringing the frontier of AI innovation to the global scale needed to ensure this technology is maintained genuinely beneficial to all. In the pursuit of inclusive and sustainable AI development, and to foster and exploit a community-driven approach, NebulOS is completely open-source on GitHub (fracapuano/NebulOS), where users can effortlessly access, modify, propose changes, and expand upon the work presented.
We researched the best launch strategy for our open-source project by interviewing individual developers and small AI acceleration research groups, concluding on the value of clear communication for community development.Â
We crafted the README file as the cover of our GitHub repository, where we conveyed the project’s aim in a short and provided an explanation of what our algorithm can accomplish. With this, potential users can quickly ramp up on the latest features released and the upcoming ones, and learn about how they themselves could contribute to the project. Furthermore, we developed a web application serving the scope of accessibly showcasing our work manner our work, as well as attracting new potential contributors.
NebulOS stands as a testament to the idea that AI development does not have to come at the expense of socio-political stability or environmental sustainability. We believe our project paves the way for a more inclusive, efficient, and responsible era in AI, attaching to the unprecedented growth and challenges this exciting sector holds.Â
Through NebulOS, we aspire to ignite conversations around socially and environmentally sustainable AI practices, empowering a fair pursuit of ideas regardless of computational resources. The next big step for NebulOS is to build a lively community around this landmark, harnessing the power of teams and diverse perspectives. A welcoming environment ensures that users and contributors feel valued and engaged and that our research gains more credibility and better positioning within AI acceleration, by fostering an active and engaged community that brings brains together to cooperate and innovate. So... join the community!Â
If you are interested in joining, we would be glad to hear from you! You can find the contact author information on the footer of this webpage. Let's make AI Fair and Green, for real.