AI That Can Explain Itself

This piece, by Onno Berkan, was published on 11/12/24. The original text, by Joseph Bakarji, was published by Nature Computational Science on 02/19/24.

This American University of Beirut study looks into how deep learning and artificial intelligence have become incredibly powerful tools in science and engineering, but they often work like mysterious black boxes– we can see what goes in and what comes out, but we don't understand how they make their decisions. This lack of transparency is particularly concerning in critical areas like healthcare and self-driving cars, where we need to be absolutely sure about how decisions are being made.

To address this challenge, researchers developed an innovative approach called "deep distilling," which uses special neural networks called Essence Neural Networks (ENNs). Think of it as teaching a computer to not only solve problems but also explain its reasoning in a way that humans can understand.

This approach is special because instead of just crunching numbers like traditional AI, ENNs use a symbolic approach– meaning they work with logical rules and patterns that are closer to how humans think. The system has two main parts: the ENN itself and a "condenser" that transforms the network's complex calculations into clear, executable computer code.

To demonstrate how well this works, the researchers showed that their system could figure out the rules of Conway's Game of Life (a famous computer simulation) just by watching it in action. This is particularly impressive because it shows that the system can discover underlying patterns and rules in complex systems, similar to how scientists try to understand natural phenomena.

The implications of this work go beyond just making better AI systems. As AI becomes more involved in our daily lives and society, it's crucial that these systems operate in ways that align with human values and understanding. By making AI more transparent and interpretable, this research helps build trust between humans and machines, which is essential for sensitive applications.

This research represents a significant step forward in making AI systems more understandable and trustworthy, potentially paving the way for better collaboration between humans and machines in critical applications where transparency is essential.

Want to submit a piece? Or trying to write a piece and struggling? Check out the guides here!

Thank you for reading. Reminder: Byte Sized is open to everyone! Feel free to submit your piece. Please read the guides first though.

All submissions to berkan@usc.edu with the header “Byte Sized Submission” in Word Doc format please. Thank you!