Worms with Reinforcement Learning

This piece, by Onno Berkan, was published on 09/24/24.  The original text, by Matthew Hutson, is based on Li et al.’s 2024 study.

A recent study published in Nature Machine Intelligence had a Deep Reinforcement Learning agent interacting with worms in a petri dish to “achieve a goal” (that is to say, to find the delicious patches of E. Coli in a dish.) The agent was trained by a camera that recorded the location and orientation of every worm’s head and body; the agent received this information three times a second, and could even turn the lights on or off. The worms were “optogenetically engineered so certain neurons would become active or inactive in response to the light, sometimes prompting movement.”

The initial data was collected by randomly flashing lights for five hours– the worms’ responses were used to train the agent. The agent was then left to direct the worms, and it was found that worms guided by the agent reached their targets faster than worms that weren’t. Moreover, it was found that the agent wasn’t simply controlling the worm; rather, they were collaborating. If the agent directed the worm towards an obstacle, the worm would be able to override the command and crawl around said obstacle. This shows how, when striving towards the same goal, brains and machines could very well complement each other.

The lead author (Chenguang Li) stated that “one can easily see how [the results may] be extended to harder problems,” stating the possibility of humans using reinforcement learning agents to help them attain skills faster.

If you’re interested in this kind of stuff, I would suggest the work of David Eagleman, who focuses on sensory substitution, allowing for all kinds of super-human abilities.

Want to submit a piece? Or trying to write a piece and struggling? Check out the guides here!

Thank you for reading. Reminder: Byte Sized is open to everyone! Feel free to submit your piece. Please read the guides first though.

All submissions to berkan@usc.edu with the header “Byte Sized Submission” in Word Doc format please. Thank you!