CMSC 191: Introduction to Neural Computing
Mathematical Foundations of Neural Computation
In this topic, we’ll explore the mathematical ideas that make neural networks both precise and powerful tools. You’ll discover how an artificial neuron can be seen as a function that takes weighted inputs and transforms them through a special process known as non-linear activation. We’ll also connect these ideas to the core concepts of vector algebra and calculus, showing how these mathematical tools give us the ability to train models, optimize them, and scale them for real-world applications.
By the end of this journey, we’ll dive into activation functions—the "magic" that brings neurons to life! These functions turn simple linear models into dynamic systems that can learn and represent complex patterns. You’ll see how math not only explains how neural networks work, but also unlocks their potential to solve big, real-world problems.
Represent artificial neurons as mathematical functions using weighted sums, bias, and activation.
Explain how vector algebra scales neural computation from single neurons to layers and networks.
Describe how calculus enables learning through weight adjustments and gradient computation.
Compare different activation functions and explain their mathematical and computational properties.
Evaluate how non-linear activation functions expand the representational power of neural networks.
How does representing a neuron as a mathematical function clarify its computational role?
Why are non-linear activation functions essential for deep learning architectures?
In what ways do calculus and gradient computation translate abstract “learning” into mathematical operations?
Mathematical Foundations of Neural Computation* (topic handout)
Mathematics: The Language of Neurons
Representing Neurons as Mathematical Functions
The Neuron as a Computational Engine
Unlocking Computational Power
Activation and Transfer Functions
Shaping the Signal: An Activation Zoo
The Activation's Impact on Training
Equations that Learn
The semester at a glance:
Mathematical Foundations . . .