From Shallow to Deep Factorization