Compressing Convolutional Neural Networks (CNNs)

Convolutional Neural Networks (CNNs) have become the de-facto tool for visual feature extraction. This success can principally be attributed to the capability of these models to generalize to a wide gamut of problems. However, these networks tend to be prohibitively memory intensive and heavily over-parameterized.

In this work, we propose a two-step algorithm to make these networks leaner while retaining their classification performance. These two steps are discussed briefly below:

• Filter Pruning

• Coreset Decomposition of Filters

As illustrated in the figure, our first step consists of pruning out the unnecessary/irrelevant filter of the network. This is determined from the activation patterns of the filter over the samples of the training set. Post the pruning, we proceed to represent the filters as a coreset. This consists of representing the set of filters as a combination of a smaller, basis set of filters. Thus the network can now be parameterized just by this smaller basis set, rather than the original set of filters.

Code:

Link

Publication:

• M. Chatterjee*, A. Dubey*, N. Ahuja, “Coreset-Based Neural Network Compression”, European Conference on Computer Vision 2018 (ECCV 2018). [* - Equal Contribution]