Despite the superior performance of deep neural networks, they are not efficient for regressing on function spaces. In particular, neural networks are unable to encode function inputs compactly as each node encodes just a real value. We propose a novel idea: to encode an entire function in a single network node. We design a network that encodes and propagates functions in single nodes for the distribution regression task. Our proposed Distribution Regression Network achieves higher prediction accuracies and more compact models compared to traditional neural networks.

We use distribution classifiers in a defense method against adversarial attacks in convolutional neural networks by integrating with existing transformation-based defenses. Exploiting the fact that the transformation methods are stochastic, our method samples a population of transformed images and performs the final classification on distributions of softmax probabilities, showing improvement on both clean and adversarial images. On the CIFAR10 dataset, the improvements over baseline majority voting are 6.4% and 3.6% on the clean and adversarial images respectively.

Current state-of-the-art methods for anomaly detection on complex high-dimensional data are based on the generative adversarial network (GAN). However, the traditional GAN loss is not directly aligned with the anomaly detection objective: it encourages the distribution of the generated samples to overlap with the real data. In this paper, we propose simple modifications to the GAN loss such that the generated samples lie at the boundary of the real data distribution. With our modified GAN loss, our anomaly detection method, called Fence GAN (FGAN), directly uses the discriminator score as an anomaly threshold.