Reliability and security of DNNs
In recent years, DNNs (Deep Neural Networks) have become very popular both in academic and practical fields. DNN applications are being employed even in safety-critical areas, such as autonomous driving, surveillance, or healthcare. Therefore, the study of reliability and security of DNN applications is imperative.
DNN Reliability
DNNs are commonly known to be inherently resilient to errors. The large number of variables provide natural redundancy, and each layer presents opportunities for error masking effects. Especially in classification models, small deviations in calculations usually do not result in classification misses. However, recent studies report that even single-bit errors are sufficient to cause mispredictions, and that FIT rates of DNN accelerators exceed the ISO 26262 standards. To make things worse, error rates are continually increasing, while traditional error protection methods are unsuitable to most DNN applications.
Misprediction examples caused by single-bit errors. Red caption above each street sign indicates the misinterpreted label output by the model. (SanityCheck)
DNN Security
DNNs applications are also susceptible to various kinds of attacks. One form of attack is the adversarial attack, which adds noise to the input data. The noise is imperceptible to the human eye but can trick the model into misclassifying the input data. The road signs in Figure 1 can serve as examples of such attacks. It is also easy for attackers to steal information from DNNs provided online in the form of APIs. By collecting data from repeated queries, the attacker may be able to recover some of the training data, or even the entire model. In addition, the stolen information could be used for subsequent attacks. For example, an attacker could forge adversarial examples using the information from the target DNN.
Given the victim’s name and access to a facial recognition system API, an attacker may recover an image of the victim (left) that resembles an image in the training set (right). (Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures)
Research goals
Find efficient error mitigation techniques for DNN applications
Find effective defenses against DNN security attacks
Examine similarities between the two to find universal protection algorithms
Reading list
Li, G., Hari, S. K. S., Sullivan, M., Tsai, T., Pattabiraman, K., Emer, J., & Keckler, S. W. (2017, November). Understanding error propagation in deep learning neural network (DNN) accelerators and applications. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 1-12).
Liu, Y., Wei, L., Luo, B., & Xu, Q. (2017, November). Fault injection attack on deep neural network. In 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) (pp. 131-138). IEEE.
Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., & Ristenpart, T. (2016). Stealing machine learning models via prediction APIs. In 25th USENIX Security Symposium (USENIX Security 16) (pp. 601-618).
Z. Chen, G. Li and K. Pattabiraman, A Low-cost Fault Corrector for Deep Neural Networks through Range Restriction 2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), Taipei, Taiwan, pp. 1-13.
Madry, Aleksander Makelov, Aleksandar Schmidt, Ludwig Tsipras, Dimitris Vladu, Adrian. (2018). Towards Deep Learning Models Resistant to Adversarial Attacks. International Conference on Learning Representations (ICLR)
Yang, Puyudi Chen, Jianbo Hsieh, Cho-Jui Wang, Jane-Ling & Jordan, Michael. (2020). ML-LOO: Detecting Adversarial Examples with Feature Attribution. (AAAI) Conference on Artificial Intelligence.
E. Ozen and A. Orailoglu, "Sanity-Check: Boosting the Reliability of Safety-Critical Deep Neural Network Applications," 2019 IEEE 28th Asian Test Symposium (ATS), Kolkata, India, pp. 7-75.