EDGE AI

"It’s time for deep learning algorithms to come down from the cloud and get into your gadgets"

IEEE SPECTRUM, 19 Mar 2018 | 13:00 GMT

ExtFL = FL + Extreme Value Theory

Same reliability as CEN

GADMM

GADMM + Model Quantization

Challenge: Quantization errors propagate over nearest-based communications, hindering convergence

Solutions:

1) difference between each model parameter’s current and previous values is stochastically quantized, such that the expected quantization error becomes 0

2) Quantization bits are chosen such that the quantization step size is non-increasing over iterations

Federated Distillation

FD achieves 82% accuracy of FL under non-IID

Multimodal Split Learning

video1.mp4

Communication-Efficient Multimodal Split Learning for mmWave Received Power Prediction

MmWave Received Signal Prediction

Distributed Camera Setting w/ Manifold Mixup

Publications (not updated)

Journal Papers

T. Nishio, Y. Koda, J. Park, M. Bennis, and K. Doppler, “When wireless communications meet computer vision in beyond 5G,” IEEE Communications Standards Magazine, accepted.
J. Park, S. Samarakoon, A. Elgabli, J. Kim, M. Bennis, S.-L. Kim, M. Debbah, "Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications," in Proceedings of the IEEE, doi: 10.1109/JPROC.2021.3055679.
C. B. Issaid, A. Elgabli, J. Park, M. Bennis, M. Debbah, "Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM " in IEEE Transactions on Communications, major revision, Feb. 2021
A. Elgabli, J. Park, A. S. Bedi, M. Bennis, V. Aggarwal, "GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning," Journal of Machine Learning Research (JMLR), 2020.
A. Elgabli, J. Park, C. B. Issaid, M. Bennis, "Harnessing Wireless Channels for Scalable and Privacy-preserving FL" in IEEE Transactions on Communications, April 2021, to appear.
Y. Koda, J. Park, M. Bennis, K. Yamamoto, T. Nishio, M. Morikura, and K. Nakashima, “Communication-efficient multimodal split learning for mmWave received power prediction,” IEEE Communications Letters, 2020
P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, et al “Advances and Open Problems in Federated Learning,” https://arxiv.org/pdf/1912.04977, Dec. 2019.
A. Elgabli, J. Park, A. S. Bedi, C. B. Issaid, M. Bennis and V. Aggarwal, "Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning," in IEEE Transactions on Communications, doi: 10.1109/TCOMM.2020.3026398, https://arxiv.org/pdf/1910.10453.
M. Abdelaziz, S. Samarakoon, M. Bennis, W. Saad, “Ultra-Reliable and Low-Latency Vehicular Communication: An Active Learning Approach,” in IEEE Communications Letters, vol. 24, no. 2, pp. 367-370, Feb. 2020.
J. Park, S. Wang, A. Elgabli, S. Oh, E. Jeong, H. Cha, H Kim, S.-L. Kim, M. Bennis, "Distilling On-Device Intelligence at the Network Edge," submitted, Mar. 2020.
- J. Park, s. Samarakoon, M. Bennis, M. Debbah, "Wireless Network Intelligence at the edge,’’ in Proceedings of the IEEE, vol. 107, no. 11, pp. 2204-2239, Nov. 2019.
S. Samarakoon, M. Bennis, W. Saad, M. Debbah, "Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications," in IEEE Transactions on Communications, vol. 68, no. 2, pp. 1146-1159, Feb. 2020.
X. Chen, Z. Han, H. Zhang, G. Xue, Y. Xiao, and M. Bennis, “Wireless resource scheduling in virtualized radio access networks using stochastic learning,” IEEE Trans. Mobile Comput., vol. 17, no. 4, pp. 961-974, Apr. 2018.
X. Chen, C. Wu, M. Bennis, Z. Zhao, and Z. Han, ‘‘Learning to Entangle Radio Resources in Vehicular Communications: An Oblivious Game-Theoretic Perspective,’’ in IEEE Transactions on Vehicular Technology, vol. 68, no. 5, pp. 4262-4274, May 2019.
X. Chen, Z. Zhao, C. Wu, M. Bennis, H. Liu, Y. Ji, H. Zhang, "Multi-Tenant Cross-Slice Resource Orchestration: A Deep Reinforcement Learning Approach", in IEEE Journal on Selected Areas in Communications, vol. 37, no. 10, pp. 2377-2392, Oct. 2019.
X. Chen, H. Zhang, C. Wu, S. Mao, Y. Ji, M. Bennis, "Optimized Computation Offloading Performance in Virtual Edge Computing Systems via Deep Reinforcement Learning", IEEE IoT Journal, 2018.

Conference Papers

Y. Koda, J. Park, M. Bennis, P. Vepakomma, and R. Raskar, “AirMixML: Over-the-Air Data Mixup for Inherently Privacy-Preserving Edge Machine Learning,” (submitted to IEEE GLOBECOM 2021).
Y. Koda, J. Park, M. Bennis, K. Yamamoto, T. Nishio, and M. Morikura, “One Pixel Image and RF Signal Based Split Learning for mmWave Received Power Prediction,” Proc. ACM CoNEXT Poster, Orland, Florida, Dec. 9-12, 2019.
A. Elgabli, J. Park, S. Ahmed, and M. Bennis, "L-FGADMM: Layer-Wise Federated Group ADMM for Communication Efficient Decentralized Deep Learning," in proc. of the IEEE WCNC 2020, Seoul, Korea.
A. Elgabli, J. Park, A. S. Bedi, M. Bennis, V. Aggarwal, "Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning," in proc. of the ICASSP 2020, invited paper.
E Jeong, S Oh, J Park, H Kim, M Bennis, SL Kim, "Multi-hop federated private data augmentation with sample compression" IJCAI 2019, Macau.
H Cha, J Park, H Kim, SL Kim, M Bennis, "Federated reinforcement distillation with proxy experience memory," IJCAI 2019, Macau.
E. Jeong, S. Oh, H. Kim, J. Park, M. Bennis, S.-L. Kim, "Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data," NIPS Workshop, Montreal, Canada, 2018.
S. Samarakoon, M. Bennis, W. Saad, M. Debbah, "Federated Learning for Ultra-Reliable Low-Latency V2V Communication," in proc. of IEEE Globecom 2018, Abu-Dhabi.
X. Chen, H. Zhang, C. Wu, S. Mao, Y. Ji, and M. Bennis, “Performance optimization in mobile-edge computing via deep reinforcement learning,” in Proc. VTC-Fall, Chicago, IL, Aug. 2018.