Home‎ > ‎


WACV-2018: Tutorial

When Blockchain Meets Computer Vision: Opportunities and Challenges

Presenters: Karthik Nandakumar, Sharathchandra Pankanti, Nalini Ratha

It is widely acknowledged that blockchain is foundational technology that will revolutionize the way transactions are conceived, executed, managed, and monetized [1]. This is further corroborated by recent publication special issues (e.g., [2]), keynote talks (e.g., [3]), mainstream news headlines (e.g., [4]). While the technological benefits of the blockchain infrastructure are imminent with significant momentum gained by investment from numerous corporations including IBM, the underlying technological problems need significant attention from researchers as acknowledged by recent IEEE initiatives (e.g., [5]). Of specific interest to computer vision researchers and application developers is the tremendous opportunity to make a connection to these emerging infrastructure capabilities and realize how their skills can be leveraged to make an impact by marrying computer vision/artificial intelligence and blockchain technologies. As the camera based infrastructure is becoming ubiquitous (over 4 billion mobile phones, millions of public surveillance cameras, and increasing presence of egocentric bodycams in professional environments)  and compute power is becoming pervasively available,  it is increasingly clear that the business world is going to look to camera as a default sensor and camera-based analytics as a de facto information channel to improve the integrity of transactions from various diverse perspectives. For instance, many complex practical challenges such as usability (e.g., camera-based intuitive stakeholder identity management [6, 7]), (ii) compliance with regulations (e.g., provenance of physical transport payloads, compliance of computer vision systems with local laws), (iii) integrity of transaction artifacts (e.g., forensic camera evidence [8, 9]), and (iv) protecting privacy of the sensitive information (e.g., video redaction, sharing of insurance claims or patient data [10, 11]) can be effectively addressed using blockchain technology.


The proposed tutorial is aimed as a gentle introduction to the broader world of distributed transaction environment, and specifically blockchain technology for computer vision researchers. The tutorial will first introduce the blockchain technical concepts [12, 13] and capabilities in the context of real computer vision applications. We will also present the emerging key developments of interest in both computer vision application development and distributed transaction infrastructure [14, 15]. The tutorial will subsequently review the real application scenarios, where blockchain has tremendous potential to accelerate its use as an enterprise transaction infrastructure and highlight key challenges. For instance, one of the key take-aways from the tutorial would be an understanding of the challenges underlying how the distributed and decentralized architecture of blockchain need be synergistically harnessed for providing hard-core functionality requirements of applications involving analytics of unstructured data. The challenges will be concretely couched in real mainstream use-cases such video surveillance or recent mainstream security news [16] and would cover many important aspects such meaningful privacy of end-users, scalability & cost-effectiveness, and user friendliness. Solving such problems requires multi-disciplinary research effort at the intersection of blockchain, artificial intelligence, and user behavior modeling.


In summary, the tutorial will be an opportunity for both budding and mature computer vision researchers to expand their sphere of influence by understanding the potential benefits of integrating computer vision and distributed transaction management infrastructure.  


Tutorial Outline:


Computer Vision Applications  (45 mins)

  • Illustrative Applications
  • Requirements
  • Gaps

Blockchain   (60 mins)

  • Motivation
  • Features
  • Architecture
  • Algorithms

Blockchain Enablement of Computer Vision Applications (60 mins)

  • ID management
  • Transaction management
  • Integrity management
  • Compliance management


1.      M. Iansiti and K. R. Lakhani, “The Truth About Blockchain”, Harvard Business Review, Jan 2017

2.      IEEE Spectrum special report: Blockchain World, Oct 2017

3.      https://project.inria.fr/wifs2017/program/keynotes/

4.      How the blockchain is helping stop the spread of conflict diamonds , Wired Magazine,  http://www.wired.co.uk/article/blockchain-conflict-diamonds-everledger

5.      https://blockchain.ieee.org/

6.      J. S.  Hammudoglu et al., Portable Trust: biometric-based authentication and blockchain storage for self-sovereign identity systems, https://arxiv.org/abs/1706.03744, Jun 2017

7.      N. Buchmann, C. Rathgeb, H. Baier, C. Busch and M. Margraf, "Enhancing Breeder Document Long-Term Security Using Blockchain Technology," IEEE 41st Annual Computer Software and Applications Conference (COMPSAC), Turin, 2017, pp. 744-748.

8.      Gipp, B., J. Kosti, and C. Breitinger. "Securing Video Integrity Using Decentralized Trusted Timestamping on the Blockchain." Proceedings of the 10th Mediterranean Conference on Information Systems (MCIS). 2016.

9.      D. Bhowmik and T. Feng, "The multimedia blockchain: A distributed and tamper-proof media transaction framework," 22nd International Conference on Digital Signal Processing (DSP), London, 2017, pp. 1-5.

10.   Kuo, T. T., C. N. Hsu, and L. Ohno-Machado. "ModelChain: Decentralized Privacy-Preserving Healthcare Predictive Modeling Framework on Private Blockchain Networks." ONC/NIST Blockchain in Healthcare and Research Workshop. 2016.

11.   Q. Xia, E. B. Sifah, K. O. Asamoah, J. Gao, X. Du and M. Guizani, "MeDShare: Trust-Less Medical Data Sharing Among Cloud Service Providers via Blockchain," in IEEE Access, vol. 5, pp. 14757-14767, 2017.

12.   S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” May 2009.

13.   J. Chen and S. Micali, ALGORAND, https://arxiv.org/pdf/1607.01341.pdf, May 2017

14.   M. Vukolić, “The Quest for Scalable Blockchain Fabric: Proof-of-Work vs. BFT Replication, Proc. of 2015 International Workshop on Open Problems in Network Security (iNetSec), 2015

15.   C. Cachin & M. Vukolić, “Blockchain Consensus Protocols in the Wild”, Proceedings of Intl. Symposium on Distributed Computing (DISC), 2017

16.   https://www.theverge.com/2017/11/13/16642690/bkav-iphone-x-faceid-mask

Proposer Bios/Webpages:

Karthik Nandakumar (http://researcher.ibm.com/researcher/view.php?person=sg-nkarthik) is a Research Staff Member at IBM Singapore Lab, where he works on technologies at the intersection of blockchain and biometrics. In the past, he has worked on video surveillance projects and developed robust algorithms for people counting and event detection. Prior to joining IBM in 2014, he was a Scientist at Institute for Infocomm Research, A*STAR, Singapore for more than six years. He received his Ph.D. degree in Computer Science from the Michigan State University in 2008. His research interests include computer vision, pattern recognition, biometric authentication, image processing, and machine learning. He has co-authored two books titled Introduction to Biometrics (Springer, 2011) and Handbook of Multibiometrics (Springer, 2006). He has received a number of awards including the 2008 Fitch H. Beach Outstanding Graduate Research Award from Michigan State University, the Best Paper award from the Pattern Recognition journal (2005), the Best Scientific Paper Award (Biometrics Track) at ICPR 2008, and the 2010 IEEE Signal Processing Society Young Author Best Paper Award.

Sharath Pankanti (http://researcher.ibm.com/person/us-sharat) is a Research Staff Member in AI Department at the Thomas J. Watson Research Center.  He received his Ph.D. degree in Computer Science from the Michigan State University.   Sharath has led a number of safety, productivity, education, healthcare, and security focused projects involving biometrics, multi-sensor surveillance, rail-safety, driver assistance technologies that entail object/event modeling, detection and recognition from information provided by static and moving sensors/cameras. Results of many of these efforts have demonstrated competitive results in scientific evaluations (NIST TRECVID-2012, 2013 and 2014, ImageClef 2013) and been integrated into real world applications. His work contributed to world’s first large scale biometric civilian fingerprint identification system in Peru and to award. winning IBM surveillance offering. that have been featured in news media (ABC/Fox/CBS/NBC), mentioned in popular TV media (CSI:Miami) and covered in social (good.is) media. He is a co-author of over 150 peer-reviewed publications (over 25,000 citations with h-index > 50 per Google Scholar) published in many reputed venues, including Scientific American, IEEE Computer, IEEE Spectrum, Comm. ACM, and Proc. IEEE.  He is also co-inventor of more than 100 inventions.

Nalini Ratha (http://researcher.watson.ibm.com/researcher/view.php?person=us-ratha) is a Research Staff Member with IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA, where he has led several projects in the area of Biometrics-Based Authentication Research. He has over 20 years of experience in the industry in the area of pattern recognition, computer vision, and image processing. He received the B.Tech. degree in electrical engineering and the M.Tech. degree in computer science and engineering from IIT Kanpur, Kanpur, India, and the Ph.D. degree in computer science from Michigan State University, East Lansing, MI, USA. He has authored over 80 research papers in biometrics. He has been the Co-Chair of several leading biometrics conferences, and served on the Editorial Boards of the IEEE T-PAMI, T-SMC, T-IP and Pattern Recognition journal. He has co-authored a popular book on biometrics entitled Guide to Biometrics and also co-edited two books entitled Automatic Fingerprint Recognition Systems and Advances in Biometrics: Sensors, Algorithms and Systems. He has offered tutorials on biometrics technology at leading IEEE conferences and also teaches courses on biometrics and security. He is a Fellow of the IEEE, IAPR and a Distinguished Scientist of the Association for Computing Machinery. From 2011 to 2012, he was the President of the IEEE Biometrics Council. He has received several awards, including the Research Division Award, the Outstanding Innovation Award, and the Outstanding Technical Accomplishment Award along with several patent achievement awards from the IBM Thomas J. Watson Research Center.