Indian Institute of Information Technology Dharwad

Data Science and Intelligent Systems

Course Objectives:

●       To understand the importance of Operating Systems.

●       To use OS algorithms for solving problems, which are related to the real-world or IT industry.

●       To extend the use of OS to the extent of other domains and fields.

https://github.com/animesh88/Operating-System

1. Introduction to the Operating System

2. Memory management and File system

3. Process Concurrency on CPUs

4. OS and Cloud Computing

5. Distributed Computing Systems

Project: Establish the Hadoop cluster and Spark cluster. Chose any open-source data. Run Spark ML-Lib algorithms. Then, report  Hadoop cluster installation (5 marks), Spark+Hadoop Cluster Demo (5 marks), Annotations (4 marks), PPT (6 marks), Write-up (10 marks), Video (10 marks).

References:

1. Mythili Vutukur. Lectures on Operating Systems, Department of Computer Science and Engineering, IIT Bombay, https://www.cse.iitb.ac.in/~mythili/os/

2. Avi Silberschatz, Peter Baer Galvin, and Greg Gagne. Operating System Concepts (Tenth Edition). https://www.os-book.com/OS10/slide-dir/index.html 

3. Online textbook Operating Systems: Three Easy Pieces (OSTEP)

4. William Stallings. “Computer Organization and Architecture”. 10th Edition

5. Fox Armando, Rean Griffith, A. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, and I. Stoica. "Above the clouds: A Berkeley view of cloud computing." Dept. Electrical Eng. and Comput. Sciences, University of California, Berkeley, Rep. UCB/EECS 28 (2009): 13.

6. Creeger, Mache. "Cloud Computing: An Overview." ACM Queue 7.5 (2009): 2.

7. http://www.w3.org/TR/2004/NOTE-ws-gloss-20040211

8. http://en.wikipedia.org/wiki/Web_Services_Description_Language,

9. http://en.wikipedia.org/wiki/Cloud_computing

10. http://aws.amazon.com/what-is-cloud-computing/

11. http://en.wikipedia.org/wiki/Data_center 

12. https://www.microsoft.com/en-us/research/blog/leslie-lamport-receives-turing-award/ 

13. https://en.wikipedia.org/wiki/Hypervisor

14. Cloud Computing: Past, Present, and Future, Professor Anthony D. Joseph, UC Berkeley Reliable Adaptive Distributed systems Lab (RAD lab) UC Berkley http://abovetheclouds.cs.berkeley.edu/ 

15. https://en.wikipedia.org/wiki/Google_File_System 

16. https://sites.google.com/site/gfsassignmentwiki/home 

17. Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung. "The Google file system." Proceedings of the nineteenth ACM symposium on Operating systems principles. 2003.

18. Jeffrey Dean, and Sanjay Ghemawat. "MapReduce: simplified data processing on large clusters." Communications of the ACM 51.1 (2008): 107-113.

19. Matei Zaharia, et al. "Apache spark: a unified engine for big data processing." Communications of the ACM 59.11 (2016): 56-65.

20. Matei Zaharia,  et al. "Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing." Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 2012.

21. https://spark.apache.org/docs/latest/cluster-overview.html 

22. https://spark.apache.org/

23. https://spark.apache.org/docs/3.3.1/ml-frequent-pattern-mining.html

24. Rakesh Agrawal, Tomasz Imieliński, and Arun Swami. "Mining association rules between sets of items in large databases." SIG-MOD. 1993.

25. Ramakrishnan Srikant, and Rakesh Agrawal. "Mining Generalized Association Rules.“ VLDB 1995.

26. Han Jiawei, Jian Pei, and  Yiwen Yin. "Mining frequent patterns without candidate generation." ACM SIGMOD Record 29.2 (2000): 1-12.

27. Haoyuan Li, et al. "PFP: Parallel FP-Growth for query recommendation." Proceedings of the 2008 ACM Conference on Recommender systems. 2008.

28. Weiser, Mark. "The computer for the 21st century." ACM SIGMOBILE Mobile Computing and Communications Review 3.3 (1999): 3-11.

29. https://en.wikipedia.org/wiki/Ubiquitous_computing

30. https://en.wikipedia.org/wiki/Edge_computing 

31. https://en.wikipedia.org/wiki/Cloudlet 

32. IEEE Standard Association. "IEEE 1934-2018-IEEE Standard for adoption of OpenFog reference architecture for fog computing." (2018).

33. https://en.wikipedia.org/wiki/Fog_computing

Course outcome:

CO1. Execute Spark based Map-Reduce algorithms using existing theories and knowledge.

CO2. Learning Distributed File Systems and Sparks Distributed Computing skills to use, create, and develop systems.

CO3. Test, evaluate, and assess a project's capability to work for Web and Data Science technologies.

CO4. Implementing a part of a project based on the Spark based algorithms.

CO5. Applying the learned theories and Distributed Systems algorithms for real world applications, projects, and systems. 

Evaluation Method

Item, Weightage (%)

Assignment 1 Write-up of the project: 10

Project Technical work: 20

Assignment 2 Presentation of Project: 10

Quiz or Viva: 10

Mid Term: 20

End Term: 30 

Prepared by: Dr. Animesh Chaturvedi