Tools and Techniques In DataScience

Note: This page is under construction and contents are dynamic. Keep visiting for updated contents!

This course, Tools and Techniques in data Science, is a part of BSCS curriculum as an optional course. Currently it is offered in fifth semester. I want to prepare students in a way that they learn those skills of this course that help them excel in AI and machine elarning field as we know  many steps are common in ML learning project and a data scinece project aka data science work flow. In the age of AI I want to engae GenAI, LLMs, agents in this course as well.

Course Contents

Data Science Work Flow

1. 1. Defining the problem

   2. Data colleciton and Prepation 

- Extract Transforkm Load (ETL)

1. 3. Data Preprocessing

      1. Categorical to Numeric Conversion

         1. One hot encoding

      2. Tacklinf Null values

      3. Centering and Scaling - Data Normalization

   4. Exploratort Data Analysis 

      1. Bar Charts, Pie Charts, Histograms, Stemplots, Timeplots (1.2)  [Optional: Stemplots ]

      2. Mode, Median, Mean, Range, and Standard Deviation (1.3)

      3. The Five Number Summary, Boxplots, and Outliers (1.6)

      4. Explanatory and Response Variables, Correlation (2.1) 

   5. Data Modeling, Analysis and/or Machine Learning 

      1. Need to write topics related statistical analysis and others as well

      2. Machine Learning

         1. ML is an optional in data science work flow and use as per 

         2. We will study some sellected topics from following skill track https://app.datacamp.com/learn/skill-tracks/supervised-machine-learning-in-python

   6. Report writing 

      1. How to write reports 

      2. Report writing using MS Word by Dr James Clark King's College London: ehealth.kcl.ac.uk/sites/physiology/LST-MSOffice5.html 

• 1. 1. 1. Inserting Figures and Figure Legends in Microsoft Word 

        2. Setting up Heading formatting and numbering as well as tables of contents for Scientific reports, documents and Theses  

        3. Adding sections BEFORE your table of contents without numbering

        4. Producing a Figure for publication containing microscopy images in MS Word

Python

1. https://media.datacamp.com/cms/python-with-ai.pdf (TBD)

Sampling

2. 1. Types of Sampling Methods (4.1) see this video https://www.youtube.com/watch?v=pTuj57uXWlk 

   2. Read the html file related to sampling we discussed in the class  [download this file and double click it to open in your web browser e.g. chrome, firefox, or edge etc]

   3. Census, Nonresponse, and Undercoverage (4.2)  https://www.youtube.com/watch?v=EZrP_av3cmA

3. The Normal Distribution and the 68-95-99.7 Rule (5.2) https://www.youtube.com/watch?v=mtbJbDwqWLE

4. Z-Scores, Standardization, and the Standard Normal Distribution (5.3) https://www.youtube.com/watch?v=2tuBREK_mgE

Experimental Design

• Causation vs Association, and an Introduction to Experiments (3.1) 

• Placebo Effect, Control Groups, and the Double Blind Experiment (3.2)

• Types of Experimental Designs (3.3)

Prompt Engineering, Vibe coding

1. Types of prompts: ICC, zero shot, n-shot, CoT

2. LLMs are not deterministic models, Temperature, Tokens, Top p, Top k, context window, tool calling- Internet Search, Hallucination

3. Importance of Domain Knowlede, and Meta Skills in use of LLM models and vibe coding

4. System and User Prompt

5. Definition of Vibe coding and practice with Google AI studio 

Assignments:

1. Creating cvs file

2. Analyzing time table

   1. understandunbg the time table and  resolving the issue of thirty minutes slot

   2. Identifying issues in time table

      1.  Is course allocation per teachers choice

      2.  Is course allocation per teacher's expertise

      3.   Was teacher given time to prepare for the course

      4. Issues in time table in current form

         1. Per teachers numbert of courses in a day, Gap between two courses of a teacher, total lecture duration of a course , number of lectures in a week

         2. Hurdles in data collection e.g. was access to time table was easy?

   3. Time table scheduling - opimization, or AI, or manual? 

5. Project: Analyzing Crime in Los Angeles - see project description from here and press continue project hyperlink. On the left window you will see various tabs related to project guides, resouces, and solution. On the Right (main window) you will see project notebook, where you will run the code

6. Note:
   - Complete it yourself and it will be part of your sessionals assesment as well.
   - Students having less than 70% attendance are not elligible to sit in the final term exams and are also not elligible for sessional marks.

7. https://www.datacamp.com/blog/sports-analytics-euro-2024?src=data-ai-world-cup (TBD)

8. https://www.datacamp.com/code-along/create-claude-skills-for-data-tasks?src=data-ai-world-cup (TBD)

   1. https://media.datacamp.com/cms/resources---create-claude-skills-for-data-tasks-v2.pdf

   2. https://media.datacamp.com/cms/claude-skills-for-data-tasks.pdf

• Writing Efficient Python Code

• Exploratory Data Analysis in Python 

• Introduction to Statistics (in addition to related youtube videos one can learn theory from here)

• Introduction to Statistics in Python (in addition to related youtube channels one can learn from here)

• Sampling in Python (in addition to related youtube videos one can learn theory from here)

• Hypothesis Testing in Python (It will complement the topic discussed in EDA related course)

Note: It is must to attempt related Practice in Python, Theory and also related Projects available at datacamp!

Activity: Read the following link and explore various projects: Link to KD-Nuggets' Ten project list for Data Scientist 

Datacamp tracks 

• Excel Fundamentals

• Introduction to Data Science in Python

• Data Analyst in Python

• Data Analyst in Power BI