DATA SCIENCE FOR DECISION MAKERS

If you are consistently missing classes, it is YOUR RESPONSIBILITY to stay on top of the course material, as well as stay ahead of the course material.  If you need an extension on an assignment, you need to contact me and ask for my permission.  An absence does not mean I will automatically give you an extension on an assignment.  Also see this link on LATE WORK.

=================================================

Bring  laptop chargers  to class.  If your laptop dies during class, I will NOT automatically give you an extension to submit your quizzes and tests at a later time.

Bring a  mouse  to class if that will help you  (and speed things up)  when coding.

=================================================

Important:  for all the python files that you are uploading to  Blackboard, make sure to: 

• replace  LASTNAME  in the file name with  your last name  and 

• in the python file, replace  [...,...,...]  with your alpha  [123456].

Otherwise, you will receive 0 on that assignment because a computer will not be able to detect that you submitted an assignment.

Collaboration/discussion is encouraged, but  each person  must upload your lab and homework to Blackboard.

You do not need to cite your classmates/peers. 

=================================================

If you registered for this course late, please read everything on this page.  Also talk to your peers to find out what you missed.  You can also talk to me before and after class, as well as during EIs to find out what you missed. 

Day 1  (Tues 9 Jan):  Course Introduction, Course Folder Setup, Anaconda Troubleshooting

• NO MIDDAY BREAK  schedule due to the weather.

• No EI this afternoon.

Day 2  (Wed 10 Jan):  Jupyter Notebooks

• TWO HOUR DELAYED START  schedule today due to the weather.

• No EI this afternoon.

• LAB 00 assigned

Day 3  (Fri 12 Jan):  Univariate Data: Tables, Arrays, Histograms

• Continue Day 02 Lecture Notes. 

• Start on Day 03 Lecture Notes.

Day 4  (Wed 17 Jan):  Univarite Data: Manipulating Tables

• LAB 00 due by 2359

• HW 00 assigned

• Please go through Day 3 and Day 4 lecture notes in small groups.

• You may also work on your lab00 or hw00.  Collaboration is allowed. 

• Section 3001 substitute:  Professor Marius Ionescu

• Section 4001 substitute:  Professor Marius Ionescu

Day 5  (Fri 19 Jan):  NO CLASSES TODAY  [SNOW DAY]  ❄ ❅ ❆ ❄ ❅ ❆

Day 6  (Mon 22 Jan):  Univariate Data: Categorical, Histogram, Bar Graphs

• Go over Day 4:  Univarite Data: Manipulating Tables

Day 7  (Wed 24 Jan):  Multivariate Data: Sorting, Scatterplots, Exploratory Data Analysis (EDA)

• Go over Day 4:  Univarite Data: Manipulating Tables  [finish the last few questions at the end]

• Example Zip File

• Go over Day 5:  Univariate Data Simple Models  [finish this lecture notes on your own  (there is just 1 problem here); the answers are posted on Day 6]

• HW 00 due by 2359

• HW 01 assigned

Day 8  (Fri 26 Jan):  Data Manipulation, Functions

• Go over Day 7:  Multivariate Data: Sorting, Scatterplots, Exploratory Data Analysis

• QUIZ 00  [this quiz will cover Day 01 - Day 05 material]

  • • You have 20 minutes to complete and upload the quiz to Blackboard.

    • Will be given as a password-protected zip folder.

    • I will write down the password on the board on the day of the quiz.

    • If you cannot get jupyter notebook open by yourself on the first quiz, you will receive 10% penalty.

    • To ensure you are set-up for success, there is a practice zip folder to test on under Day 07 Website Folder.  The password is "password".  If you can unzip it, save it in a local folder, and open the jupyter notebook, you should be good to go for the quiz.  

    • Objectives that may be on Quiz 00: 

      • how to read a data table from a CSV file

      • how to use the .column, .select, and .with_column functions

      • how to use .column with another function (such as .mean) to calculate an average

      • how to use .where and a predicate (such as are.equal_to) to filter a data table

      • how to use .sort to sort the values of a table from low to high or high to low

      • make visualizations such as scatter plot, horizontal bar graph, and histogram

      • interpret a histogram

Day 9  (Mon 29 Jan):  Multivariate Data: Applying Functions

• Go over Day 8:  Data Manipulation, Functions

• Go over Day 9:  Multivariate Data: Applying Functions

• HW 02 assigned

Day 10  (Wed 31 Jan):  Multivaritate Data: Lab

• Section 3001 ONLY:  finish last two questions on Day 9 Lecture Notes

• LAB 01 assigned

• HW 01 due by 2359

Day 11  (Fri 2 Feb):  Advanced Data Manipulation: Joining and merging

Day 12  (Mon 5 Feb):  Advanced Data Manipulation: Pivot Tables

• HW 02 due by 2359

Day 13  (Wed 7 Feb):  Advanced Data Manipulation: Branching and Function Wrapping

• Go over Day 12:  Advanced Data Manipulation: Pivot Tables

• Go over Day 13:  Advanced Data Manipulation: Branching and Function Wrapping

• QUIZ 01

• LAB 01 due by 2359

• HW 03 assigned

Day 14  (Fri 9 Feb):  Advanced Visualization: Storytelling

• Go over Day 12:  Advanced Data Manipulation: Pivot Tables

• Go over Day 13:  Advanced Data Manipulation: Branching and Function Wrapping

• Go over Day 14:  Advanced Visualization: Storytelling  [go through this on your own]

Day 15  (Mon 12 Feb):  Review

• Go through any of the 4 practice tests in preparation for Friday's in-class test.

Day 16  (Wed 14 Feb):  Review

• Go through any of the 4 practice tests from Day 15 in preparation for Friday's in-class test.

• HW 03 due by 2359  [this is NO LONGER due by Mon 12 Feb]

Day 17  (Fri 16 Feb):  Test 00

• This is an in-class exam during your regularly scheduled class. 

Day 18  (Wed 21 Feb):  Probability

Day 19  (Fri 23 Feb):  Probability

• HW 04 assigned

Day 20  (Mon 26 Feb):  Expectation and Variance (Split Day 19/20 into 3 lectures)

Day 21  (Wed 28 Feb):  Binomial Random Variables

• Let's talk about  variance  and  standard deviation  a little bit more before moving onto binomial random variables.

• Complete Probability Practice  (it's under Day 21/22) and upload no later than Wed 6 March by 2359 for +10 bonus points on your lowest LAB grade.

Day 22  (Fri 1 Mar):  Normal Random Variables

• HW 04 due by 2359

• HW 05 assigned

• Complete Probability Practice  (it's under Day 21/22) and upload no later than Wed 6 March by 2359 for +10 bonus points on your lowest LAB grade.

Day 23  (Mon 4 Mar):  Sampling and Iteration

• Day 20:  let's talk about the z-score before moving onto Day 23 notes.

• Complete Probability Practice  (it's under Day 21/22) and upload no later than Wed 6 March by 2359 for +10 bonus points on your lowest LAB grade.

Day 24  (Wed 6 Mar):  Center and Spread  and  Central Limit Theorem

• QUIZ 02  (25 minutes,  8 problems)

• Complete Probability Practice  (it's under Day 21/22) and upload no later than Wed 6 March by 2359 for +10 bonus points on your lowest LAB grade.

Day 25  (Fri 8 Mar):  Variability of Sample Average

• Day 24 Notes:  complete it

• HW 05 due by 2359

• Have a great Spring Break!!

Day 26  (Mon 18 Mar):  Hypothesis Testing: One Variable

• HW 06 assigned

Day 27  (Wed 20 Mar):  Hypothesis Testing Lab

• LAB 02 assigned

• Everyone, please re-download  radios.csv  file.  You've been using an old  radios.csv  file.  But all the work that you did should remain the same.

Day 28  (Fri 22 Mar):  Quiz, HW/Lab Review

• Complete Practice Quiz 03 by Wed 27 Mar at 2359 for +5 bonus on your highest quiz grade.

Day 29:  (Mon 25 Mar):  Hypothesis Testing: Two Sample, A/B Testing

• Complete Practice Quiz 03 by Wed 27 Mar at 2359 for +5 bonus on your highest quiz grade.

• HW 06 due by 2359

• HW 07 assigned

Day 30:  (Wed 27 Mar):  Hypothesis Testing: Two Sample, A/B Testing

• Complete Practice Quiz 03 by Wed 27 Mar at 2359 for +5 bonus on your highest quiz grade.

• QUIZ 03  (will be given at the beginning of class),  25 minutes,  8 questions.

Day 31  (Fri 29 Mar):  Bootstrapping

• Section 3001 substitute:  Professor Jad Salem

• Section 4001 substitute:  Professor Marius Ionescu

• LAB 02 due by 2359

Day 32  (Mon 1 Apr):  Bootstrapping  and  Review

• HW 07 due by 2359

• 4 practice tests

Day 33  (Wed 3 Apr):  Review

• 4 practice tests

Day 34  (Fri 5 Apr):  Test 01  (17 questions,  worth a total of 23 points)

• Section 3001 substitute:  Maj Mitch Graves

• Section 4001 substitute:  Maj Mitch Graves

Day 35  (Mon 8 Apr):  Introduction to Prediction

Day 36  (Wed 10 Apr):  Introduction to Correlation

Day 37  (Fri 12 Apr):  Regression Lines

• 4th period only: MIDN Zeta Williamson will complete Day 36 Correlation notes (the last 3 questions)

• Additional notes from Day 36:  Correlation does not imply causation

Day 38  (Mon 15 Apr):  Regression Lines Variability

• HW 08 assigned

Day 39  (Wed 17 Apr):  Inference for Regression

Day 40  (Fri 19 Apr):  Machine Learning: Train/Test Sets, Classification, Accuracy

• HW 09 assigned

Day 41  (Mon 22 Apr):  Ethical Issues in Data Science

• HW 08 due by 2359

  • • Work on HW08

    • Work on HW09

    • Go through today's python file

    • Go through Practice Quiz 04 python file

• I'm taking several midshipmen to the U.S. Air Force Academy.  Please be on your best behavior with Professor Traves!

• Section 3001 substitute:  Professor Will Traves

• Section 4001 substitute:  Professor Will Traves

Day 42  (Wed 24 Apr):  Khost Call Lab  You'll be working on a DIFFERENT project  (see below for  Lab 03  instructions)

• Partner List;  upload  1 python file  and  1 csv file  per group to Blackboard.

• Points:  LAB03  is worth  2 LABS,  i.e.,  200 points.

• Python collaboration website:  cocalc.com

• Details:

  • 1. Find a data set with at least 2 variables.

    2. For full credit, use techniques from ALL three sections of the course listed below, and then ANSWER ALL of the questions below.  All techniques and answers must be in your python file.

    3. Due:  Monday 29 April 2024  at  2359

    4. Late Work:  will be docked 25% of your grade.  All assignments are due by Wednesday  1 May 2024  at  2359 (last day of the semester).  USNA faculty members are  not  allowed to accept any assignment after  1 May. 

• Data Analysis

  • 1. Visualization  (.hist,  .barh,  .scatter,  .plot)

    2. .group  (Day 5)

    3. Functions and applying them using  .apply  (Day 9)

    4. .where  to filter through the table  (Day 11)

    5. .sort  (Day 7)

• Hypothesis Testing

  • 1. General hypothesis testing  (Day 26)

    2. A/B testing  (Day 29)

    3. Bootstrapping of slope of line of best fit, with results explained on a confidence interval you deem is appropriate.  (Day 31)

• Linear Regression and Machine Learning

  • 1. Correlation  (Day 36)

    2. Regression  (Day 37 and 38)

    3. Predictions using Regression  (Day 39)

    4. k-nearest neighbors clustering  (Day 40)

• Questions to Answer  (answer each question in a separate line in python)

  • 1. Which visualizations help us to see the big picture quickly?

    2. Does time of year or temperature influence your data set? 

    3. What affects your data set?

    4. Are there any correlations among your variables?

    5. Compute the average for each variable.

    6. Can we get a sense of how the averages may change with new data?

    7. What sort of predictions can you make?

Day 43  (Fri 26 Apr):  Khost Call Lab  You'll be working on a DIFFERENT project  (instructions are under Day 42)

• QUIZ 04

• HW 09 due by 2359

Day 44  (Mon 29 Apr):  Review & SOFs

• SOF

• LAB03

• Partner List;  upload  1 python file  and  1 csv file  per group to Blackboard.

• Points:  LAB03  is worth  2 LABS,  i.e.,  200 points.

• Python collaboration website:  cocalc.com

• Details:

  • • Find a data set with at least 2 variables.

    • For full credit, use techniques from ALL three sections of the course listed below, and then ANSWER ALL of the questions below.  All techniques and answers must be in your python file.

    • Due:  Monday 29 April 2024  at  2359

    • Late Work:  will be docked 25% of your grade.  All assignments are due by Wednesday  1 May 2024  at  2359 (last day of the semester).  USNA faculty members are  not  allowed to accept any assignment after  1 May. 

• Data Analysis

  • • Visualization  (.hist,  .barh,  .scatter,  .plot)

    • .group  (Day 5)

    • Functions and applying them using  .apply  (Day 9)

    • .where  to filter through the table  (Day 11)

    • .sort  (Day 7)

• Hypothesis Testing

  • • General hypothesis testing  (Day 26)

    • A/B testing  (Day 29)

    • Bootstrapping of slope of line of best fit, with results explained on a confidence interval you deem is appropriate.  (Day 31)

• Linear Regression and Machine Learning

  • • Correlation  (Day 36)

    • Regression  (Day 37 and 38)

    • Predictions using Regression  (Day 39)

    • k-nearest neighbors clustering  (Day 40)

• Questions to Answer  (answer each question in a separate line in python)

  • • Which visualizations help us to see the big picture quickly?

    • Does time of year or temperature influence your data set? 

    • What affects your data set?

    • Are there any correlations among your variables?

    • Compute the average for each variable.

    • Can we get a sense of how the averages may change with new data?

    • What sort of predictions can you make?

• Review

Day 45  (Wed 1 May): Review

• Section 3001 substitute:  Professor Marius Ionescu

• Section 4001 substitute:  Professor Marius Ionescu

• Review

Final Exam  (Mon 6 May):  1300 - 1600

• Section 3001:  CH002  (20 midshipmen)

• Section 4001:  CH001  (20 midshipmen)

Alternate Final Exam  (Fri 10 May):  0755 - 1055

• Room:  CH160  (2 midshipmen, with Professor Mark Magsino)

Wed 24 Jan:

• Lecture Notes answers will be posted on the Lessons Link at the end of each day so that you can continue to work through them at your own pace.

• Day 05 Lecture Notes answer: 

• Question 1. 

  • • • # We've broken the computation down into smaller steps -- always a good idea when coding

      • nba_grouped = nba.group('Position', ...) 

      • nba_fewer_cols = nba_grouped.drop(..., ...)

      • nba_renamed = nba_fewer_cols.relabel(make_array(..., ...),

      •                                  make_array(..., ...))

      • nba_renamed.... # make horizontal bar graph

• Answer to Question 1.

  • • • # We've broken the computation down into smaller steps -- always a good idea when coding

      • nba_grouped = nba.group('Position', np.mean) 

      • nba_fewer_cols = nba_grouped.drop(1, 4)

      • nba_renamed = nba_fewer_cols.relabel(make_array("Height mean", "Weight mean"),

      •                                  make_array("Mean height", "Mean weight"))

      • nba_renamed.barh(0) # make horizontal bar graph

Mon 29 Jan:

• Bring your laptop charger to class, as well as a mouse, if these will help you when using python.  Also, I will not automatically give you an extension on an assignment, quiz, etc. all because your laptop died in the middle of the class. 

• Make sure to bring the  reference sheet  to class every day.

Wed 31 Jan:

• LAB00, HW00, and Quiz00  have been graded today.  Feedback will be emailed out to you after this Friday afternoon 2 Feb 2024.

Fri 2 Feb:

• There are BUGS in the python autograder  (if you got a problem correct, it is marked as wrong while an incorrect answer is marked as partially correct)!  So I will re-grade  LAB00, HW00, and Quiz00  over the weekend.

Sat 3 Feb: 

• I started  re-grading  your Quiz00 yesterday.  It should be done today and feedback will be emailed to you by tonight.

• Your  Quiz00  has been re-graded today!  You should have received an automated email with feedback.

Sun 4 Feb:

• I will re-grade LAB00 and HW00 today, and get these feedback emailed out to you.

• HW00  has been re-graded today.  I gave everyone +1 bonus point and I did not deduct 25% off of your grade if you uploaded your assignment late.

• LAB00  has been re-graded today.  I gave everyone +1 bonus point and I did not deduct 25% off of your grade if you uploaded your assignment late.

Tues 6 Feb:

• For Quiz 00, the correct solution to #6  create a scatterplot of Sodium vs Fat (g)  should have been: 

  • • • starbucks.scatter('Fat (g)', 'Sodium') 

• So you have to flip the order for a scatterplot.

• Our class average is:  52.6776%.

Wed 7 Feb:

• HW01 has been graded and feedback has been emailed to you. 

• Please DOUBLE CHECK to make sure you are not typing your alpha INCORRECTLY!  Continuing to make this mistake will result in a 0 for that assignment.

• Please do NOT delete the square brackets around your alpha, or else, you will get a zero on your assignment:

  • • • global alphas # leave this line alone and replace ... 

      •               # in the next line with your alpha number, e.g. alphas = [230000]

      • alphas = [123456]

• Please do NOT put multiple alphas in your python file.  Just put YOUR and ONLY YOUR alpha on your python file.

• Please complete your assignments much in advance-- if you do not upload and submit your file 1 day before the due date, I will not continue to override and give you +1 bonus point.

• If you upload a WRONG file to Blackboard, that is an automatic 0.  So please double-check what you upload to Blackboard.

• Keep in mind that I will accept late work (HWs and LABs), up to 7 days after the due date.  After this 7 day period has ended, I will NOT accept any late work  (see my website on  Late Work  as well as  Course Policy Statement).

Thurs 8 Feb:

• QUIZ01  has been graded and feedback has been emailed to you.

• Our class average is:  72.8895%.

• You MUST upload your quizzes and tests to Blackboard before you leave the classroom.  If you upload your file AFTER you leave the classroom without my authorization, this is an AUTOMATIC 0.

Tues 13 Feb:

• HW02  has been graded and feedback has been emailed to you.

Thurs 15 Feb:

• LAB01  has been graded and feedback has been emailed to you.

Sat 17 Feb:

• TEST00  has been graded and feedback has been emailed to you.

  • • • Our class average is:  81.7451%.

      • This is also the very first time that EVERY SINGLE ONE OF YOU uploaded your file correctly, on time, and you didn't omit your alpha, type wrong alpha, remove square brackets from your alpha, etc.  Great job!

      • Have a wonderful 3-day weekend!

Tues 20 Feb:

• Your 6-week grades and MAPRs have been submitted to MIDS.

Thurs 22 Feb:

• HW03  has been graded and feedback has been emailed to you.

Sat 24 Feb:

• HINTS for HW04:

  • •  I didn't read the instructions correctly and thought that "A" in a stack of cards is a face card.  I have fixed these solutions.

    • The first 4 digits after the decimal sum to:

      • • Question 1.  13

        • Question 2.  18

        • Question 3.  20

        • Question 4.    7

        • Question 5.  12

        • Question 6.  11  15

        • Question 7.  24  25

        • Question 8.  13

        • Question 9.  21

        • Question 10.  8

        • Question 11.  19

        • Question 12.  True  or  False?

          • • P(cloudy) = 18/31,  P(cloudy and Friday) = 3/31,  and  P(Friday) = 4/31 or 5/31,  depending on the year (some years have 4 Fridays in January while other years have 5 Fridays in January). 

Tues 27 Feb: 

• Complete Probability Practice  (it's under Day 21/22) and upload no later than Wed 6 March by 2359 for +10 bonus points on your lowest LAB grade.

Wed 28 Feb:

• HINTS for Probability Practice  (it's under Day 21/22).  Complete this python file and upload this to Blackboard no later than Wed 6 March by 2359 for a bonus of +10 points.  If your solution is different from mine, then contact me.  It is possible that I may have a typo.

  • • The first 4 digits after the decimal sum to:

      • • Question 1.  16

        • Question 2.  12

        • Question 3.  21

        • Question 4.  10  (assume that you are choosing two cards with replacement)

        • Question 5.    0

        • Question 6.  26

        • Question 7.  15

        • Question 8.  13

        • Question 9.  10

        • Question 10.  25

        • Question 11.  4

        • Question 12.  19

        • Question 13.  17

        • Question 14.  21

        • Question 15.  13

        • Question 16.  27

Thurs 7 Mar:

• QUIZ02  has been graded and feedback has been emailed to you.

• Our class average is:  72.7531%.

Thurs 7 Mar:

• Probability Practice-BONUS  has been graded.

Sat 9 Mar:

• HW04  has been graded and feedback has been emailed to you.

Sat 23 Mar:

• HW05  has been graded and feedback has been emailed to you.

Thurs 28 Mar:

• Practice Quiz 03  has been graded.  I'll aim to grade Quiz 03 by tonight.

• Quiz 03  has been graded and feedback has been emailed to you.  Our class average is  64.6886%.

Sat 30 Mar:

• HW06  has been graded and feedback has been emailed to you.

Sat 6 Apr:

• LAB02  has been graded and feedback has been emailed to you.

• TEST01  has been graded and feedback has been emailed to you.

  • • • Our class average is:  76.3932%.  This includes a bonus of  +2.17  points. 

• Your 12-week grades have been submitted to MIDS!

Tues 9 April:

• All MAPRs have been submitted today.

Fri 12 April:

• The autograder to grade your python files has quite a lot of errors, including it has been deleting all of my feedback to you!  I did not know about this until last week during an EI.  Therefore, feedback has been emailed to you for: 

  • • Test00 material:  Quiz 00, Quiz 01, Lab 00, Lab 01, HW00, HW01, HW02, HW03, Test00.   

    • Test01 material:  Quiz 02, Quiz 03, Lab 02, HW04, HW05, HW06, Test01.

      • • I still can't grade HW07 since the autograder is still broken.  Once the Course Coordinator encodes the autograder into the solutions, then I will grade HW07.

Sat 13 April: 

• HW07  has been graded and feedback has been emailed to you.

Sat 27 April: 

• HW08  has been graded and feedback has been emailed to you.

Mon 29 April:

• QUIZ04  has been graded and feedback has been emailed to you.

Tues 30 April:

• HW09  has been graded and feedback has been emailed to you.

Tues 30 April - Fri 3 May: 

• Your  Lab03 - PROJECT  has been graded.  Your  LAB03  average is  89.7561%.

Sat 4 May:

• Your  Lab03 - PROJECT feedback  has been emailed to you.

Tues 7 May:

• Your  Final Exam  has been graded.  Our class average is  78.2506%.

• Your  final grades  have been submitted to  MIDS.

• I will work on  MAPRs  in the next several days.