2. Graded Quiz: Classification

Question 1 What can we infer about our kNN model when the value of K is too big?

A. The model will capture a lot of noise as a result of overfitting.

B. The model will be too complex and not interpretable.

C. The training accuracy will be high, while the out-of-sample accuracy will be low.

D. The model is overly generalized and underfitted to the data.

Answer: D. The model is overly generalized and underfitted to the data.

Question 2 When splitting data into branches for a decision tree, what kind of feature is favored and chosen first?

A. The feature with the greatest number of categories.

B. The feature that increases entropy in the tree nodes.

C. The feature that splits the data equally into groups.

D. The feature that increases purity in the tree nodes.

Answer: D. The feature that increases purity in the tree nodes.

Question 3 What is the relationship between entropy and information gain?

A. When information gain increases, entropy decreases.

B. When information gain decreases, entropy decreases.

C. High entropy and high information gain is desired.

D. High entropy and low information gain is desired.

Answer: A. When information gain increases, entropy decreases.

Question 4 Predicting whether a customer responds to a particular advertising campaign or not is an example of what?

A. Classification problem

B. Machine learning

C. Regression

D. None of the above

Answer: A. Classification problem

Question 5 For a new observation, how do we predict its response value (categorical) using a KNN model with k=5?

A. Form 5 clusters and assign the new observation to the most similar cluster, taking the mean value as prediction.

B. Take the average among 5 points whose features are closest to the new observation.

C. Take the majority vote among 5 points who are the most similar to each other.

D. Take majority vote among 5 points whose features are closest to the new observation.

Answer: D. Take majority vote among 5 points whose features are closest to the new observation.