Challenge: Applying logistic regression to classify a product as a fruit or not

The next Python code deals with a data set with several features:

1. si_transaction_id - shipment instruction Id

2. harmonized_system_code - declared commodity code

3. container_uncode - container code denoting the container characteristics

4. package_count - the number of packages

5. allocated_container_count - the number of containers in the shipment instruction

6. weight_kg - the container weight

7. pol_city_unlocode - the place of loading that contains the city UNLOCODE

And two additional columns could be computed employing the data of the previous features:

8. weight_per_unit - the container weight / the number of packages

9. volume_per_unit - the volume of the container, which  was pre-calculated based on its code / the number of packages

After cleaning and selecting the proper input variables the following final result should present the confusion matrix and ROC curve like the graphics presented in the next Figures.

It is important to observe that these figures had been obtained after the application of the following order of operations in data processing:

1. From the data frame select input variables and store them in variable X, while the output variable should be in y variable.

2. Apply a transformation to the categorical variable related to values in column ''pol_city_unlocode''. Although transformation to a dummy (binary) variable is possible, it is not practical since the values in this column could be a range about of 400 cities, this is not a proper strategy. A more adequate strategy is to employ an encoding that maps each city name into a real number in an interval [0, 1].

3. Apply scale transformation to avoid numerical instabilities during the application of the classification method.

4. Perform the Train-Test Split.

Another possible sequence of data processing steps could be 1, 4, 2, 3.

As it can be seen there will a slight difference in the final results.