Integrate forensics into software source code.
Software Quality Assurance (Auburn University, Spring 2023, Fall 2023)
Mobile Security (Tuskegee University, Fall 2023)
Information Security Administration (Kennesaw State University, Spring 2023)
Pre-lab Content Dissemination
We first need to know what to log and how to log. Let us use the following heuristics.
Heuristics on what to log
Resources: Logging resource-related information helps monitor the system's utilization and availability of critical assets, including data about memory usage, CPU usage, disk space, network activity, and other resource-related metrics. Resource logs aid in detecting performance bottlenecks, resource leaks, and overall system health. They provide insights into whether the system operates within expected resource thresholds or requires optimization.
Exceptions: Logging exceptions and errors is crucial for analyzing and troubleshooting issues within the application. These logs capture details about unexpected behaviors, software bugs, and runtime errors. Exception logs often include stack traces, error messages, error codes, and contextual information to help developers pinpoint the root causes of problems and facilitate quicker resolution.
Change Issues: Logging changes and events in the system provides an audit trail of actions taken within the application, including user actions, configuration modifications, database updates, and other significant events. Change issue logs are valuable for tracking the history of system changes, diagnosing security breaches, and ensuring compliance with regulatory requirements. They offer transparency into who did what and when.
Heuristics on how to log
Include the name of the identity provider or security realm that vouched for the username if that information is available.
Include the affected system component or other object (such as a user account, data resource, or file).
Include the status that says if the object succeeded or failed.
Include the application context, such as the initiator and target systems, applications, or components.
Include “from where” for network connectivity or distributed application operation messages.
Include the time stamp and time zone to help answer “when.” The time zone is essential for distributed applications.
In-class Hands-on Experience
Go to the URL- https://docs.python.org/3/library/logging.html for a detailed Python logging mechanism. You can find a basic example here.
Learn about the common logging methods using Python.
A code for a custom logger is shared using screenshots.
Post Lab Experience
Imagine this scenario: Dolly, a data science specialist at University of West Florida. University has written machine learning (ML) code in `workshop9.py`. The output is correct, but she just learned about adversarial machine learning, which discusses how ML code can be attacked. She realized that ML models are susceptible to security issues:
`poisoning attacks` are introduced by passing in erroneous dataset files into ML code
`model tricking` results from models being attacked, which gives false results. Incorrect prediction performance is an indicator of ML models being attacked.
Assist Dolly by writing logging code in the correct locations in `workshop8.py` with comments so that the two issues, as mentioned earlier, are logged for all provided ML functions. After writing the code, put in comments to justify your code, save `workshop9.py`.
You must add at least find and add logging code to seven locations with comments.
Complete the survey at_ https://uwf.co1.qualtrics.com/jfe/form/SV_etcH0FRKr2IIo5g.
Sample Input/Output:
Input: workshop8.py
Expected Output: Code and Comments
iris = datasets.load_iris()
'''
Inserted logging statement because data can be poisoned.
'''
simpleLogger.info('Generic information: getting results: %s', str(res))
Step 1: Set Up Google Colab Environment
Go to Google Colab
Create a new notebook: File > New Notebook
Rename the notebook to adversarial_ml_logging.ipynb
Step 2: Install Required Packages
!pip install scikit-learn pandas numpy
Step 3: Create workshop9.py with Security Logging
%%writefile workshop9.py
import logging
import numpy as np
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
# ===== SECURITY LOGGING SETUP =====
logging.basicConfig(
filename='/content/ml_security.log',
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s'
)
security_logger = logging.getLogger('MLSecurity')
def validate_data_distribution(data):
"""Check for unusual data patterns that might indicate poisoning"""
if np.isnan(data).any():
security_logger.warning("NaN values detected in dataset - possible poisoning")
if (data < 0).any():
security_logger.warning("Negative values detected - possible poisoning")
return True
# ===== ML PIPELINE WITH SECURITY CHECKS =====
# 1. Data Loading
iris = datasets.load_iris()
'''
Security Log: Baseline dataset characteristics before any processing
'''
security_logger.info("Original dataset loaded - samples: %d, features: %d",
iris.data.shape[0], iris.data.shape[1])
validate_data_distribution(iris.data)
# 2. Data Splitting
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.2)
'''
Security Log: Verify split maintains class distribution
'''
security_logger.info("Train class distribution: %s", np.bincount(y_train))
security_logger.info("Test class distribution: %s", np.bincount(y_test))
# 3. Model Training
model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)
'''
Security Log: Model parameters and training completion
'''
security_logger.info("Model trained with params: %s", model.get_params())
# 4. Model Evaluation
train_score = model.score(X_train, y_train)
test_score = model.score(X_test, y_test)
'''
Security Log: Performance metrics for anomaly detection
'''
security_logger.info("Model performance - Train: %.3f, Test: %.3f", train_score, test_score)
if abs(train_score - test_score) > 0.3:
security_logger.warning("Large train-test gap detected - possible model tricking")
# 5. Predictions
predictions = model.predict(X_test)
'''
Security Log: Prediction sanity checks
'''
security_logger.info("First 5 predictions: %s", predictions[:5])
security_logger.info("Corresponding true labels: %s", y_test[:5])
# 6. Adversarial Check
confidence = model.predict_proba(X_test).max(axis=1)
if (confidence < 0.5).any():
security_logger.warning("Low-confidence predictions detected - possible adversarial samples")
# 7. Final Validation
validate_data_distribution(X_test)
security_logger.info("Pipeline execution completed")
Step 4: Run and Monitor in Colab
# Execute the ML pipeline
!python workshop9.py
# View security logs
print("\n=== SECURITY LOGS ===")
!cat /content/ml_security.log
# Optional: Download log file
from google.colab import files
files.download('/content/ml_security.log')
Step 5: Expected Output
=== SECURITY LOGS ===
2023-08-20 14:30:00,000 - INFO - Original dataset loaded - samples: 150, features: 4
2023-08-20 14:30:00,001 - INFO - Train class distribution: [42 42 36]
2023-08-20 14:30:00,002 - INFO - Test class distribution: [ 8 8 14]
2023-08-20 14:30:00,003 - INFO - Model trained with params: {'C': 1.0, ...}
2023-08-20 14:30:00,004 - INFO - Model performance - Train: 0.975, Test: 0.933
2023-08-20 14:30:00,005 - INFO - First 5 predictions: [0 1 1 0 2]
2023-08-20 14:30:00,006 - INFO - Corresponding true labels: [0 1 1 0 2]
2023-08-20 14:30:00,007 - INFO - Pipeline execution completed