Platform Selection

Introduction
The system is an AI-powered analytics platform designed to enhance buying-center engagement for enterprise B2B technology firms. The goal is to develop a 3x3x3 Engagement Cube that scores accounts across Interest, Intent, and Ideal Customer Profile (ICP) dimensions, enabling marketing, sales, and RevOps teams to identify which accounts are most ready to buy. The application serves as a decision-support system for strategists, marketers, and data scientists, visualizing engagement data, forecasting growth signals, and aligning teams around high-intent opportunities. The project focuses on model experimentation, parameter tuning, calibrated scoring, SHAP-based explanations, and an RL-based validation layer using real SFDC outcomes.

Selections
Programming Language: Python
IDE: Visual Studio Code, Jupyter Notebook
Core Libraries: Pandas, NumPy, LightGBM, PyTorch, SHAP, imbalanced-learn, boto3/s3fs
Data Source: Local CSV files, optional AWS S3 integration
Hardware: Local development machines, Longleaf Cluster (CPU-based)
Operating Systems: macOS, Windows, Linux

Alternatives

Programming Language
Python (Chosen)
Pros:
- Rich ecosystem for ML and data science (pandas, NumPy, LightGBM, PyTorch, SHAP)
- Strong support for tabular modeling, calibration, and explainability
- Simple integration with Jupyter, CLI tools, and AWS SDKs for S3 data access
Cons:
- Slower than compiled languages (C++, Java) for tight, performance-critical loops
- Requires careful environment management for heavier libraries (LightGBM, PyTorch)

R
Pros:
- Excellent for statistical modeling and exploratory data analysis
- Strong plotting and reporting libraries (ggplot2, Shiny)
Cons:
- Less natural fit for RL and deep learning workflows
- Smaller ecosystem for production-style pipelines and web/API integration

Decision: Python selected for its mature ML ecosystem, strong support for tabular models (LightGBM), and compatibility with PyTorch for RL validators. It aligns with the team’s goal of iterative experimentation and model-driven insights using real enterprise data.

Development Environment
Visual Studio Code + Jupyter Notebook (Chosen)
Pros:
- VS Code: Lightweight, extensible, integrates well with Git and Python tooling
- Jupyter: Ideal for interactive development, quick testing, and feature/label exploration
Cons:
- Requires moving between notebooks for exploration and modules/scripts for production code
- Less “all-in-one” than a heavyweight IDE for advanced debugging and profiling

PyCharm
Pros:
- Strong Python debugging, refactoring, and profiling tools
- Integrated unit test support and project management features
Cons:
- Heavier IDE with more overhead for quick, ad-hoc experiments
- Less convenient for notebook-style analysis compared to Jupyter

Decision: A dual-environment approach is used — VS Code for modular pipeline development (under src/ and CLI scripts) and Jupyter Notebook for data exploration, sanity checks, and one-off experiments. This balances productivity for data work with maintainable code for the pipeline.

Modeling and ML Framework
LightGBM + PyTorch (Chosen)
Pros:
- LightGBM: High performance on structured/tabular data and fast training
- LightGBM: Handles missing values and class imbalance well, and works cleanly with SMOTE and calibration
- PyTorch: Flexible for custom RL architectures (DQN, LSTM) and iterative experimentation
Cons:
- Two major ML libraries to install and manage across environments
- RL components add complexity beyond a standard supervised-only pipeline

TensorFlow / Keras
Pros:
- Strong deployment tooling (TF Serving, TF Lite) and GPU support
- Large ecosystem for deep learning production scenarios
Cons:
- Less natural fit for tree-based tabular models compared to LightGBM
- More rigid and verbose for experimental RL work relative to PyTorch

Decision: LightGBM is used for the core Interest and Intent models, where gradient-boosted trees work best on account-level features. PyTorch is used specifically for the RL validators that adjust scores based on historical conversion and pipeline outcomes. This split matches the problem: tree models for prediction, deep RL for post-hoc correction.

Data Libraries and Storage
Pandas + NumPy + SHAP + imbalanced-learn + AWS S3 (Chosen)
Pros:
- Pandas/NumPy: Powerful for joins, aggregations, and windowed features at the account level
- SHAP: Provides per-account explanations that can be surfaced directly to GTM teams
- imbalanced-learn (SMOTE): Addresses rare-event labels such as conversions and pipeline creations
- AWS S3: Allows the pipeline to read real enterprise data without hard-coding file paths
Cons:
- In-memory DataFrames can become a bottleneck for very large datasets
- SHAP can be computationally expensive without background sampling or constraints

Spark / Dask and Database Backends
Pros:
- Better suited for distributed processing and very large datasets
- Databases (Postgres, Snowflake, etc.) provide stronger governance, concurrency, and SQL access
Cons:
- Higher setup and maintenance overhead for this course project
- Adds infrastructure complexity beyond what is needed for local-first experimentation

Decision: Pandas and NumPy form the core data layer for feature building and label generation, with SHAP and imbalanced-learn layered on top for explainability and imbalance handling. Data is stored as local CSV/Parquet files, with optional S3 download configured in config.yaml. This remains simple to run on a single machine or CPU cluster node while still supporting real SFDC-based data.

Hardware
Local Development + Longleaf Cluster (Chosen)
Pros:
- Local machines are sufficient for day-to-day iteration, testing, and smaller training runs
- Longleaf cluster supports batch jobs, backtesting, and heavier experiments on CPUs
Cons:
- Requires basic knowledge of job scheduling and cluster usage
- No dedicated GPU in the current setup, so RL training remains CPU-bound

Cloud Compute (AWS, GCP)
Pros:
- Elastic scaling and access to CPU/GPU instances for faster RL or large backtests
- Can integrate tightly with S3 and other managed services
Cons:
- Additional setup, permissions, and cost management
- Overkill for the current academic scope and local-first design

Decision: The pipeline is designed to run end-to-end on local machines, with the option to offload heavier training and backtesting to Longleaf using the same scripts. This avoids cloud complexity and cost while still providing a scalable path when more compute is needed.

Operating System
macOS, Windows, Linux (Chosen)
Pros:
- Supports cross-platform collaboration among team members
- Increases portability to different servers or future container-based deployments
Cons:
- Requires consistent dependency management (virtualenv/conda, pinned versions) across OSes

Decision: A multi-OS development approach is adopted to match team preferences and typical enterprise environments. Python, VS Code, LightGBM, PyTorch, and related libraries are all cross-platform, so future developers primarily need to follow the requirements.txt and Makefile workflow to reproduce the environment.