Data Scientist Specializing in Predictive Modeling, Machine Learning, and Data-Driven Insights

I am a data scientist passionate about turning complex datasets into actionable insights. With expertise in machine learning, Bayesian statistics, and big data analytics, I craft solutions that drive decision-making and innovation.

Check out my highlighted project below or any of the tabs at the top right to see previous projects of mine ranging from Big Data Bowls, Shiny Apps, and my work at the NFL with Next Gen Stats.

Get in touch at Seth.Lanza@gmail.com, Connect on Linkedin, Check my Twitter, or Check out my Github

Background:

Started to learn about data science because of my love for the game of football. I first started paying closer attention to the NFL through the hype of the 2018 QB draft cycle which led to me diving into prospect evaluation, draft strategies, salary cap management, positional value, analytics, PFF, etc. Continued working on projects and developing my technical skills to advance into more complex methodologies for understanding and deriving insights from data.

Completed my Master's of Science in Business Analytics from the College of William & Mary in 2023 to further develop my technical skills. Worked at Booz Allen Hamilton as a Senior Consultant, Data Scientist and then with the National Football League as a Next Gen Stats Research Analyst honing my skills and industry knowledge.

Competencies:

Programs: R, Python, SQL, Tableau, PowerBI, Microsoft Office, Alteryx, and Git.

Core Skills: Proficient in predictive modeling, development of heuristic algorithms, optimization & linear programming, statistical computations, Bayesian statistics, natural language processing (NLP), and web scraping & automation.

Cloud Platform Skills: Databricks, Spark in Python, Google Colab. AWS, Azure, & Google Cloud applications. AWS Cloud Certified Practitioner

Machine Learning Management: Mlflow experiement tracking, Poetry dependency manager, Uv virtual environment manager, Docker

Supervised Models: Random forest, Xgboost, BartMachine, SVM (Support Vector Machines), TensorFlow/PyTorch (NN, RNN, CNN, LSTM), Gurobi, NetworkX, GLM (Linear, Logistic, Poisson regression), KNN clustering

Unsupervised Models: PCA (Principal Component Analysis), K-means clustering, Hierarchical clustering,

Highlighted Projects

NFL General Manager Analysis

Project outline
- Develop a model to predict which General Manager is most likely to take a specific player based on their pre-draft information.
Data Used
- For this model I used data from the 2003 through 2024 drafts along with labels for the team and GM associated with that team. The GM labels were found through researching the team pages and pfr, with the draft pick information coming from nflfastr.
Features Included
- Athletic testing information: This includes the measurements of the players and their testing numbers from the combine. Additionally, there is a column for estimated forty yard dash time so that even players without combine data would still have some athletic information. Missing values were imputed with the median.
- NGS scores: These are the athleticism, production, and final scores from Next-Gen Stats for the prospects. These have been trained using an XGBoost on if the player was likely to be a starter or pro-bowler and I was part of the most recent training of these models.
- Other Features
  - Consensus Big Board information: Taking the consensus big board rank and converting it into an imputed grade scraped from NFL scouting grades
  - Underclassman label: If the player declared for the draft as an underclassman. This is as close of a proxy for age as I could easily find.
  - Conference label: Label for if the player was in school at the Power 5 or Group of 5 level in their last year. My conversion table is from a few years ago hence the 5 and not 4 labels, but given that the data goes back further and I didn't want to create a conversion table for each season, this was good enough for me.
Model Approach
- I utilized an XGBoost classifier trained to predict on the label of General Manager. My data was split into 60% train, 20% validation, and 20% test with stratified layers to ensure that each GM has data in test and train. I also included sample weights to force some the model to care more about more recent drafts and GMs with an exponential function centered on 2021 with an alpha of 0.5 (2024 = 4.48, 2021 = 1.0, and 2003 = 0.00012).
- I did a grid search for hyperparameter tuning with the average_precision_score as the eval metric and settled on 'colsample_bytree': 1.0, 'learning_rate': 0.01, 'max_depth': 3, 'n_estimators': 200, 'subsample': 0.8
- The feature importance plot for the top 10 features is shown below and I feel good that the model didn't latch too heavily onto any individual feature to dominate the predictions.
- Model Performance: log_loss: 4.2819, top_1_accuracy: 0.0279, top_5_accuracy: 0.1540. The model didn't perform particularly effectively, though I feel it was more accurate on recent data, which was the point of the exercise. I also don't feel that it was overfit as the test error rates were actually better than the validation error rates (log_loss: 4.3468, top_1_accuracy: 0.0425, top_5_accuracy: 0.1222)
Model Outputs
- I generated a handful of outputs from this model including the obvious predictions on the upcoming class as shown below. Additionally, I generated tables for all players in the sample with their top 5 most likely predictions along with the actual GM that drafted that player and a table of all the players by if a GM was in their top 5 predictions at all, for easier quick sorting and filtering.
- Additionally, I created a few SHAP plots to aid in model explainability and further understanding of how a particular GMs predictions were created.
- I also wanted to see the General Manager's likely types and how they compared to other GMs so I created a principal component plot with the first two principal components (explained 70% of the total variance) and a table showing their average features across all their picks.
- All of these outputs are available in a streamlit app that I created to show off these tables and plots which can be found here: https://nfl-gm-analysis.streamlit.app
Project Summary and Observations
- Overall, I would say that an analysis like this is very challenging since there are all kinds of bias and difficulties with availability that occur in the draft, but I feel good about this model. The model doesn't have particularly great evaluation metrics with a pretty poor roc_auc overall, but the model is more accurate on more recent draft picks which was the purpose of this exercise. Not to mention the fact that player body types have been changing as well so older GMs are on average drafting bigger players than newer GMs.
- It is very interesting to see the clear body type preferences that certain GMs have, like Buffalo's Brandon Beane having a clear preference for taller, bigger wide receivers and the Raven's Eric DeCosta having the opposite with a lot of smaller, lighter wide receivers being predicted to be taken by them.

NFL Draft Modeling

Project outline
- Develop models to predict Wide Receiver and Running back success at the NFL level from their college production profile
Data Used
- For this analysis I used data from PFF from the 2014 through 2024 college season as well as athleticism data from the combine. Additionally I used NFL data from PFF to generate the target variable.
Features Included
- Athletic testing information: This includes the measurements of the players and their testing numbers from the combine. Missing values were imputed using KNN based on their college production profiles to get a more accurate account of the body types associated with that profile. Additionally the athleticism score from NGS was utilized in the WR model.
- Volume and Efficiency metrics: This changed depending on the model, but I utilized a mixture of volume metrics (total attempts, total routes run) as well as efficiency metrics (yards per route run, explosive rush rate, etc.). I also generated features for their best and worst season to try and capture players with a breakout season that might not be captured by a raw mean.
- Scouting Report Data from Dane Brugler: For the WR model I incorporated some sentiment analysis on the scouting reports from Dane Brugler in his infamous "Beast" draft guides. I took the strengths and weaknesses sections and processed them using NLP and ran through a custom sentiment analysis for different aspects like playmaking, athleticism, and off-field. This adds further dimension to the model and boosted the predictive power and removed outlier misses.
- For a more thorough breakdown of the features used, please check out my streamlit app with the collapsible definitions menu. https://nfl-gm-analysis.streamlit.app/
Model Approach
- I utilized an XGBoost model trained on a composite target variable comprised of a weighted average of volume and efficiency metrics at the NFL level over a player's first four seasons. I also passed in a sample weight that was scaled using the logit function to the target variable because I am more interested in predicting the top end players rather than correctly predicting lower value players. I also ensured that the features had monotonic constraints placed on them to ensure that the model would learn a feature was important without finding pockets where a slower 40 yard dash time could be correlated to better players and rewarding slower 40 times for example.
- I did a grid search for hyperparameter tuning with the rmse as the eval metric and settled on these parameters
  - WR: [learning_rate: 0.05, max_depth: 5, num_estimators: 100, col_subsample: 1.0, subsample: 0.8]
  - RB: [learning_rate: 0.05, max_depth: 5, num_estimators: 100, col_subsample: 0.8, subsample: 0.8]
- I evaluated the models on a few criteria including their rmse and r^2 values which can be found for the test set below. I also plotted the correlation between predictions and actual values to get a gauge of the model in terms of over or under predicting on average. I also compared these models with the validation set (train 60%, validation 20%, test 20%) to ensure the model could generalize properly and wasn't overfitting. I also tried leave-one-season-out cross validation with no real boost in model performance.
  - WR: [R^2: 0.350, rmse: 0.182, mae: 0.140]
  - RB: [R^2: 0.220, rmse: 0.187, mae: 0.145]
Model Outputs
- From these core models I wanted to create a bit of a distribution of outcomes based on how closely the player performed to their "true" athleticism at the combine. To accomplish this I stepped the testing results up and down and re-ran the model with the same production to generate a range of predicted outcomes. From this I could generate a distribution and highlight players who might could have higher ceilings in the NFL if their on-field athleticism is higher than their listed scores. Additionally, I can identify players with higher floors who would have similar projections even with slightly worse on-field athleticism.
- I also output the predictions from the model and the target variable for all the players in the sample and discuss some of the patterns in the consistent over and under predictions between the models.
- Finally, I generated a number of SHAP plots to aid in the explanability of the model. This was in the form of bar plots, beeswarm plots, scatter plots, and waterfall plots.
Project Summary and Observations
- Overall I feel really good about these models, particularly the WR model. I have felt confident in my WR rankings for a while by utilizing cutoffs and thresholds for these productions numbers, but having a model and feeling confident in its projections is much nicer. Overall the over-predictions tend to be guys who produced well at the college level, often at a group of 5 school and then couldn't step up their game as much. The under-predictions were a bit more frustrating, but I would rather lean towards false negatives than false positives in this case. A decent number of those eventual hits had higher max predictions, highlighting the higher upside if they could play slightly more athletic than they tested.
- The RB model is also pretty good and I feel pretty good about the predictions moving forward, though the model isn't as good as the WR model. I had more notable misses in both the over and under prediction groups and there wasn't as clear of a trend for some of the bigger names. You could apply the same logic as above for some of the under predicted players with higher upside, but I feel a bigger issue is simply one of sample size and opportunity cost. Most teams play with only 1 running back on the field at a time so there are simply fewer players with significant snaps at the NFL level compared to the 3 wide receivers on the field for most plays. Additionally, if you are drafted to a team with a good running back you likely won't see the field to earn snaps because you are replacing a better player than replacing the 3rd wide receiver. Given those two facts, the output of the model is pretty good all things considered and I think it does a good job of highlighting talent and upside and it relies more on the situation for running backs than receivers.