Embedded Files
Updated Weighted Errors Project in R (AUG 2024)
Updated Weighted Errors Project in R (AUG 2024)
Pieces of code can be found here: https://github.com/duffyd08/Weighted-Errors
Goals:
1. Find a way to quantify "errors" based on their game impact
2. Use this aspect of fielding to create a more overall understanding of a defender's ability
3. Produce data on which fielders commit more costly errors and how we can use that to evaluate defensive effectiveness
4. Create a second order effect to continue the stream of thought and the "trickle down" to other players with this model
As I test out my error and win probability model into R, I have come to some new understandings about my original processes that started over two years ago.
After releasing the first iteration of my project on “Weighted Errors”, I was very fortunate to receive some incredible feedback from many people within the game of baseball. MLB front office members, professional and college coaches, and even players offered insight into what they thought about the statistic, how they thought it could be applied, and most importantly, what needed to be improved going forward. I have spent some time over the past two year looking into ways to iterate my methods. First, I needed to make the process more efficient. One of the first individuals in a front office that I spoke with wondered if I had the capabilities to automate this process. At the time, I did not. Having spent a few months in a graduate analytics program, I have now been able to take some of my ideas into R. This has allowed me to produce visualizations, extract insights, and work with large datasets in a much easier way. Second, though the idea of “counting bases” is the way that I see the impact of an error on the field, combining leverage index and bases has a very similar effect to WPA. Now, I have used win probability added to be the main metric in calculating my “weighted error” view. Third, many issues with the idea was that it would penalize players who are able to reach balls but may make an error, over their counterparts who do not have the same “range”. I had originally considered this as a supplement to defensive evaluation, not the entire story. This project will not replace what we know about range, but it might help with teaching our different players about their tendencies to limit mistakes.
Underlying Issues in my Process:
1. Inconsistency in Data
2. Error Totals Do Not Align Across All Sources
3. Only a few weeks into R
4. Issues with Out Probability Model
I have decided to try to quantify players on their overall defensive impact (defensive win probability added) and see which players are creating more outs than expected. My out-probability model is still very nascent and quite basic due to lack of access to data that would be more indicative of out making ability. Still, I think it helps to see some of the differences between overall outs over expected numbers and the same view on plays that were deemed errors. The out probability model below has stark differences between infielders and outfielders. My main takeaways:
1. Infielders have higher WPA and lower Outs over Expected. This is likely due to volume of opportunities (which would increase WPA over time and decrease Outs over Expected). On the flip side, OOE reflects the defensive value relative to what is "statistically" expected. Infielders, due to their frequent involvement, might not always exceed the expected outs by a large margin in every play. This results in a lower OOE because their defensive plays are often expected and less likely to significantly exceed expectations
2. Outfielders have the opposite of their infield counterparts (which, again, should make some sense). In this model, the outs are judged mostly based on xWOBA and hit location. If there is a high expected WOBA play that is hit directly at an outfielder, he gets credited with an out over expected, regardless of how much he has to move. But, their WPA takes a hit because of their frequent involvement in scoring plays (singles, doubles, triples hit on the ground or in the air to them), where they may not have had a real chance to convert an out, but get credited with the full change in WPA.
Below, we see the same scatterplots, though focused only on plays that were deemed errors. The outfielders see a much different trend and much tighter distributions. The third basemen and shortstops see a lot more variation.
Second Order Effects:
During my outreach, I was told to look a little into the second and third order effects that may arise. I decided to try and quantify the trickle down effect of the pitchers who suffered a poor play in the field behind them. While an imperfect way to most accurately find the effect of defensive plays on pitchers, it does give a little bit more insight into how defenders are affecting pitchers aside from earned runs.
I calculated the average pitches per out for pitchers in the 2023 season by simply dividing their total pitches and outs pitched (3*IP). Then, I found the sum of each position player's errors for each pitcher. For this study, I have assumed that each error costs an opportunity for an out, but maybe it's too simplistic. Ex: Trea Turner committed 14 errors that came up on the Statcast data. Since each pitcher has a different "average pitches per out", I summed each to get his league leading 90.1 pitches.
Below is a histogram of the average pitches per out in 2023:
The scatterplot below shows the dWPA lost and total pitches cost for each team on plays deemed errors in 2023. The drawbacks to some of this data is that average pitches per out is a flawed metric, especially with a small sample size. There are pitchers who have tough outings and could even be sent to the minor leagues, denoting a high number of pitches. I included the standard deviations for pitches cost for each team. Interesting to note that Arizona and Texas (the two World Series teams) had 2 of the 3 lowest standard deviation figures. Perhaps just a coincidence :)
Next Steps:
1a. Continue to evolve the out probability metric to be more applicable
1b. Use a better out probability model to create my own definition of what an error is
2. Find trends for year over year, what players or types of players are committing worse errors
3. Continue my exploration in R and Python
------------------
Conclusions:
Again, I am still aware of the many shortcomings about my defensive model and where it can improve, but I am pleased with the work that I have done over the last few months to get this out to more eyes. I am excited about my potential in R as I continue to do these projects. Having only started in the program about two months ago, I know I am picking it up well.
I hoped to keep this page short and brief, as it should read a bit easier than my original findings. The brevity should allow for easier understanding and more focus on the visual aids and read more as an exploratory rather than a persuasive paper. As George E. P. Box once put, "all models are wrong, but some are useful". My hope is that this ever-evolving defensive model will bring some more substance to my original idea that I came up with over 2 years ago. I am looking forward to tweaking this in an effort to tell more of a story in a season, player, or play type through numbers and cannot wait to hear more feedback.
As always, please leave your thoughts with me via email : duffydigest@gmail.com / djduffy4@gmail.com or on Twitter/X as I grow my presence there @DigestDuffy . While you are on the page, feel free to read my past work on Weighted Errors in 2022 here. Check out my profile page too, as I put more of what I am learning in class on that page and my recent defensive model for evaluating the best first baseman in the league as well.
Thank you for taking the time to read and explore with me!
-Drew
Page updated
Google Sites
Report abuse