Machine Learning and B.E.R.T.H.A

The mixture of computing, statistics, and baseball. Something that's as good of a mixture as ice cream, hot fudge, and a banana.

Using computers and advanced statistics in baseball isn't something remotely new now a days. But what we use the math on is changing. New technologies like PitchFX and Trackman have made in-game data collection analysis easier, more efficient, and allowed for the collection of data that is hard or impossible to see with the human eye. This has lead to two data points that have exploded onto the seen of baseball in the past few years: launch angle and exit velocity.

If you don't know what those are, I'll do a quick explanation of both. Exit velocity is the velocity of the ball after contact is made, normally in miles per hour. Launch angle is the angle created when the ball leaves the bat at contact. Both of these are recorded on PitchFX and Trackman during games.

This post will be about using machine learning in R and using launch angle and exit velocity to predict the play result of that hit, either a single, double, triple, home run, or an out. I'll be using three different models during the post, and compare all three to each other.

B.E.R.T.H.A is what I call these machine learning graphs and algorithms I use for this project. It means Big Efficiency Run Time on Hitting Algorithm. This as been a running joke with some of the other manager here at Iowa.

DAta/Packages used

The data gather is from Baseball Savant, and is all the data from the 2018 regular season and postseason games. I used R and RStudio to help run machine learning algorithms on the data. These algorithms came from the packages "e1071", "plyr", "caret", and "plotly". Another resource I used is an article found on FanGraphs, written by "The Kudzu Kid". The article is the main stepping stone to this project. I'll post all websites and packages used on the bottom of the article as a work cited/reference page.

First MODEL: SVM

The first algorithm we will look at is the support vector machine, or SVM. This one of the six algorithms that are popular in classification task. SVM uses hyperplanes to classify each of the classes in the data. The classes that we're using are the play results of the hits, and we're using launch angle and exit velocity as the hyperplane slopes.

First, I sorted the data into balls put into play. After this, I ran put the data through the "svm" function, and it ran for about 20-30 minutes before it was finally finished. After this, I used the model created to predict training data, which was then compared against testing data from the actual data set in a confusion matrix. We see in the matrix that the model has a accuracy of 76.18%, meaning it roughly predicted three of every four right. Looking at the class accuracy, we see that it really struggled to predict doubles. It's accuracy was 53.41%, which was about 20 percent lower than the rest (minus triples, which the model was unable to predict any in the testing data). It did predict home runs relatively well, with an accuracy of 82.39%

This graph shows a visual of the data above, but instead the model is used to predict the play result by launch angle and exit velocity. Looking at it, you can see the hyperplanes, especially for singles and doubles.

Here is a link for an interactive version of the plot: http://rpubs.com/wgentz22/441228

SECOND MODEL: RANDOM FOREST

The second algorithm we'll use is random forest. This is another one of the six algorithms that are popularly used for classification. In random forest, it uses lesser learning algorithms together to build a stronger combined model. The lesser algorithm is a decision tree, and when combined together create a "forest".

Again, I used the same data sets, but this time instead of using the "svm" function, I used a random forest method (the FanGraphs article shows how to do this in R). It was another 20-30 minutes to let the algorithm run through the data to create the model. This time, we see the model has an accuracy of 73.95% a little bit worse than using SVM model, but still usable. Again, the double the model struggles to predict double, but with an accuracy of 58.64%, it is an improvement from SVM. In this model, it also predicts triples inside the testing data, a 1% increase from SVM, to 51.05%. These two struggle to keep up with outs, home runs and singles. However, the accuracy of these three classes dropped about 2% for each class.

This is a visualization of the model using random forest, like the one above. Comparing this graph to the one made using SVM, you can see the same regions on the graph. However, because it is less accurate, there is a lot of "noise" on the graph. Because of the inaccuracy, the predictions on each speed and launch angle are also more inaccurate.

Here is a link for an interactive version of the plot: http://rpubs.com/wgentz22/441248

Final Model: Naive Bayes

The final algorithm is naive Bayes, which is based on Bayes statistics and theorem. This algorithm is used more in real time situations to calculate probabilities for fraud detection or spam filters.

Again, I used the same data, and ran it through the algorithm to create the model, this time it took about 10 seconds to finish. To create the model, we used "naiveBayes" function from the "e1071" package. In this model, we see an accuracy of 69.78%. This is nice, however I would not use this model as a predictor. The only reason I would use this model is for help predicting home runs, as is accuracy is 85.74%, higher than both the random forest and SVM models. The model is unable to predict any triple or doubles in the testing data, and is very inaccurate for singles and outs.

This is the visualization of the model using Bayes. Comparing this two the other two, we see that it has the singles move to the way back of the exit velocities, and we see a full circle around 100 mph to 110 mph and 20 degrees to 35 degrees. This circles is closely related to the other two for home runs, but is more accurate according to the confusion matrix.

Here is an interactive version of the graph: http://rpubs.com/wgentz22/441268

Conclusions

There are a few conclusions we can draw about the algorithms:

If we were to do this again, I would use the Random Forest algorithm as for a collective predictor for all the factors. This is because this was the only model to be able to make predictions on all five factors. However, SVM was a more accurate model as a whole, but you couldn't use this model for all five factors.
For individual factors, I would use Bayes ONLY for home runs, SVM for outs and singles, and finally use random forest for doubles and triples

Now for the conclusions about the data and what we see from the models:

It's hard to predict for doubles and triples. This is most likely because of different parks and also the hitters all have a different speed running the bases. If we were to do this again, I'd most likely only split into hits that stayed in the field of play, outs, and home runs.
The harder the ball is hit, the better the probability that it will end up being a hit. At about 100 mph, it will most likely be a hit regardless of launch angle.
For singles on the SVM and Random Forest models, it looks like it forms an asymptote starting at 60 mph and 40 degrees and going down to 120 mph and 0 degrees.

This was a really interesting project for me. Using different algorithms for machine learning on the data was fun, because when I had done this before I had only use Random Forest. Now, knowing what I know, I would use the different algorithms to better predict the results we would want to happen.

Let me know if you have any questions or comments through my direct messages on Twitter. Thanks for reading!

GITHUb/References

Link to a GitHub for the model building: https://github.com/wgentz/MLBertha/blob/master/RCode

Carson Sievert, Chris Parmer, Toby Hocking,

Scott Chamberlain, Karthik Ram, Marianne

Corvellec and Pedro Despouy (2017). plotly:

Create Interactive Web Graphics via

'plotly.js'. R package version 4.7.1.

https://CRAN.R-project.org/package=plotly

David Meyer, Evgenia Dimitriadou, Kurt

Hornik, Andreas Weingessel and Friedrich

Leisch (2017). e1071: Misc Functions of the

Department of Statistics, Probability Theory

Group (Formerly: E1071), TU Wien. R package

version 1.6-8.

https://CRAN.R-project.org/package=e1071

Hadley Wickham (2011). The

Split-Apply-Combine Strategy for Data

Analysis. Journal of Statistical Software,

40(1), 1-29. URL

http://www.jstatsoft.org/v40/i01/.

Max Kuhn. Contributions from Jed Wing, Steve

Weston, Andre Williams, Chris Keefer, Allan

Engelhardt, Tony Cooper, Zachary Mayer,

Brenton Kenkel, the R Core Team, Michael

Benesty, Reynald Lescarbeau, Andrew Ziem,

Luca Scrucca, Yuan Tang, Can Candan and

Tyler Hunt. (2018). caret: Classification

and Regression Training. R package version

6.0-79.

https://CRAN.R-project.org/package=caret

The Kudzu Kid. “Basic Machine Learning With R (Part 3).” FanGraphs Baseball, 7 Mar. 2017, www.fangraphs.com/community/basic-machine-learning-with-r-part-3/.

All data was gather from Baseball Savant (baseballsavant.mlb.com)

Google Sites

Report abuse