Reinforcement Learning (RL) in Propulsion Systems 01/2014- present

Research topics:

  • RL function mechanism interpretation based on hybrid propulsion system energy management application

  • Learning time reduction of RL via warm start (e.g., value/ policy functions initialization with expert knowledge or conventional control)

  • Adaptiveness investigation of RL over various environmental and vehicular conditions (e.g., activation and deactivation mechanism of adaptiveness process)

  • Ensemble RL (e.g., combination of various RL algorithms and conventional controls to improve robustness and optimality of the results)

  • RL Convergence study via recent Basal Ganglia Neuroscience studies (e.g., new computational model for RL learning end process, which will replace existing learning rate)

  • Unifying theories of tonic (action) and phasic (learning) Dopamine release considering neuroscience experimental study.

  • Investigation of sensory, learning, and motoring circuit in Basal Ganglia by combining neuroscience experimental data and computational models.

Featured Publication:

  • Xu, B., Rathod, D., Zhang, D.R., Yebi, A., Zhang X.Y., Li, X.Y., Filipi, Z., “Parametric Study on Reinforcement Learning Optimized Energy Management Strategy for a Hybrid Electric Vehicle”, Applied Energy, vol.259, p.114200, 2020. (PDF)

  • Xu, B., Hu, X.S., Lin, X.K., Li, H.Y., Rathod, D., Filipi, Z., “Ensemble Reinforcement Learning as a Hybrid Electric Vehicle Supervisory Control for Fuel Economy Improvement”, IEEE Transactions on Transportation Electrification, 2020, doi: 10.1109/TTE.2020.2991079. (PDF)

  • Xu, B., Hou, J., Shi, J., Li, H., Wang, Z., Rathod, D., and Filipi, Z., “Learning Time Reduction Using Warm Start Methods for a Reinforcement Learning Based Supervisory Control Strategy in Hybrid Electric Vehicle Applications”, IEEE Transactions on Transportation Electrification, 2020, doi: 10.1109/TTE.2020.3019009. (PDF)

  • Xu, B., J. Shi, S. Li, H. Li, and Z. Wang, “Energy consumption and battery aging minimization using a Q-learning strategy for a battery/ultracapacitor electric vehicle,” Energy, vol. 229, p. 120705, 2021. (PDF)

  • Xu, B., Tang, X.L., Hu, X.S., Lin, X.K., Li, X.Y., Rathod, D., and Wang, Z., “Q-learning Based Supervisory Control Adaptability Investigation for Hybrid Electric Vehicles, IEEE Transactions on Intelligent Transportation Systems, 2021, doi: 10.1109/TITS.2021.3062179. (PDF)

  • Xu, B., and Li, X.Y., “A Q-learning Based Transient Power Optimization Method for Organic Rankine Cycle Waste Heat Recovery System in Heavy Duty Diesel Engine Applications”, Applied Energy, 2021, doi: https://doi.org/10.1016/j.apenergy.2021.116532. (PDF)

Supervised Learning in Propulsion Systems 09/2017-04/2020

Research topics:

  • Development of Data-Driven Lithium-ion battery design assistant tool by building machine learning models connecting vehicle specification and battery design parameters

  • Real-time realization of offline Dynamic Programming optimization results by utilizing machine learning models to extract optimal rules from Dynamic Programming results

  • Multi-layer hierarchical ensemble machine learning to improve the prediction accuracy

Featured Publication:

  • Xu, B., Rathod, D., Yebi, A., Onori, S., Filipi, Z., “Real-Time Realization of Dynamic Programming Using Machine Learning Methods for IC Engine Waste Heat Recovery System Power Optimization”, Applied Energy, vol.262, p.114514, 2020. (PDF)

  • Xu, B., Rizzo, D., and Onori, S., “Machine Learning Based Optimal Energy Storage Devices Selection Assistance for Vehicle Propulsion Systems", SAE 2020 World Congress & Exhibition, 2020. (PDF)

  • Xu, B., Zhang, D., and Tang, S., “Malware Classification Utilizing Supervised Learning in Autonomous Driving Applications”, SAE - 19th Asian Pacific Automotive Engineering Conference (APAC), Shanghai, China, 2017. (PDF)

Exhaust Waste Heat Recovery on Heavy-Duty Diesel Engine Project (BorgWarner) 09/2013- 11/2020

Research topics:

  • Plant modeling and experimental validation using a 13L heavy-duty diesel engine and ORC test rig.

  • Control-oriented model and nonlinear model predictive control development.

  • Power optimization utilizing reinforcement learning for the benefits of model-free Q-learning.

  • Extracting rules from offline dynamic programming results using machine learning models and implementing the trained models in real-time.

Featured Publication:

  • Xu, B., and Li, X.Y., “A Q-learning Based Transient Power Optimization Method for Organic Rankine Cycle Waste Heat Recovery System in Heavy Duty Diesel Engine Applications”, Applied Energy, 2021, doi: https://doi.org/10.1016/j.apenergy.2021.116532. (PDF)

  • Xu, B., Rathod, D., Yebi, A., Onori, S., Filipi, Z., “Real-Time Realization of Dynamic Programming Using Machine Learning Methods for IC Engine Waste Heat Recovery System Power Optimization”, Applied Energy, vol.262, p.114514, 2020. (PDF)

  • Xu, B., Rathod, D., Kulkarni, S., Yebi, A., Filipi, Z., Onori, S., Hoffman, H., “Transient Dynamic Modeling and Validation of an Organic Rankine Cycle Waste Heat Recovery System for Heavy Duty Diesel Engine Applications,” Applied Energy, 205: pp. 260-279, 2017. (PDF).

  • Xu, B., Yebi, A., Hoffman, M., and Onori, S., “A Rigorous Model Order Reduction Framework for Waste Heat Recovery Systems Based on Proper Orthogonal Decomposition and Galerkin Projection", IEEE Transactions on Control Systems Technology, 2018, doi: 10.1109/TCST.2018.2878810. (PDF)

  • Xu, B., Rathod, D., Yebi, A., Filipi, Z., "A Comparative Analysis of Real-time Power Optimization for Organic Rankine Cycle Waste Heat Recovery Systems," Applied Thermal Engineering, vol. 164, 114442, 2020. (PDF)

  • Xu, B., Rathod, D., Yebi, A., Onori, S., Filipi, Z. and Hoffman, M., “A Comparative Analysis of Dynamic Evaporator Models for Organic Rankine Cycle Waste Heat Recovery Systems”, Applied Thermal Engineering. p. 114576, 2019. (PDF)

  • Xu, B., Yebi, A., Rathod, D., Onori, S., Filipi, Z., Hoffman, H., “Experimental Validation of Nonlinear Model Predictive Control for a Heavy-Duty Diesel Engine Waste Heat Recovery System", ASME Journal of dynamic systems measurement and control. vol. 142, Issue 5, 2020. (PDF)

  • Xu, B., Rathod, D., Yebi, A., Onori, S., Filipi, Z., Hoffman, H., “A Comprehensive Review of Organic Rankine Cycle Waste Heat Recovery for Heavy Duty Diesel Engine Applications", Renewable & Sustainable Energy Reviews. vol. 207, pp. 145-170, 2019. (PDF)

Hybrid Electric Vehicle Energy Management 12/2017 - 05/2019

Research topics:

  • Implementing real-time reinforcement learning based supervisory control to split the torque between engine and electrical motor.

  • Battery aging modeling using severity factor empirical method.

  • Proposing battery aging ECMS method by integrating fuel economy and battery life in the cost function.

  • Vehicle topology analysis with different propulsion architectures.

  • Battery, electrical motor and battery cooling fan size optimization using Particle Swarm Optimization and Latin Hyper Cube Sampling methods.

Featured Publication:

  • Xu, B., Malmir, F., Rathod, D., and Filipi, Z., “Real-time Reinforcement Learning Optimized Energy Management for a 48V Mild Hybrid Electric Vehicle", 2019-01-1208, SAE 2019 World Congress & Exhibition, 2019. (PDF)

  • Falmir, F., Xu, B., and Filipi, Z., “A heuristic supervisory controller for a 48V hybrid electric vehicle considering fuel economy and battery aging”, SAE International Powertrains, Fuels & Lubricants Meeting. 2019. (PDF)

Supervised Learning in Smart Manufacturing 01/2017 - 05/2020

Research topics:

  • Real-time products failure prediction based on 1000+ sensor signals collected in production line to reduce test cost and improve product quality using unsupervised and supervised learning

  • Real-time product quality monitoring in injection molding process to improve products quality using supervised learning

  • Raw data cleaning, data reduction, key feature extraction and data visualization.

  • Multilayer machine learning model construction and model selection. Hyper-parameter optimization.

  • Trained model real-time prediction implementation.

  • Cloud computing for the periodic model training and real-time optimization.

  • API in Linux Python Flask environment.

Featured Publication:

  • Farahani, S., Xu, B., Filipi, Z., Pilla, S., “A Machine Learning Approach to Quality Monitoring of Injection Molding Process Using Regression Models”, The International Journal of Computer Integrated Manufacturing, 2021, doi: https://doi.org/10.1080/0951192X.2021.1963485.

  • Zhang, D., Xu, B., and Wood, J. “Predict failures in production lines: A two-stage approach with clustering and supervised learning”, 2016 IEEE International Conference on Big Data, 2070-2074, Washington DC, 2016. (PDF)