Demonstrating a Vision-Based AI Robot for Strategic Board Games

Human-Robot Interaction (HRI) system designed for seamless gameplay in Gomoku, integrating robotics and AI for dynamic interaction.

Board Calibration & Vision Monitoring Module

• Real-time game state detection using circle detection algorithms.

• Ensures accurate board updates for reliable gameplay.

Gomoku AI Decision-Making System

• Combines a policy-value network with penalized Monte Carlo Tree Search (MCTS).

• Enables strategic gameplay while accounting for physical constraints.

Robot Control Module

• Utilizes an Inverse Kinematics (IK) solver and MuJoCo for precise piece manipulation.

• Ensures smooth execution of robot moves on the gameboard.

The vision system ensures accurate and real-time detection of the Gomoku board state using traditional image processing techniques. It begins with board calibration through homography transformation, which aligns the board to a square grid for consistent tracking. The initial state is set by detecting grid lines using the Hough Transform and validating them for precision.

During gameplay, the system uses circle detection (Hough Transform-based) to identify the positions of stones on the board. Adaptive preprocessing techniques, such as Contrast Limited Adaptive Histogram Equalization (CLAHE), are applied to enhance visibility under varying lighting conditions. Detected stones are assigned to the nearest grid positions, ensuring accurate state updates while resolving conflicts like overlapping or misaligned detections. This robust and efficient pipeline provides the AI and robot modules with reliable input for seamless game progression.

The Gomoku AI is designed based on an AlphaZero-inspired framework, integrating a Policy-value network with Monte Carlo Tree Search (MCTS) for strategic decision-making. The policy network identifies the most promising moves, while the value network evaluates the strength of board positions. During self-play training, the AI refines its strategy by simulating games against itself, continuously improving its decision-making abilities.

The hardware and software are available at Github , promoting accessibility and further innovation.

[1] J. Ding, “ChessVision: Chess Board and Piece Recognition,” in Proceedings of the Conference, 2016. Available: \url{https://api.semanticscholar.org/CorpusID:11949747}

[2] A. Masouris and J. van Gemert, “End-to-End Chess Recognition,” arXiv preprint arXiv:2310.04086, 2023.

[3] D. M. Quintana, A. A. del Barrio García, and M. P. Matías, “LiveChess2FEN: A framework for classifying chess pieces based on CNNs,” arXiv preprint arXiv:2012.06858, 2020.

[4] C. Zheng, H. Wang, L. Zhang, J. Hao, R. Gomez, K. Nakamura, and G. Li, “Imitating Human Strategy for Social Robot in Real-Time Two-Player Games,” in International Conference on Social Robotics, Springer, 2022, pp. 427–438.

[5] D. Rato, F. Correia, A. Pereira, and R. Prada, “Robots in Games,” International Journal of Social Robotics, vol. 15, no. 1, pp. 37–57, 2023.