Perception/semantic understanding 

We created a framework for commanding the robot to perform moving actions in conjunction with the vision team. These included moving forwards and backwards and rotating clockwise and counterclockwise and stopping. Each motion could be executed at a velocity specified by the caller.

The framework would allow communication with the vision team via a ROS2 topic subscription to search for the block, and once detected, navigate towards it. Upon reaching the block, the robot would pick it up, search for the bin, and navigate towards it.

By implementing the navigation pipeline in this modular fashion, we were able to test the software separately and integrate it cleanly once it was working reliably.

Challenge

The numerical IK solver exhibited a significantly high failure rate when attempting to reach objects located far from the robot's current position. Over these long Cartesian distances, the solver frequently became trapped in local minima, failing to converge on a valid joint configuration regardless of the position tolerance.

Solution

Implemented a "Hover Point" (approach waypoint) strategy. By first routing the end-effector to a safe, elevated position directly above the target, the long trajectory was broken down into shorter, highly solvable segments. This approach bypassed the local minima traps, dramatically improving both the IK success rate and the overall reaching reliability.

Challenge

 The numerical IK solver exhibits a significant trade-off: tightening the position tolerance (ik_dt) improves Cartesian accuracy for distant targets but dramatically increases the IK failure rate (convergence fails in local minima traps). Conversely, relaxing the tolerance improves IK solvability but often leads to real-world grasp failures due to excessive positional error at the target object.

Solution 

Implemented an "Adaptive Dynamic Tolerance" retry loop. The manipulation logic was adjusted to adopt a relaxed tolerance for the initial attempt, prioritizing a valid, collision-free trajectory over perfect precision. Upon grasp verification failure, subsequent attempts automatically tighten the tolerance (threshold reduction), iteratively enhancing positional accuracy until a successful enclosed grasp is confirmed.

Challenge 

Reliably confirming a successful grasp is critical for autonomous state transitions. Without dedicated force or tactile sensors, the robot must distinguish between an Empty Grasp (where the fingers close completely) and a Successful Grasp (where an object physically blocks the fingers from closing).

Solution 

We implemented a software-based check that monitors the discrepancy between the Gripper Command and the actual Joint State.

• The Logic: If the system sends a CLOSE command but the encoders report a state that is NOT CLOSED, the system identifies that an object is present.

• Thresholding: A small tolerance is used to account for encoder noise, ensuring that any state significantly different from "fully closed" is registered as a successful pick.

Verification Workflow

1. Command: Send CLOSE signal to the gripper.

2. Wait: Allow time for the gripper to reach its physical limit.

3. Compare: * If Command == State (Both Closed) -> Missed Grasp. Trigger retry loop.

   • If Command != State (Blocked) -> Grasping Detected. Proceed to Navigation.