The steps for Grasping and Object Placement were very similar, and thus both handled by the Control Node, which kept track of whether the robot was trying to grab an object or place an object in order to handle the minor differences in steps appropriately. The process of Grasping/Placing an object starts with the Control Node receiving a pixel coordinate from either the Vision Node (if trying to grab an object or place an object in the basket) or Palm Detection Node (if trying to place an object in a hand). These pixel coordinates are used (alongside code that aligns the depth camera to the RGB camera) to extract the X coordinate of the target with respect to the camera frame (Trossen made the unorthodox decision to have the X-axis represent depth instead of the Z-axis). Using this X-coordinate and the RGB camera intrinsics, the pixels from before could be computed into Y and Z coordinates in the camera frame. A ROS2 Frame Listener was then used to convert these coordinates into Base Frame Coordinates, which was necessary since Trossen's functions for controlling the End Effector required the desired pose to be in the Base Frame.
Once the target's coordinates were inside the workspace of the robot, object grasping/placement could commence. The first step was to place the end-effector 0.1 meters above the target. This offset was needed to prevent the end effector from crashing into the target and pushing the target away. Next, the gripper was opened. If the robot was trying to grab an object, the end effector would then be lowered to the position of the object, and then the gripper was closed. If the robot was trying to place an object, then the act of opening the gripper accomplished the task of placing the object, so the end effector was closed right away (the end effector was not lowered like in the previous scenario). Finally, the end effector was sent back to the home position so that it wouldn't obstruct the camera's view.
For all actions, the end effector was oriented such that the gripper faced downwards, with gripper motion parallel to the robot's Y-axis. This orientation made for a simple grasp that could easily grab any item small enough along all axes to fit inside the gripper.