In inaccessible environments with uncertain task demands, robots often rely on general-purpose tools that lack predefined usage strategies. These tools are not tailored for particular operations, making their longevity highly sensitive to how they are used. This creates a fundamental challenge: how can a robot learn a tool-use policy that both completes the task and prolongs the tool’s lifespan? In this work, we address this challenge by introducing a reinforcement learning (RL) framework that incorporates tool lifespan as a factor during policy optimization. Our framework leverages Finite Element Analysis (FEA) and Miner’s Rule to estimate Remaining Useful Life (RUL) based on accumulated stress, and integrates the RUL into the RL reward to guide policy learning toward lifespan-guided behavior. To handle the fact that RUL can only be estimated after task execution, we introduce an Adaptive Reward Normalization (ARN) mechanism that dynamically adjusts reward scaling based on estimated RULs, ensuring stable learning signals. We validate our method across simulated and real-world tool use tasks, including Object-Moving and Door-Opening with multiple general-purpose tools. The learned policies consistently prolong tool lifespan (up to 8.01× in simulation) and transfer effectively to real-world settings, demonstrating the practical value of learning lifespan-guided tool use strategies

Overview of the proposed method integrating lifespan-guided reward into reinforcement learning.  During each rollout, the agent interacts with the environment, collecting state, action, force, and task reward information. At the end of each episode, the stress history is calculated via finite element analysis (FEA) and processed with Miner’s rule to estimate the remaining useful life (RUL). The RUL value is stored in a history buffer, which is used by the adaptive reward normalization (ARN) mechanism to determine dynamic upper and lower bounds. These bounds are applied to normalize the life reward for subsequent episodes, ensuring stable and meaningful reward signals for policy learning.