RQ3

Experiment Setup

To address RQ3, we investigate the prevalence of Spurious Guessing which refers to instances where models predict the correct final label despite failing to provide a valid reasoning trace. This analysis aims to quantify the discrepancy between outcome-based accuracy and true reasoning performance. If a model relies on statistical shortcuts rather than logical deduction, outcome-based metrics serve as an inflated proxy for actual performance. Table illustrates the Spurious Guessing Rate across different difficulty levels. 

Result