RQ4 (User Study)

Demographic details

Average age: 22.56

Participants are undergraduate, master’s, and PhD students majoring in Computer Science, with an average age of 22.56, equivalent to late undergraduate or early postgraduate level.

Assistance contribution ratio: 2.94 / 5

The average contribution ratio of AI assistants was 2.94 out of 5, implying that AI tools played a supporting but not dominant role in participants’ coding processes.

Mostly used features: Code completion & Next edit prediction

Code completion is the most commonly used AI coding feature, followed by next edit suggestions and code explanation, while test case generation is the least used.

Seriousness of recommendation verification: 3.25 / 5

Participants demonstrated a mean verification seriousness score of 3.25 out of 5, suggesting that they generally review AI-generated edits with reasonable caution and attention.

Coding frequency: 4.86 days / week

Participants coded on average 4.86 days per week, reflecting frequent and consistent engagement with programming activities.

Education level: evenly distrubted

Undergraduate participants form the largest group at 37.5%, while Master’s and PhD participants are equally represented at 31.2% each.

Most time-consuming action: Mosifying suggestion

Modifying recommended edits is perceived as the most time-consuming activity, followed by interacting with AI in multiple rounds, while waiting for recommendations is considered the least time-consuming.

Estimated time saved by AI assistants: 5.94 / 10

On average, participants reported that AI assistants helped them save 5.94 out of 10 in perceived programming time, indicating a moderate level of efficiency gain.

Instrumentation-based analysis

EditFlow substantially reduces overall interaction rounds, indicating smoother and more efficient development flow.
Users view far fewer recommendations, suggesting reduced noise and less cognitive overload.
The total number of generated edit hunks decreases, showing that the system becomes more selective rather than more aggressive.
Acceptance rate consistently improves across systems, indicating better alignment with developers’ immediate needs.
Rejection and dismissal behaviors significantly decrease, suggesting fewer disruptive or mistimed recommendations.
Requests for the next edit are reduced, implying that recommendations arrive at more appropriate moments.
Average attention per viewed recommendation slightly increases, suggesting that each suggestion carries higher relevance and value.
Overall, EditFlow reduces interaction friction while improving recommendation quality, demonstrating stronger alignment with developers’ mental flow.

Page updated

Google Sites

Report abuse