Descripción:
The specification and maintenance of high-quality user stories are critical in agile software development, yet they are often hindered by natural-language ambiguity, evolving business requirements, and the effort required for manual backlog refinement in industrial settings. This paper investigates the use of large language models (LLMs), specifically GPT-5.1, to support the automated evaluation and improvement of user-story quality using the INVEST framework. Building on prior expert-based assessments, we propose a human-in-the-loop procedure that combines LLM automation with requirements-engineering expertise. We conduct an industrial replication study using 49 real user stories from a scholarship‑management system, preserving the evaluation–improvement–reevaluation design of prior expert‑based work. Results show alignment between GPT-5.1 and expert judgments, particularly after an evaluation–improvement–reevaluation cycle, with strong semantic agreement and convergence in key INVEST dimensions. GPT‑5.1 assigns slightly lower scores than experts for Independent, Negotiable, Estimable, and Small, with moderate monotonic correlations (ρ≈0.53–0.65). After the improvement cycle, expert medians reach 5 across all INVEST criteria, and GPT‑5.1 converges strongly on Valuable and Testable while remaining more conservative on Independent and Small; semantic agreement exceeds 85–90% across most dimensions. These findings indicate that GPT‑5.1 could reduce manual assessment effort, reinforce structural quality, and support consistent requirements evaluation, while highlighting the complementary role of human oversight in industrial requirements‑engineering workflows.
Tipo de publicación: Conference Paper
Publicado en: Anais do XXIX Congresso Ibero-Americano em Engenharia de Software (CIbSE 2026)
Autores- Hernández-Agüero, Erika
- Quesada-López, Christian
- Chaves-Sánchez, José P.
Investigadores del CITIC asociados a la publicación
Dr. Christian Quesada-López
Proyecto asociado a la publicación