An Empirical Evaluation of Automated Function Points

Background: Function point analysis (FPA) has become widely used to measure software functional size in the industry. FPA is usually performed manually, which is a time consuming and expensive process. Automated Function Point (AFP) specification states the guidelines for automating FPA counting from software source code, generally consistent with current FPA practices. Objective: This paper analyzes the relationships between FPA and AFP measures and measurement process. This includes total function points and basic functional components (BFC) size. The goals are to evaluate accuracy, reproducibility, perceived ease of use, usefulness, and intention to use and to understand the nature of the differences between measures. Method: A controlled experiment was conducted to evaluate the FPA and AFP measurement process with 14 participants (E1). We have subsequently conducted two replications of the original experiment with 12 participants each (E2 and E3). Statistical analyses were conducted to find differences between the methods regarding effectiveness and adoption properties. Results: Participants using FPA performed as well as participants using AFP to measure total functional size. Treatment and control groups did not differ significantly in terms of accuracy and reproducibility (p{\textless}0.05) throughout E1, E2 and E3. FPA presented similar accuracy to AFP for each replication. Participants between experiments performed different; data function (DF) results show an important difference between methods. Conclusions: The results confirm the potential for developing automation tools for function point counting that could produce more consistent measurement results; however, this process should be conducted carefully, depending on the implementation of a given requirements specification, AFP could produce different results. Although encouraging results were obtained, further research is needed to confirm performance results and to draw more conclusions on the perceived adoption properties.