RT info:eu-repo/semantics/article
T1 Analysing Agreement Among Different Evaluators in God Class and Feature Envy Detection
A1 Alkharabsheh, Khalid
A1 Alawadi, Sadi
A1 Crespo, Yania
A1 Manso, M. Esperanza
A1 Gonzalez, Jose A. Taboada
K1 Tools;Software;Code
K1 Feature extraction
K1 Maintenance engineering
K1 Formal concept analysis
K1 Design smell
K1 survey;empirical study;experiment
K1 inter-rater agreement;Kappa-Fleiss
AB The automatic detection of Design Smells has evolved in parallel to the evolution of automatic refactoring tools. There was a huge rise in research activity regarding Design Smell detection from 2010 to the present. However, it should be noted that the adoption of Design Smell detection in real software development practice is not comparable to the adoption of automatic refactoring tools. On the basis of the assumption that it is the objectiveness of a refactoring operation as opposed to the subjectivity in definition and identification of Design Smells that makes the difference, in this paper, the lack of agreement between different evaluators when detecting Design Smells is empirically studied. To do so, a series of experiments and studies were designed and conducted to analyse the concordance in Design Smell detection of different persons and tools, including a comparison between them. This work focuses on two well known Design Smells: God Class and Feature Envy. Concordance analysis is based on the Kappa statistic for inter-rater agreement (particularly Kappa-Fleiss). The results obtained show that there is no agreement in detection in general, and, in those cases where a certain agreement appears, it is considered to be a fair or poor degree of agreement, according to a Kappa-Fleiss interpretation scale. This seems to confirm that there is a subjective component which makes the raters evaluate the presence of Design Smells differently. The study also raises the question of a lack of training and experience regarding Design Smells.
PB IEEE
SN 2169-3536
YR 2021
FD 2021
LK https://uvadoc.uva.es/handle/10324/74392
UL https://uvadoc.uva.es/handle/10324/74392
LA eng
NO IEEE Access, vol. 9, pp. 145191-145211, 2021
NO Producción Científica
DS UVaDOC
RD 05-feb-2025