A Formal Protocol to Conduct Usability Heuristic Evaluations in the Context of the Software Development Process

  • Abstract
  • Keywords
  • References
  • PDF
  • Abstract

    Heuristic evaluation is one of the most used techniques to evaluate the level of usability of a software product. In this research, we performed a comprehensive analysis of the recent studies which report the use of this method in the context of a software development process. The purpose was to identify the specific way in which each author performs this usability evaluation method, in order to propose a formal protocol. After an indeed examination of these studies, we have determined there are several differences in the way this technique is conducted according to the literature. There is no agreement about the number of inspectors that should participate, the usability principles that should be used, the profile of the specialists who must be part of the assessment team, or the evaluation process that should be followed. This work highlights the available settings and a detailed procedure to perform a heuristic evaluation in the domain of software products.



  • Keywords

    assessment protocol; heuristic evaluation; Human-Computer Interaction; usability evaluation methods; usability principles.

  • References

      [1] Davids MR, Chikte U, Grimmer-Somers K, Halperin ML, “Usability testing of a multimedia e-learning resource for electrolyte and acid-base disorders”, British Journal of Educational Technology, vol. 45, no. 2, 2014, pp. 367-381.

      [2] Desurvire H & Wiberg C, “Game usability heuristics (PLAY) for evaluating and designing better games: The next iteration”, Proceedings of the 13th International Conference on Human-Computer Interaction (HCI International 2009), 2009, pp. 557-566.

      [3] Díaz J, Rusu C & Collazos CA, “Experimental validation of a set of cultural-oriented usability heuristics: e-commerce websites evaluation”, Computer Standards & Interfaces, vol. 50, 2017, pp. 160-178.

      [4] Fernandez A, Insfran E & Abrahão S, “Usability evaluation methods for the web: A systematic mapping study”, Information and Software Technology, vol. 53, no. 8, 2011, pp. 789-817.

      [5] Hermawati S & Lawson G, “Establishing usability heuristics for heuristics evaluation in a specific domain: Is there a consensus?”, Applied Ergonomics, vol. 56, 2016, pp. 34-51.

      [6] Holzinger A, “Usability engineering methods for software developers”, Communications of the ACM, vol. 48, no. 1, 2005, pp. 71-74.

      [7] Hurtado N, Ruiz M, Orta E & Torres J, “Using simulation to aid decision making in managing the usability evaluation process”, Information and Software Technology, vol. 57, no. 1, 2015, pp. 509-526.

      [8] Inostroza R, Rusu C, Roncagliolo S, Rusu V & Collazos CA, “Developing SMASH: A set of SMArtphone's uSability Heuristics”, Computer Standards & Interfaces, vol. 43, 2016, pp. 40-52.

      [9] Isbandiputra IN, Santoso HB & Hasibuan ZA, “Usability evaluation of Indonesia open educational resources using multi-methods”, Journal of Engineering and Applied Sciences, vol. 12, no. 12, 2017, pp. 3272-3276.

      [10] ISO, Ergonomic requirements for office work with visual display terminals (vdts), -- Part 11: Guidance of usability, ISO Std. 9241-11:1998, International Organization for Standardization, 1998.

      [11] ISO, Software engineering – Product quality – Part 1: Quality model, ISO Std. 9126-1:2001, International Organization for Standardization, 2001.

      [12] Ivory MY & Hearst MA, “The state of the art in automating usability evaluation of user interfaces”, ACM Computing Surveys, vol. 33, no. 4, 2001, pp. 470-516.

      [13] Nagpal R, Mehrotra D & Bhatia PK, “Analytical modelling approach to measure the usability of website”, International Journal of Software Engineering and Its Applications, vol. 10, no. 5, 2016, pp. 125-142.

      [14] Nielsen J, 10 Usability Heuristics for User Interface Design, 1995, available online:


      [15] Nielsen J, Usability engineering, 1st ed., San Diego, CA, USA: Academic Press, 1993.

      [16] Nielsen J, “Usability inspection methods”, Proceedings of the Conference Companion on Human Factors in Computing Systems (CHI’94), 1994, pp. 413-414.

      [17] Paz F, Paz FA & Pow-Sang JA, “Experimental case study of new usability heuristics”, Proceedings of the 17th International Conference on Human-Computer Interaction (HCI International 2015), 2015, pp. 212-223.

      [18] Paz F, Paz FA, Villanueva D & Pow-Sang JA, “Heuristic evaluation as a complement to usability testing: A case study in web domain”, Proceedings of the 12th International Conference on Information Technology: New Generations (ITNG 2015), 2015, pp. 546-551.

      [19] Paz F & Pow-Sang JA, “A systematic mapping review of usability evaluation methods for software development process”, International Journal of Software Engineering and Its Applications, vol. 10, no. 1, 2016, pp. 165-178.

      [20] Paz F & Pow-Sang JA, “Current trends in usability evaluation methods: A systematic review”, Proceeding of the 7th International Conference on Advanced Software Engineering & Its Applications (ASEA 2014), 2014, pp. 11-15.

      [21] Paz F, Villanueva D, Rusu C, Roncagliolo S & Pow-Sang JA, “Experimental evaluation of usability heuristics”, Proceedings of the 10th International Conference on Information Technology: New Generations (ITNG 2013), 2013, pp. 119-126.

      [22] Ten AC, Paz F & Pow-Sang JA, “A framework for the evaluation of user experience in information-driven websites”, Journal of Engineering and Applied Sciences, vol. 12 (special issue 5), 2017, pp. 7093-7097.

      [23] Whitefield A, Wilson F & Dowell J, “A framework for human factors evaluation”, Behaviour & Information Technology, vol. 10, no. 1, 1991, pp. 65-79.




Article ID: 12874
DOI: 10.14419/ijet.v7i2.28.12874

Copyright © 2012-2015 Science Publishing Corporation Inc. All rights reserved.