Abstract
Digital maturity models are increasingly used by organizations to evaluate their transformation capabilities and define strategic roadmaps. However, despite their popularity, little is known about the consistency and psychometric soundness of the most widely adopted self-assessment instruments. This study has two objectives: (1) to empirically demonstrate that the results obtained from different digital maturity questionnaires vary significantly for the same organization, and (2) to establish an objective benchmarking framework to evaluate the quality of these tools. The analysis focuses on three prominent models: PwC, MinTIC & iNNpulsa, and MIT & Capgemini. Based on a structured case study and survey responses from 90 non-expert participants and 15 digital transformation experts, we assess each instrument using four key criteria: uncertainty, internal consistency, theoretical validity, and suitability for self-assessment. Results reveal statistically significant discrepancies between instruments and highlight the limitations of relying on unvalidated tools for strategic decision-making. The proposed benchmarking framework offers a practical and replicable approach to guide the selection and development of digital maturity instruments, especially in resource-constrained contexts such as small and medium-sized enterprises (SMEs).
References
Acevedo A. (2018) Modelo de Madurez para la Transformación Digital, iN-Npulsa Colombia, Bogotá Desarrollo Empresarial MINTIC –Subdirección de Comercio Electrónico.
Aghazadeh H., Zandi F., Amoozad Mahdiraji H., Sadraei R. (2024) Digital transformation and SME internationalisation: Unravelling the moderated-mediation role of digital capabilities, digital resilience and digital maturity. Journal of Enterprise Information Management, 37(5), 1499–1526. https://doi.org/10.1108/JEIM-02-2023-0092
Berghaus S., Back A. (2016) Stages in digital business transformation: Results of an empirical maturity study. Paper presented at the 10th Mediterranean Conference on Information Systems (MCIS 2016), Paphos, Cyprus, 4-6 September 2016.
Bonett D.G. (2002) Sample size requirements for testing and estimating coefficient alpha. Journal of Educational and Behavioral Statistics, 27(4), 335–340. https://doi.org/10.3102/10769986027004335
Cerchione R., Esposito E. (2017) Using knowledge management systems: A taxonomy of SME strategies. International Journal of Information Management, 37(1), 1551–1562. https://doi.org/10.1016/j.ijinfomgt.2016.10.007
Chae H.C., Chang E.K., Kwang O., Park K.O (2017) Information technology capability and firm performance: Role of industry. Information & Management, Volume 55, Issue 5, July 2018, Pages 525-546. https://doi.org/10.1016/j.im.2017.10.001
Cohen J. (1988) Statistical Power Analysis for the Behavioral Sciences (2nd ed.), New York: Routledge.
Cronbach L.J. (1951) Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
Dimitrov D.M. (2010) Testing for factorial invariance in the context of con-struct validation. Measurement and Evaluation in Counseling and Development, 43 (2) 121–149. https://doi.org/10.1177/0748175610373459
Field A. (2013) Discovering Statistics. Using IBM SPSS Statistics (4th ed.), Thousand Oaks, CA: SAGE.
Gill M., VanBoskirk S. (2016) The digital maturity model 4.0, Cambridge, MA: Forrester.
Goetzinger P., Spremić M., Jaković B. (2025) The Role of Digital Leadership Capabilities in Enterprise-Wide Digital Transformation. Foresight and STI Governance, 19(2), 68–76. https://doi.org/10.17323/fstig.2025.27129
Griffin H., Kühnis N., Warnking P. (2016) Digitalisierung – wo stehen Schweizer KMU, Geneva: PwC Schweiz, Google Switzerland GmbH, Digital Switzerland.
Hu L., Bentler P. M. (1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
Kane G.C., Palmer D., Phillips A.N., Kiron D., Buckley N. (2015) Strategy, not technology, drives digital transformation. MIT Sloan Management Review, July 14. https://sloanreview.mit.edu/projects/strategy-drives-digital-transformation/, accessed 18.03.2026.
Kargas A., Gialeris E., Komisopoulos F., Lymperiou A., Salmon I. (2023) Digital maturity and digital transformation strategy among greek small and medium enterprises. Administrative Sciences, 13(11), 236. https://doi.org/10.3390/admsci13110236
Kline R.B. (2023) Principles and practice of structural equation modeling, New York: Guilford Publications.
Ko Y.-C., Zigan K., Liu Y.-L. (2021) Carbon capture and storage in South Africa: A technological innovation system with a political economy focus. Technological Forecasting and Social Change, 166 120633. https://doi.org/10.1016/j.techfore.2021.120633
Nunnally J.C., Bernstein I.H. (1994) Psychometric Theory (3rd ed.), New York: McGraw-Hill.
Ochoa-Urrego R.-L., Peña-Reyes J.-I. (2021) Digital maturity models: A systematic literature review. In: Digitalization. Management for Professionals (eds. D.R.A. Schallmo, J. Tidd, Cham: Springer, pp. 71–85. https://doi.org/10.1007/978-3-030-69380-0_5
Panda S. (2025) Effects of information technology and knowledge management capabilities on organizational innovation: the mediating role of organizational agility. VINE Journal of Information and Knowledge Management Systems, 55(6), 1527–1552. https://doi.org/10.1108/VJIKMS-11-2023-0306
Pereira R., Serrano J. (2020) A review of methods used on it maturity mod-els development: A systematic literature review and a critical analysis, Journal of Information Technology, 35(2), 161–178. https://doi.org/10.1177/0268396219886874
Pöppelbuß J., Röglinger M. (2011) What makes a useful maturity model? A framework of general design principles for maturity models and its demonstration in business process management. Paper presented at the 19th European Conference on Information Systems, ECIS 2011, Helsinki, Finland, June 9-11, 2011.
Ramírez N., Ordóñez L. (2019) Modelo de madurez para la transformación digital (5th ed.), Bogotá: Ministerio de Tecnologías de Información y Comunicaciones.
Sándor Á., Gubán Á. (2022) A multi-dimensional model to the digital maturity life-cycle for SMEs. International Journal of Information Systems and Project Management, 10(3), 100303. https://doi.org/10.12821/ijispm100303
Sacoto-Cabrera E.J., Perez-Torres A. (2023) Digital transformation: A review of enabling technologies, maturity models, and open research issues. Paper presented at the 2023 IEEE Seventh Ecuador Technical Chapters Meeting (ECTM) 10-13 October 2023, Ambato, Ecuador. https://doi.org/10.1109/ETCM58927.2023.10308970
Schallmo D., Williams C.A. (2018) Digital Transformation Now, Cham: Springer.
Schermelleh-Engel K., Moosbrugger H., Müller H. (1988) Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research, 8(2), 23–74.
Schumacher A., Erol S., Sihn W. (2016) A maturity model for assessing industry 4.0 readiness and maturity of manufacturing enterprises. Procedia CIRP, 52, 161–166. https://doi.org/10.1016/j.procir.2016.07.040
Sihler F., Pietzschmann L., Straub R., Tichy M., Diera A., Dahou A. (2024) On the anatomy of real-world r-code for static analysis. Paper presented at the 21st International Conference on Mining Software Repositories, Lisbon, Portugal, April 15-16, 2024.
Sunil M., Ramasubbu N., Sambamurthy V. (2011) How Information Management Capability Influences Firm Performance. MIS Quarterly, 35(1), 237–256. https://doi.org/10.2307/23043496
Teichert R. (2019) Digital transformation maturity: A systematic review of literature. Acta Universitatis Agriculturae et Silviculturae Mendelianae Brunensis, 67(6), 1673–1687. https://doi.org/10.11118/actaun201967061673
Tucker L., Lewis C. (1973) A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10. https://doi.org/10.1007/BF02291170
Vial G. (2019) Understanding digital transformation: A review and a research agenda. Journal of Strategic Information Systems, 28(2), 118–144. https://doi.org/10.1016/j.jsis.2019.01.003
Viloria-Núñez C., Vázquez F.J., Fernández-Márquez C.M. (2022) A review of the digital transformation maturity models for smes in search of a self-assessment. Paper presented at the 2022 IEEE ANDESCON Conference, 16-19 November 2022, Barranquilla, Colombia. https://doi.org/10.1109/ANDESCON56260.2022.9989889
Westerman G., Bonnet D., McAfee A. (2011) Digital transformation: A roadmap for billion-dollar organizations, Cambridge, MA: MIT Center for Digital Business and Capgemini Consulting.
Worthington R.L., Whittaker T.A. (2006) Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806–838. https://doi.org/10.1177/0011000006288127
This work is licensed under a Creative Commons Attribution 4.0 International License.