Evaluating AGILE adoption in software delivery organizations

Evaluating AGILE adoption in software delivery organizations


Software development methodologies play an important role in the organizations’ ability to respond to new stimuli originating from new requirements, features, vulnerabilities and bugs.  AGILE methodologies are widely used, based on the premise of fast reaction and minimal expenditure in responding to these stimuli.  Research in the field of AGILE is evaluated against (a) culture transformation (Gregory & Taylor, 2019), (b) projects, (c) frameworks (Dingsøyr et al., 2012; Diebold et al., 2018) or (d) specific IT functions. Table 1 demonstrates the extensive research undertaken in the evaluation of AGILE against specific IT functions. Software delivery organizations necessitate the use of the above functions in their overall software delivery success. 

DevelopmentBianchi et al., 2020
Perkusich et al., 2020
Zaitsev et al. 2020
ArchitectureBeecham et al., 2021
Project managementBass & Haxby, 2019
DevOpsWiedemann, 2018
Hemon et al., 2019
Galup et al., 2020
Hemon-Hildgen et al. 2020
Wiedemann et al., 2020
SecurityZaydi & Nassereddine, 2019
GovernanceVejseli et al., 2018
Amorim et al., 2020
Existing literature evaluating AGILE with various IT functions

IT functions in AGILE teams

The below list explains the role of each IT function within the software development organization.

  • Development (D) – The actual development teams.
  • IT Governance (G) – The overall IT function which ensures that all IT functions are aligned and co-ordinated toward the organization’s IT strategy.
  • Architecture (A) – The IT function responsible to leverage standardization, reduce complexity and maximizing efficiency and quality. The architecture function works closely with Product Owner, Project manager and development leads.
  • Operations (O) – The IT function responsible to facilitate, enable, deliver and maintain software artefacts, produced by development teams.  In today’s environments this is normally referred to as DEVOPS. Other IT functions can be included in this category including security (DEVSECOPS) and DATAOPS.

The IT Governance, Architecture and Operations functions are supporting agents to the development teams. Yet they are essential towards successful software delivery cycles. Based on the above, a logical model is presented in Figure 1 involving external (green) and internal (blue) agents affecting the success of agile software delivery. These agents are subject to unexpected and frequent changes (Dooley, 1997).

Figure 1 – Logical model of various IT functions towards software delivery success

AGILE teams as Complex Adaptive Systems

The theoretical foundation enables the researcher to investigate the research problem.  The Complex Adaptive Systems theory is suitable for such investigation since it evaluates the AGILE teams within the broader ecosystem. Agile teams are complex due to their characteristics, whereby every person has relationships and interaction and behavior is unpredictable (Alaa & Fitzgerald, 2013).  Utilizing Ashby’s law of requisite variety, AGILE teams require internal capabilities to meet external requirements (Nerur & Balijepally, 2007). The study conducted by Jain and Meso (2004) demonstrate the lack of documentation produced by AGILE methodologies which can potentially compromise operations through refactoring, maintenance and support.   Another issue related to AGILE methodologies is related to fast resolution of issues (software fixing; due to velocity-constraints the teams do not engage in adequate problem solving engagement (Nerur & Balijepally, 2007) thereby falling apart from architecture and governance paradigms. Figure 2 depicts AGILE-driven IT functions working in harmony within a complex system.

Figure 2 – AGILE teams as part of a Complex Adaptive System within a software delivery organization

Validated instruments to measure the success of AGILE transformation

The instrumentation methods proposed in table 2 may be utilized to collect the necessary information from the appropriate IT functions. Such data allows researchers to evaluate the adoption and effectiveness of AGILE teams within the ecosystem.

Strategic alignment maturity assessmentMeasure adoption of governance, architecture, operations within the organizationLuftman, 2003
DEVOPS assessmentMeasure DEVOPS maturity within the organizationGupta et al., 2017
AGILE maturity modelMeasure AGILE maturity within the organizationGren et al., 2015
AGILE transformationMeasure AGILE impact on software organizationOlszewska et al., 2016


The above research allows organizations to evaluate the agile adoption amongst all the IT functions, whereby the entire ecosystems is assessed and tuned accordingly. Such reality checks allow organizations to continuously improve their process and workforce towards improved software delivery cycles.


Alaa, G., & Fitzgerald, G. (2013). Re-Conceptualizing AGILE information systems development using complex adaptive systems. Emergence: Complexity & Organization, 15(3), 1-23. Retrieved January 9, 2022, from

Amorim, A. C., Mira da Silva, M., Pereira, R., & Gonçalves, M. (2020). Using AGILE methodologies for adopting COBIT. Information Systems, 101496. 

Bass, J. M., & Haxby, A. (2019). Tailoring product ownership in large-scale Agile projects: Managing scale, distance, and governance. IEEE Software, 36(2), 58–63.

Beecham, S., Clear, T., Lal, R., & Noll, J. (2021). Do scaling agile frameworks address global software development risks? An empirical study. Journal of Systems and Software, 171, 110823.

Bianchi, M., Marzi, G., & Guerini, M. (2020). Agile, Stage-Gate and their combination: Exploring how they relate to performance in software development. Journal of Business Research, 110, 538–553. 

Buchmann, F., Nord, R. L., & Ozakaya, I. (2012). Architectural tactics to support rapid and agile stability. Carnegie-Mellon Univ Pittsburgh PA Software Engineering Inst.

Canat, M., Catala, N. P., Jourkovski, A., Petrov, S., Wellme, M., & Lagerstrom, R. (2018). Enterprise architecture and Agile development: Friends or foes? 2018 IEEE 22nd International Enterprise Distributed Object Computing Workshop (EDOCW).

Chow, T., & Cao, D. (2008). A survey study of critical success factors in agile software projects. Journal of Systems and Software, 81(6), 961-971.

Creswell, J. W., Creswell, J. D.  (2018). Research design: Qualitative, quantitative, and mixed methods (5th ed.). Thousand Oaks, CA: Sage.

Diebold, P., Schmitt, A., & Theobald, S. (2018). Scaling Agile – How to Select the Most Appropriate Framework . Proceedings of the 19th International Conference on Agile Software Development Companion – XP ’18.

Dingsøyr, T., Nerur, S., Balijepally, V., & Moe, N. B. (2012). A decade of agile methodologies: Towards explaining agile software development. Journal of Systems and Software, 85(6), 1213–1221.

Dooley, K. J. (1997). A complex adaptive systems model of organization change. Nonlinear Dynamics, Psychology, and Life Sciences, 1(1), 69–97.

Erder, M., & Pureur, P. (2016). What’s the architect’s role in an agile, cloud-centric world? IEEE Software, 33(5), 30-33.

Gregory, P., & Taylor, K. (2019). Defining Agile culture: A collaborative and practitioner-led approach. 2019 IEEE/ACM 12th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE).

Galup, S., Dattero, R., & Quan, J. (2020). What do agile, lean, and ITIL mean to DevOps? Communications of the ACM, 63(10), 48–53.

Gren, L., Torkar, R., & Feldt, R. (2015). The prospects of a quantitative measurement of agility: A validation study on an agile maturity model. Journal of Systems and Software, 107, 38-49.

Gupta, V., Kapur, P., & Kumar, D. (2017). Modeling and measuring attributes influencing DevOps implementation in an enterprise using structural equation modeling. Information and Software Technology, 92, 75-91.

Hemon, A., Lyonnet, B., Rowe, F., & Fitzgerald, B. (2019). From Agile to DevOps: Smart skills and collaborations. Information Systems Frontiers, 22(4), 927–945.

Hemon-Hildgen, A., Rowe, F., & Monnier-Senicourt, L. (2020). Orchestrating automation and sharing in DevOps teams: A revelatory case of job satisfaction factors, risk and work conditions. European Journal of Information Systems, 29(5), 474–499.

Jain, R., & Meso, P. (2004). Theory of complex adaptive systems and agile software development. AMCIS 2004 Proceedings, 197. RetrievedJanuary 9, 2022, from

Johnson, B., Holness, K., Porter, W., & Hernandez, A. (2018). Complex adaptive systems of systems: A grounded theory approach. Grounded Theory Review, 7(1). RetrievedJanuary 9, 2022, from

Luftman, J. (2003). Assessing IT: Business alignment. Information Systems Management, 20, 9-15.

Luna, A. J. H. de O., Kruchten, P., & Moura, H. P. de. (2015). The Conceptual Development Of The Agile Governance Theory. Information Systems and Technology Management 2, 177–201.

Morrell, J. (2005). Complex adaptive systems. In S. Mathison (Ed.), Encyclopedia of evaluation, 72-72. Thousand Oaks, CA: SAGE Publications, Inc.

Nerur, S., & Balijepally, V. (2007). Theoretical reflections on agile development methodologies. Communications of the ACM, 50(3), 79–83.

Olszewska, M., Heidenberg, J., Weijola, M., Mikkonen, K., & Porres, I. (2016). Quantitatively measuring a large-scale agile transformation. Journal of Systems and Software, 117, 258-273.

Onix, M. F. A., Fielt, E., & Gable, G. G. (2017). Complex adaptive systems theory in information systems research: A systematic literature review. In Proceedings of the 21st Pacific Asia Conference on Information Systems (PACIS 2017). RetrievedJanuary 9, 2022, from

Perkusich, M., Chaves e Silva, L., Costa, A., Ramos, F., Saraiva, R., Freire, A., … Perkusich, A. (2020). Intelligent software engineering in the context of agile software development: A systematic literature review. Information and Software Technology, 119, 106241.

Porter, C. (2017). An agile agenda wow CIOs can navigate the post-Agile era. RetrievedJanuary 9, 2022, from

SCRUM Alliance. (2017). State of SCRUM 2017-2018 – Scaling and agile transformation. RetrievedJanuary 9, 2022, from

Snyder, B., & Curtis, B. (2018). Using analytics to guide improvement during an Agile–DevOps transformation. IEEE Software, 35(1), 78–83.

Vejseli, S., Proba, D., Rossmann, A., & Jung, R. (2018). The agile strategies in IT Governance: Towards a framework of agile IT Governance in the banking industry. In Twenty-Sixth European Conference on Information Systems (ECIS 2018). 1-17. University of Portsmouth.

Virmani, M. (2015). Understanding DevOps & bridging the gap from continuous integration to continuous delivery. Fifth International Conference on the Innovative Computing Technology (INTECH 2015).

Vizard, M. (2020, May 29). Survey sees increased agile adoption during COVID-19. DevOps.Com. Retrieved January 9, 2022, from

Wiedemann, A. (2018). IT governance mechanisms for DevOps teams – How incumbent companies achieve competitive advantages. Proceedings of the 51st Hawaii International Conference on System Sciences.

Wiedemann, A., Wiesche, M., Gewald, H., & Krcmar, H. (2020). Understanding how DevOps aligns development and operations: A tripartite model of intra-IT alignment. European Journal of Information Systems, 29(5), 458–473.

Woods, E. (2015). Aligning architecture work with agile teams. IEEE Software, 32(5), 24-26.

Zaitsev, A., Gal, U., & Tan, B. (2020). Coordination artifacts in Agile software development. Information and Organization, 30(2), 100288.

Zaydi, M., & Nassereddine, B. (2019). DevSecOps practices for an agile and secure IT service management. Journal of Management Information and Decision Sciences, 22(4), 527-540.