When I had my first training on agile delivery, I recall very well the agile coach discussing the Agile Manifesto (see https://agilemanifesto.org/ ) with the 4 principles and the 12 practices, we know by now all inside out. At this day more than 10 years ago, we had a long discussion regarding the 2nd principle “Working software over comprehensive documentation” – and whether agile delivery means no documentation or how to interpret this principle. It was a discussion that ended as many discussions do in the consulting space – with an answer “it depends”. It pointed out the ambiguity that is embedded in agile. And it changed the way how I today practice agile, especially in AI and Data Science project work.
Over time we’ve seen many agile projects, some of them more successful, some of them less. Some of them claimed to follow agile, some of them actually did agile delivery. What makes a difference? Where are the small nuances that make a project successful and what makes them fail? In this blog article, I would like to share my views on common mistakes and clean out some misbeliefs regarding agile and offer a simple formula to keep in mind when you setup and run your next agile project.
How to understand the Agile Manifesto
Back to the Agile Manifesto and it’s four principles:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
While the first part of the principles (before the word “over”) seem to make common sense – I was very much interested in the second parts. We had many discussions in the past on how to interpret that part of the Agile Manifesto. A bit provocative, one could state that agile projects have no processes, tools, documentation, negotiations or plans at all.
Now, after many years of experience in agile delivery, I have seen many situations which all demonstrate, that such a ruthless interpretation is far from what the principles want to say. The Agile Manifesto wants to get the priorities right – but the principles are by no means exclusive nor radical statements.
On the other hand, a very naïve view would be to assume that the agile and its manifesto is simple and straight forward. What initially seems simple and straight (at least looking at the principles) is after a closer look far away from being that. In my experience, I have hardly seen any straightforward and easy agile project – every project has its complexity, its specialty, its character. And so does the Agile Manifesto – hence, the answer on how to interpret the manifesto truly depends on the actual situation a project is in and the ambition, what the project wants to achieve.
Let’s have a look at some examples around the four principles.
Agile AI and processes
First, while it’s important to be human centered, agile is far more than anything else, structured and radically steered through a strong governance process. That seems counter-intuitive – but taking a closer look, it makes a lot of sense. This gets clearer if you look at it from various angles. E.g. agile has a single dedicated role to facilitate the method, the process and to remove impediments – the scrum master. Where have you seen this in other delivery methods? It seems that agile is far more structured that initially it seems to be. Also have a thought about the ceremonies such as the daily stand-ups, the scrum of scrums or also the retrospectives or playbacks – the iteration planning in general. Agile intentionally sets a frame, a structure around the delivery.
Agile AI and plans
Second, agile delivery follows an incremental plan, that allows for change – but it is far away from operating without a plan. Here again, agile puts structure as an answer to keep the flexibility for change – agile’s answer to planning are the sprint & increment grid in addition to the product owner, who sets right priorities. Have you ever been in an agile project where the product owner was not able to make decisions on the backlog priorities? Quite quick the project turns from a green status to dark red as the team needs continuous validation and discussion with the best person to set the priorities. Consequently, the burn down chart or velocity is hit and you’re getting behind the plan. Rethinking KPIs like these ones mentioned above, in fact try to give a visibility on how much ahead or behind an expectation (aka a plan) a team / project is. The structures support the measurement of agile projects.
Agile AI and documentation
Third, I recall intense discussion regarding documentation requirements in agile projects. The client believed this is not required at all and hence, we should increase our velocity expectations for the initial iteration of a project. I had to object to this view. To me there is one thing very clear – there is no agile project, especially not in the AI and Data Science space, that will be delivered and accepted without documentation. Traceability, transparency, and explanations are key to foster trust and proper interpretation of the results such projects give. Without proper documentation, service receivers tend to criticize, question and challenge results. Only if trusted teams have a chance to review the code base and documentation, such customers will adopt the change and follow recommendations. Additionally, teams work in iterations and to push boundaries and support the cross-functionality in such scrum teams, it is key to have job rotations. To support such motivating and experience building activities, documentation will be key.
Agile AI and contracting
Fourth and finally, agile sets with the structure it has the right guidelines to steer projects towards success – this is done through strong integration of the end customers (often the business) and additional stakeholders from the customer’s organization such as IT, Change & Communication, and senior leadership executives. A good contract hence agrees the processes, the delivery models, the escalation paths, the KPIs and the way the teams measure themselves. Most important in my view is the definition of the client owned roles and dedication to the project, i.e., the product owner, the product management, the business owners and the epic owners, the shares services teams and the system teams. There is a lot of collaboration, discussion and alignment required in agile delivery to get things right. These counterparts must be available. E.g., one of the most complex points in AI and Data Science projects is to get access to the data. This is a task which normally must be supported and led by the internal IT department. Good contracts outline such responsibilities and to some extent also Service / Support Level Agreements (SLAs) with such stakeholder groups. Good contracts give the right framework to deliver in an efficient and targeted way. Hence, it’s important to agree on the ways of working. Based on my learning the most crucial part in contract negotiations is to convince and motivate the procurement department, which naturally prefers to have fix price, fix outcome projects (best to control), to support an agile delivery contract. Discussions like clearly defined deliverables and implemented functionalities are harder to embed into agile contracts. All stakeholders must deal with ambiguity which normally requires a good understanding on how agile delivery works, a strong relationship based on trust (that has normally to be earned) and support from the business to convince procurement for the agile contract form.
By now, we have seen that the Agile Manifesto is more a framework of principles that allows for flexibility & judgment, and it’s not rule based nor black and white by any matter. Agile is built on communication, exchange, and alignment. To make this efficient, the schedule and the roles are very clearly defined upfront – that reduces at least on the governance side the ambiguity – it is clear who is responsible for what.
Maximizing the delivered value with agile – a heuristic formula
Now taking a step back – agile tries to maximize the delivered value by setting the focus on the right things while allowing for quick change adoption in an environment that supports creative working. It is based on motivated & self-directed teams that “buy-in” on the sprint backlog. This is supported & guided by a structural framework that defines the roles, responsibilities, escalations, and ways of working (processes). While this is all good, there needs to be strong discipline and experience from each stakeholder in an agile project. The experience to deliver in a disciplined and structured way is one of the key success factors and demands a mature organization with experienced scrum masters (at least) to be successful. Hence, I came up with the following heuristic & illustrative formula which positions discipline & structure as important factors for the maximization of delivered value in agile projects.
Delivered\ Value\ =\ g( Capability,Flexibility,Creativity) \ \cdot \ f( Discipline,Structure) \ ^{\frac{1}{f( Discipline,Stucture)}}This formula, of course indicative and illustrative, is based on the logic of two concepts, i.e. f(x) and g(y) which have a strong relationship to each other. This follows the mathematical formula outlined hereunder.
Delivered\ Value=g( y) \cdot f( x)^{\frac{1}{f( x)}}What can we learn from the formula?
What does it mean? As outlined on the graphs below, the combination of capability, flexibility and creativity will increase the value of agile delivery in a linear way. This means – very simplified – the more the better. It is more complex for the combination of discipline and structure. Here it is important to find a good balance. If e.g. you have not enough discipline and structure, you will miss the opportunity to maximize the delivered value. On the other hand, if you have too much of it, you will again destroy value. There needs to be a good level of discipline and structure to unlock the value of the available capability, flexibility, and creativity.
Fun fact – the maximum on these graphs is always a multiple of e, Euler’s number. Maybe you could say that Euler already figured out back in 1731 the golden formula for value in agile delivery. This has yet to be proven, let’s see.
Key take-aways and how to start right
Consequently, we strongly suggest making sure your agile squad has a foundational discussion early in the project (best before its start) regarding how to best implement the Agile Manifesto and its principles. Teams must find the optimal balance of discipline and structure to maximize the value of the delivery.
Net-net – the Agile Manifest intends to setup good practices – and practices must not be mistaken with black-and-white rules. It’s not binary, it depends on the situation. And it also requires expertise, know-how, continuous re-validation and steering and above all, applied common sense. Hence, agile delivery is powerful in mature and well-trained organizations and can have many pitfalls for projects that only claim to follow “agile delivery” without proper setup. We generally suggest starting with smaller & simpler projects to create an environment to learn and grow. Making sure that the team gets the right support from expert scrum masters and also have access to training. Agile must be lived – hence, people delivering following this method, must truly buy-in. Agile eventually is also change of an enterprise’s culture – which normally requires time and continuous efforts and role modeling from the very top.
I would be very keen to get your practical expertise and insights in agile and finding the right balance of structure & discipline. I would be very happy to discuss and engage with you over any channel such as LinkedIn, Twitter or email.
Michi Wegmüller
Co-Founder – Empowering Agile Analytics at Scale
Michi Wegmüller is co-founder of Artifact SA and responsible of Artifact’s Analytics Garage offering. He has more than 15 years of experience in Data and Analytics consulting and has supported a diverse set of Swiss and international clients across industries. He has helped to realize analytics initiatives that are sustainably growing and continuously delivering value to the business and functional units. He is passioned about agile analytics at scale.
Artifact SA
Accelerating Impact with AI & Data Science
Spearheading in AI & Data Science to accelerate impact for your business in Switzerland. Pragmatic analytics services leader for consulting & implementation.
0 Comments