This post is a work-in-progress.
If you were asked what picture you associate with the term ‘project management’, you’ll probably imagine a shallow set of steps heading downwards and inexorably towards a successful project completion milestone. This is the ubiquitous Gantt chart and is as closely associated with project managers as a wrench is to a plumber. You’d be hard-pressed to find a tech manager who has not drawn many a Gantt chart, that staircase down which teams will march towards project delivery. Or will they?
The constraints of traditional project management
Project management evolved from operational research and systems engineering and is based on a reductionist philosophy. In other words, a complex problem can be progressively broken down into smaller and smaller constituent parts, with clear definitions, hard boundaries and clear relationships. The underlying assumption is that uncertainty within a project can be eliminated by breaking it down into parts that are small enough to be modelled with confidence. Based on this, a linear plan can be put together, including a prediction of how long the project will last and the time it will take to deliver.
Despite all the best efforts of the project management profession over decades to instil predictability in project delivery, few people who have worked in software or IT will not have experienced projects that have significantly overrun in cost or time. A study by the University of Oxford and McKinsey of around 5,400 projects, found that cumulatively, these had a cost overrun of $66 billion. For projects with a budget in excess of $15m, software projects over-ran their budgets by 66% and their schedule by 33%. 17% of these projects went so bad that they threatened the very existence of the company.
Software development is intrinsically unpredictable
So why is keeping software projects on track so difficult? The 19th century Chief of Staff of the Prussian army, General von Moltke the elder stated that ‘no plan of operations extends with any certainty beyond the first contact with the main hostile force.’ General, and later US President, Dwight Eisenhower built on this by saying “plans are of little importance, but planning is everything… The very definition of ’emergency is that it is unexpected, therefore it is not going to happen the way you are planning.’
These military leaders knew that any strategy or plans that failed to accommodate or to adapt to the unknown and the unpredictable is doomed to failure. Whilst hopefully most software projects are not faced with an adversary aiming to disrupt and destroy their progress, much like military operations, they operate in environments of high unpredictability. Most complex software projects do not follow a fixed template, which makes the estimation of effort in software pretty challenging.
In the first instance, when forecasting the trajectory a project may take, any estimate depends upon the intrinsic complexity of the required solution – i.e. how hard the problem is to solve. Often you are dependent on other systems and software, of which you will only have an imperfect understanding at the outset, which may well turn out to be wrong. Even if the project manager’s understanding is perfect, the project forecast also relies on the ability of the teams building the solution and any dependent components to implement it correctly. This may well include mastering new software tools, languages or environments. Even if all is well-known and understood, external factors often come into play – such as the reliability, quality and timeliness of dependent teams or services.
Even if all the above factors are well-controlled, you are then faced with what is often considered the number one enemy of any developer – changing requirements. The ephemeral nature of software products means that it is often difficult to predict with certainty the impact a software product has on a business’ output. You may deliver a piece of software on time and to budget, but your customers may ignore it, or even have a strong negative reaction to your carefully-crafted creation. Perhaps the market moves on, or a new feature from a competitor has shifted your customers’ expectations. Being on time and on budget does not on its own make a successful project. Your customers will be the arbiter of that.
So what to do about it?
Luckily there are a number of techniques and tools at the modern project manager’s disposal for dealing with this uncertainty. Most of these form part of the Systems Thinking toolkit, a somewhat amorphous collection of techniques, all of which are grounded in understanding a problem as a whole, rather than breaking it down into its constituent parts.
A previous post explored the characteristics of complex systems, and how systems thinking could aid in dealing with them. The Cyefin framework was used to classify problems into chaotic, complex, complicated and simple. Any software project that involves a number (say more than ten) inter-dependent teams may be considered to be complex. In other words, there are unknown unknowns. In addition to all the sources of unpredictability described above, large scale projects are often highly interdependent. Generally, dependencies are a good thing in software, as they allow teams to stand on the shoulders of giants (or at least of colleagues, partners or suppliers) and use software that is already in existence or that someone else has decided to build.
Probabilistic Modelling
One approach that can bring rigour to project management forecasting is probabilistic forecasting. As the name suggests, this uses statistical techniques to forecast the range of likely project outcomes, using the same principles of weather forecasting. So if a weather app forecasts that tomorrow there is a 64% probability of rain, probabilistic techniques can also give a similar indication of when a project or part of a project is likely to complete.
Consider its use in projects being carried out by a number of scrum teams of software engineers. Each scrum team will have a track record of how long it takes to implement work items, be they new features or bug fixes. These can be plotted as a scatter diagram of cycle times (time to implement). Given enough samples of work items, these will eventually begin to form a distribution of a discernable shape when plotted as a histogram, often in the form of bell shape. This track record can then be fed into a Monte Carlo simulation. Named after the casinos of the eponymous principality, a Monte Carlo simulation seeks to predict the range of likely outcomes by carrying out a large number of individual simulations according to the known characteristics of a model. This is akin to throwing a dice a thousand times and counting the number of times it lands on each of its sides. When applied to project management, the past behaviour of a team can be used to create a mathematical model, and then this model is simulated thousands of times to understand what are the likely outcomes.
Causal Loop Modelling
While statistical modelling can provide a certain degree of mathematical rigour to project forecasting, its main weakness is that it is fundamentally a static model. It does not take into consideration the changes in the human and environmental factors that influence how fast work can get done. It predicts the future based on past performance, assuming nothing changes. However, very few organisation is static, and most projects will experience all or some of the following: team members joining, leaving, organisational stress levels rising as key deadlines approach, money being thrown at problems in order to accelerate delivery dates, budgets being cut or diverted, experienced team members spending time to train and mentor new joiners and so on. Off course, much of this can be planned for, but the crux is that there are interactions that impact the overall project velocity in ways that are difficult to predict.
One way of visualising and understanding these factors is the use of system dynamics – drawing out the feedback loops at play within an organisation and the assumptions being made by the organisation’s management. These loops, known as causal loops, aim to make visible the positive and negative relationships between factors (or variables) that can influence the success of a project. For example, a fairly safe hypothesis is that a higher feature velocity can result in a larger number of defects, simply because more code is being created. However, what is the impact of increasing the number of developers on feature velocity? The answer is, “it depends.” It depends upon the related experience and skills of the new developers and the ease within which they can be integrated into the project. The LeSS (Large Scale Scrum) framework website shows a great example causal model of the factors at play in a typical software organisation. The aim of this approach is to dive deep and understand what are the fundamental dynamics that are constraining or driving development velocity. Crucially, it seeks to get to the root cause behaviour of your project and challenge assumptions, which may be deeply ingrained.
Systems Optimisation
In my opinion, the most fundamental task of a leader of a complex project is to keep a holistic (i.e. a systems-wide) view of the entire project – understanding how and when value will be delivered to customers. This means keeping their eyes focused on doing what is necessary to deliver this goal. A corollary of this is to ensure that individual contributing teams do not optimise their activities in a way that makes sense from their local point of view, but which may be detrimental to the overall success of the project. This is the over-optimisation trap.
Consider the example where a development team is creating the mobile app that is part of a wider project creating that is building a new online food delivery service. This team may focus its efforts on automating the testing of its user testing. This is a technically sound endeavour, but if the project is behind schedule due to constraints in the development of its back-end services, then this local optimisation is not only irrelevant but is also setting back the overall project. All project activities have an opportunity cost associated with them. By choosing to invest in one part of a project, you are by definition, redirecting resources from elsewhere in the project. Successful project leaders need to ensure that investment is directed to where it makes a difference and that organisational silos don’t get in the way of delivering what really matters.
Learn to Love your Dependencies
Ask anyone managing a software project what their biggest challenge is, and after the inevitable mention of resources, chances are that they’d mention dependencies – in other words, any item of work where their deliveries or success depends on others, be they other internal teams, external partners or suppliers. Dependencies are typically considered as problems to be mitigated – indeed the 2020 Scrum Guide makes no mention on how to work with other teams, while a previous version said that “cross-functional teams have all competencies needed to accomplish the work without depending on others not part of the team.” Not only is this nonsense, as anyone who has any experience of managing software projects of more than a modicum of complexity will recognise, but it treats dependencies as inherently bad.
It is certainly true that you should create as self-sufficient teams as possible, and iterate and self-inspect to achieve independence. Nevertheless, don’t assume that all dependencies are a bad thing. If you are dependent on other existing systems or services, be they external libraries, APIs, or any other software component, then that is fantastic news, you don’t have to build it yourself! Even if such services are paid-for, either as a one-off or licensing fee, there are many reasons why you may still be better off by using these services. If these services are built by experts in that field, then you are likely to be better off using these services.
More challenging is the case when you are dependent on other teams to do work before you can deliver your output. This could be either because you lack the resources, time or expertise to build the capability yourself. The first task is to see the extent to which these can be reasonably reduced. E.g. through architectural design practices, or by grouping as much of the expertise required to build your software in one team. Once this is done, you are left with the task of tracking and managing throughout the project. So critical is dependency management to the success of multi-team agile projects, that most agile scaling frameworks focus their coordinating efforts on managing dependencies. Scrum@Scale assigns the task of inspecting and managing cross-team dependencies to the Scrum of Scrum Master. The Scaled Agile Framework (SAFe) makes use of a program board to track dependencies, both during planning phases as well as during individual sprints. In fact, one criticism of SAFe is that it is more focused on keeping track of dependencies than on encouraging teams to be as self-contained feature teams. Whatever techniques you use, stop wishing you didn’t have dependencies, and instead embrace them. You, and your projects, will be happier for it.
References
- Association of Project Management – Systems Thinking – How it is used in Project Management
- Systems Thinking – the Theory behind Agile
- The Flow System: Leadership for Solving Complex Problems
- Actionable Agile Jira Plugin
- Create Faster and More Accurate Forecasts using Probabilities
- Factors Affecting On-Time Delivery in Large-Scale Agile Software Development
- Delivering Large-Scale IT projects on Time, On Budget and On Value
- Choosing a Development Model: Predictability vs. Velocity
- Exploring software development at the very large-scale: a revelatory case study and research agenda for agile method adaptation
- Large Scale Scrum – Systems Thinking
- Dependency Management – the Good, the Bad, the Ugly