At some point during this process, some concepts came to me that I had never considered before. I had always been a staunch member of the camp of developers who felt deep down that procedures such as Configuration Management were a complete waste of time at best and an active detriment to progress at worst. That is because, in my experience, they had been a complete waste of time or an active detriment to progress. I am sure you have seen the same thing.
A project, typically because of contractual obligations, makes the formal declaration that they are going to follow some methodology that is supposed to make everything sweetness and light. They have various names (in order of “all the rage”): TQM, CMMI, ISO 9000, Six Sigma, ITIL and various others. Each of these “Management Methodologies” suffered from the same malady that the “Social Sciences” suffer from. They all try to explain everything, or in the case of management methodologies, they all tried to fix everything.
Of course the developers just wanted to code something, but with these processes, coding came way at the end of the line. In the meantime, an eternity was spent gathering requirements, creating goofy little charts with stick figures doing odd things, then sitting around in meetings talking about “actors”, “stories”, and various other touchy-feely things instead of cranking out the stupid code.
This lasted for a while, but slowly developers revolted. As a direct result of these over-burdensome methodologies, developers began rethinking things. Somehow they managed to get the idea that management should just leave them alone and let them write something, then let the customer tell them what sucked about it, then go back and fix those things, write some more things, and just keep doing this until they had an awesome software product. The main hints that it was developers, not managers that came up with this process is that meetings were discouraged, working with your buddy was required, and working flexible schedules and ignoring deadlines was mandated. If you weren’t going to hit a release deadline, so what? Just take that piece of code out and release what you have. They will hate it anyway and it will be back for rework in a few days, so why worry?
The name given to this developer’s erotic dream was “Agile”. Who could hate something that was agile? It sure sounded better in a sales pitch. Consider the difference:
“Our developers are all Six Sigma and CMMI certified”
vs.
“Our developers are all Agile certified”
If I were a clueless customer, my money would be on Agile every time.
-
Agile Development Training.
The problem with ALL of these methodologies, Agile included, was that they never really addressed the core problem. I am not surprised by this because I never even realized what the core problem was until after I had already solved it. The other problem with them is that they were so complex, so time consuming, and so utterly head-banging dull that they were soon forgotten once development kicked off. Then towards the end of a project, managers would realize that there was no documentation to prove that they had followed said methodology (because they hadn’t), so they would pull a bunch of unlucky interns or contractors in and have them start creating history in the form of back-dated documents. Trust me, it is very common.
As I continued to work on the required Configuration Management Plan, some patterns started to fall into place. I realized that the reason projects go so bogged down and suddenly crumbled was often because they just got too complex to manage in the traditional ways of software development. At first, everything seems good. There is a set of requirements (we are assuming they are pretty good) and a team or two of developers. There is no source code. Everything that the project can and will be lies in the future, and that future is exciting. Coding starts. The first milestones are met with ease and the first release it sent to testers.
As testers find bugs (and they will find bugs), bug tickets are opened and worked by developers. Now there are two streams of work. Developers are still working on new code, but they are also working on fixing bugs in the existing code. Things get a little more complicated as new work is managed along side old work. If a project is using source control, they will likely branch the code at each release to test. Bugs must be fixed in the branch that they were discovered in, and then those changes must be propagated to the new branch(es) as well. A few more cycles of this process and things are really getting weird. Testers are not sure what environment they should be testing against. Most projects just resort to testing all code in the most current environment, but that environment slowly gets polluted and starts to smell bad. Phantom bugs are discovered that cannot be reproduced in the development environment.
Meanwhile, documentation efforts are ongoing. When a bug is discovered and the design must be changed to correct it, the existing design documents become incorrect. When missed requirements are realized, the designs again change and the design documents become even more incorrect.
Databases soon start to suffer the same problems as other components as their table structures, procedures, views, indexes and constraints are changed to keep up with bug fixes and new requirements. Some tables fall to disuse and some stored procedures become obsolete. Procedures suddenly expect parameters that the application code is not providing and data being returned to the application is not in the proper format or of the proper data type, resulting in more bugs.
What is happening here is that the project is getting complicated. There are many streams of work effort at various stages of completion, there are multiple environments that must support the changing code, and it becomes difficult if not impossible to manage it all.
The worst part of this often repeated scenario is that it usually gets worse the longer a project goes on. While budgets are being overrun, delivery dates are being missed, and developers are growing tired of sprinting, the complexity just keeps growing.
Complexity is not pretty.
The reason is simple to state, but difficult to correct:
Complexity – Order = Chaos
At the stage that has been described, most projects I have seen are, whether they admit it or not, in total chaos. There is no logical way to predict what a change will affect, there is no way to estimate how long anything will take, and there is no way to be sure it is going to work in a production environment. This is the point where many a project manager will, despite what he has learned in management classes, just throw more people at the problem. As we all know, this just adds to the complexity as new developers come up to speed on the existing codebase and environments. Since what little documentation is available is almost certainly outdated if not absolutely wrong, the over-worked developers must take time out of their work day to teach the new developers what they need to know.
Chaos – Direction = Apathy
Without any real hope of getting the project back to a known good state, many developers become disengaged and apathetic to the deadlines and mounting bug counts. Many start looking for a better project, sometimes from their work computers during business hours.
Apathy – Solutions = Death
Before long developers who have been on the project from the beginning are leaving. They hate the project and everything that reminds them of it. The dreams they had for it are gone and all that is left is disgust at the stinking, bug-infested mess that it has become. As the experienced developers leave, they often poison the well, advising the newer developers to get out while they can. Project death is soon to follow.
I have been through this cycle more times than I care to count. I never could figure out when it started, but I could sense it like something creeping in the dark behind me. I knew when death was near, and, coward that I am, I would be one of the developers looking for a new project (although I never did it on the job!). All of those times of going through this, I never understood why it happened, but as I was working through the details of this Configuration Management Plan, I began to slowly see what had happened, and I realized that it happened much earlier in the project than I would have believed.
In fact, a project’s death begins before the first line of code is written.
