Money to Burn? Invest in Functional Debt

But is there a flip side to this?  I think there is and the term I would use is Functional Debt.  This is tied firmly in the YAGNI camp and relates to functionality that is developed without a need (or worse still a test).  Applying too much design, or developing generic frameworks with no business reason to do so inevitably leads to a solution which is over-engineered.  Of course, over-engineering as a term has been around for a long time, but I prefer the term Functional Debt, because this ties it back to money in a similar way to Technical Debt.

Debt is a term that evokes emotion and is easy for people to identify with and it is this capacity of the term to clarify the issue with a certain practise.  Over-engineering as a term doesn't evoke the same response and certainly doesn't suggest a loss of money in the same way that Debt does.

There are of course direct, easily measurable costs involved in creating unused functionality and that is the development costs, however, there are many more subtle costs that are easy to overlook.  There is the missed opportunity costs associated with not doing the right thing.  There is the project overhead costs in maintaining code that is not used.  There is the project overhead costs in increased complexity and time for the standard day to day activities of testing and refactoring.  There is the increased maintenance costs since it is now harder to understand the code for support personnel...

One of the biggest causes for Functional Debt I have seen is a lack of customer (business) involvement or direction.  Left to their own devices, IS departments naturally build overly- complex solutions to simple problems.  Without a business value attached to a piece of functionality (actually to a problem that is solved by a piece of functionality) it is only too easy for the IS department to burn money like there's no tomorrow.

        

It is worth noting that innovation and invention are solutions and not problems. When you apply for a patent, you don't patent the problem, you patent the solution, why? If there are multiple solutions to any given problem then wouldn't we want to patent the problem?

I also find that quite often the solutions I am exploring are the result of a problem that I have assumed was well known and in actuality is not; it is in these moments that using the solution to derive the lowest common denominator of a problem stated as a user story can ensure that everyone is speaking the same language.

But given a solution, do we always need to know the problem? My pragmatism (and a few lean principles, i.e. waste) would state that if the solution brings value then run with it, if not ditch it. If on the other hand the solution seems too constrained or lacking in features/functionality, then why not exploit the solution as the basis for a vision? Indeed many visions are just that, solutions to problems, expressed in a language alien to most developers.

So given a vision expressed in an abstract form vs a vision that is pinned against an actual solution which is easier to understand and less likely to be misinterpreted?  I know which I'd prefer...

Do solutions have value, obviously; must we always mine problems, hmmm...

        

The following commandments would be my first stab at such a list which would apply to any person who is part of or interacts with an Agile team.

1. Thou shalt not negotiate quality
2. Thou shalt not prophesize without proof
3. Thou shalt not falsify the facts
4. Thou shalt not force thy wishes on others
5. Thou shalt not discourage those around you
6. Thou shalt not disparage the actions of others
7. Thou shalt not work without committment
8. Thou shalt not focus on more than a singular item
9. Thou shalt not demonstrate incomplete work
10. Thou shalt not act in isolation

Of course, this is just a bit of fun on my part, but I fail to see why there can't be a set of golden rules of Agile? This is also a very quick list, so who knows, I may revisit this at some time to refine it...

Also, please note, I'm not being remotely fuzzy about this...  I'm not saying

"I value not negotiating quality over allowing any old crap in the code..."

        

DNA

I've been researching material to support an article on my company blog entitled "Agile Dictators" and it left me thinking about how Agile started in the first place.

The more I reflect on this, the more I am left feeling that Agile is actually a mutation in software development and this is one of the major reasons why Agile is so difficult to master. I'm wondering whether agile would ever naturally evolve in a small team left to their own devices and I simply can't envisage it. Of course, this will now remain an academic hypothesis since Agile has now stamped it's influence indelibly on the DNA of software development.

Software development as a craft suffers from unnecessary complexity and I fear that Agile, which initially thrived in simplifying (or removing) this complexity, is now becoming itself encompassed in unnecessary complexity.  Despite agreeing with much of the sentiment of the software craftmanship manifesto I just can't bring myself to sign up to it yet.  I struggle to see the benefit of more fuzzy aspirational statements and would prefer to see a clarity of vision and roadmap to achieving it.

Fundamentally Agile is fantastic, but sadly the passionate discussions, raging debates and conflicting methodologies don't clarify anything. If Agile doesn't clearly define itself soon I fear yet another mutation may take centre stage and Agile will end up being just a blip (albeit a very significant blip, where we gained sight) in the evolution of software development.

        

I've been working on a few internal projects for my company of late and this has involved working in a distributed fashion (yes, pairing by phone), and during some of these sessions I can immediately envisage the benefits of using Wave.

Surely there is at least one complete, tested feature they can roll out to production?  My concern is there are still some fundamental challenges facing the team, and I would not be surprised to find it being concurrency issues, so as the potential end-user of this technology I'm going to make it clear, I don't particularly care about have multiple users editing the same document in real-time with transparent, asynchronous updating...  Let me edit, commit, refresh, etc as a simple starting point...  But whatever you do, let me do something before I find a simple alternative...

To find out more about wave, visit the wave site http://wave.google.com/, and especially watch the google I/O presentation...

        

TDD found the simple solution

Test Driven is a loaded term and means different things to different people.  I much prefer the term Test First which clearly states that the test comes before the implementation.  However, for me, the value is not necessarily in creating an executable test, but in the thought processes that Test First brings out.

One of my colleagues at Exoftware is constantly reminding me that when presented with a solution you need to ask yourself what is the problem. This is what lead me to question TDD and it's variants.  If you search the web for Test Driven Development, you'll uncover a wealth of information from many of the authors in my BlogRoll, as well as many variants on a theme. I think the wikipedia entry is a particularly good summary of Test Driven as it is currently understood by the community but for the real meat and bones you need to look at the articles from the thought leaders behind the practises.

However, when I look at this wealth of information, I'm now faced with the question...  OK, these are all solutions, but what is the fundamental problem they are solving?  Is it

• Code quality/design is poor
• Code is overly complex/difficult to test
• Unused functionality within the code
• Too many bugs

It is only when we understand the underlying problem fully that we can then evaluate the applicability/suitability of a particular approach.  Indeed, it is this lack of a clearly defined problem which makes it impossible to determine which approach is best, since we can't define any tests up front...  This is actually a little bit of a paradox, but highlights for me one of the most important points about TDD.  TDD is a solution to too many problems and in certain cases is not a very good solution.

Testing should be about proving functionality, but unfortunately we too often see TDD trying to address non testing issues like design and code quality.  Of course, code needs to be testable, but it is not the responsibility of the Testing to enforce this, it is the responsibility of the design, but this is (unfortunately IMHO) the missing piece in most methodologies...  Indeed, the approach of writing the "Simplest Possible Thing" to pass a test is possibly my biggest bug-bear with TDD.  This approach often means you can pass tests with no functionality other than hard coded return values, now that has to be the biggest process smell ever.

I'm going to stop using TDD, or Test Driven now and use instead the term I prefer which is Test First.  For me, this means that before I do anything, I will determine ahead of time how I would test it.  The immediate benefit (a.k.a value) I get from determining how to test something is I'm also starting to think about (dare I suggest design) things like apis/design/interaction/responsibilities.  This is not the same as BDUF (big design up front), instead it is just enough thought at just the right moment to (hopefully)  prevent a disaster.  I can then apply some cost/benefit analysis with a pinch of risk analysis to assign a value to actually writing all the tests that TDD would have forced you to write, or just a small percentage to cover the important or more complex functionality.

Of course, in many cases I will actually create executable tests for many of those uncovered during this thought exercise, but will I go through the whole Red/Green/Refactor cycle, possibly not.  For me the Red/Green/Refactor is like micro context switching.  I prefer a slightly longer period of focus and therefore I may write quite a few tests at the same time before applying that context switch to go into coding mode.  This of course is my personal preference and undoubtedly would be scorned upon by the dogmatic TDD zealots, but if this is what makes me more productive, and without a baseline problem statement I challenge them to prove that this is not the best way to do your testing.

Test First allows me to

• Understand the problem I'm trying to solve
• Think about how I will solve it (just enough design)
• Uncover any unknowns or risks hidden in the initial problem statement
• Produce high quality code which has just enough tests

Many of the Test Driven approaches do indeed address many of the initial problems stated earlier, but are they the only solution to these problems?  Indeed, are these problems actually just symptoms of even deeper problems?  If the problem is simply that your code is very tightly coupled and difficult to test then the best ROI will probably come not from TDD but from up-skilling your development team on fundamental design principles (unskilled developers was the real problem in the first place and TDD can't solve that).

Resources:

• Wikipedia entry for Test Driven Development:  http://en.wikipedia.org/wiki/Test-driven_development
• Introducing BDD by Dan North: http://dannorth.net/introducing-bdd (highly recommended reading)
• C2 wiki entry for Test Driven: http://c2.com/cgi/wiki?TestDriven

        

As a blogger,
I want to use Threely as my url shortener.
So that I get more letters on Twitter.

Pretty simple story, and also very simple to implement, but impossible to test?  I can certainly test the API to twitter, but given I use a wordpress plugin (developed by someone else) to do the auto-posting is it worth the effort for the addition of 6 lines of code?

For me, the pragmatic solution is to take the risk based approach.  I'm going live with Threely (http://3.ly) (again, fixed a small typo), but have an escape plan (extremely important when there are known risks) which is the ability to revert if anything untowards happens and revisit my approach.

So, here goes, pressing the big red button

        

The MMF wasn't quite complete.

I've seen plenty of fancy graphs demonstrating how agile provides faster ROI through early delivery and while this can certainly be true in some cases, it doesn't necessarily hold true for all cases.

Just because a feature is done, doesn't mean you could, or even should deliver it...  The Minimal Marketable Feature set (MMF) is very important for determining when to go live.  When you take the MMF into account, then agile doesn't deliver any value until the MMF is complete and this certainly won't be within the first iteration.

Of course, once the MMF is done, agile will win hands down delivering new functionality, enhancements and bug fixes (what bugs?)...

So what went wrong in the picture?  What was the MMF

1. Wheels/Chassis
2. Engine/Gearbox
3. Steering
4. A Seat

And guess what, they've ticked all the boxes, I'm so excited I'm going to run off now to my local dealership and  snap one up...

        

Some ppl complain agile only works w/ highly skilled developers. Never been clear 2 me that ANY dev. method works w/o highly skilled devs.

I think the subtle distinction is that agile REQUIRES skilled developers to be successful, whereas some of the "heavier" methodologies would be better with skilled developers but don't actually require them.  I also think that when we look at many of the agile successes, it would be interesting to determine whether there was any correlation between the level of skill of the developers compared to the level of skill on the non-successful and/or non-agile projects.

And taking this just a little bit further...  Would a small group of skilled developers be successful regardless of the methodology?  Truly skilled developers tend to be very pragmatic and will always find ways to simplify the complexity around them, so I'm sure that if you took 8 highly skilled, highly successful agile developers and stuck them on a waterfall project they would deliver a successful result, at least in terms of the customer...

What's also quite interesting about this dynamic is that once a developer "sees the light" and becomes "agile" they can't imagine going back to waterfall, despite the fact they can add immense value by being part of a waterfall project and improving the processes.  There is something very selfish about this which has not yet been picked up in the mainstream...  This is also possibly one of the reasons why agile suffers an identity crisis, often being regarded as a cult.

And no, I'm not advocating waterfall, I'm just wondering whether skilled developers have more impact on success compared to the methodology.

        

For me, slack is just another way to improve the transparency of an agile project and is also the ideal vehicle for introducing diversity, continuous learning and continuous improvement. Given slack, developers will naturally use this to either improve their own networks, improve their understanding on a topic they found difficult or explore new technologies. Any of these will ultimately add significant value to your project.