To Document or not to Document: Stop fighting documentation by integrating it.

In an earlier post I talked about the great agile/waterfall documentation debate.  One method, waterfall, extensively documents everything while agile, another method, consciously cuts down documentation to the minimum amount that is viable.  While both have good points, an ideal system would incorporate documentation naturally into the process, where the team does not even realize that they are documenting as they go.  A classic example of this is the automated test.  By looking at the test and its results, you should be able to gather most the necessary information about the test.  In this article, I will demonstrate some other examples of how to do this in other parts of a software development life cycle.  This will of course not be a comprehensive list.  Rather, this is a place to start discussion and ideas of how to integrate documentation naturally.

One very simple method to create a natural trail is to use meetings and agendas.  Meetings are a place where requirements, changes, baselines, and other major decisions can be decided; certainly prime material to document.  Sometimes these are not captured very well though.  Have you ever walked out of a meeting feeling great, then arrive at your desk and realize 'Now what am I supposed to do'?  Next time you have a meeting:

•  assign someone to be the 'timekeeper', 
• have him create and/or distribute an agenda
•  give him the responsibility of capturing meeting outcomes

If the timekeeper keeps an agenda and writes out important outcomes and decisions, a trail is naturally created  that captures documentation information.  It will also help you avoid the 'now what?' issue.  There are some barriers to this though.  To encourage free conversation, guidelines will need to be created to define idea sharing and brainstorming from final decisions to document.  Otherwise idea generation will be impaired and non-binding decisions and ideas will be captured in the formal documentation.  It may be difficult as well to naturally add certain assumptions and knowledge that everyone already knows before the meeting, and therefore won't be discussed and captured during discussion.

In our wonderful high technology world, there are some very great software tools that can be used to document a whole variety of project elements.  While there are dozens of choices that range from free, localized editions to giant enterprise SaaS implementations, I'm only going to mention AtTask and Jira here.  In AtTask you automatically capture update conversations between collaborators, can attach documents, pictures and other items, and know at any time the status and team members of a given task or project, along with a host of other options.  Jira actually started out as mainly bug-tracking software but is a very flexible platform with tools for updating, sharing, and discussing project elements.  There are many great software tools that automatically create a great trail but it is important to remember that they are only tools.  They won't capture face to face conversations, for example, and if you want to capture specific information, you will need to set those guidelines down in your organization.  For instance, QA may require that issues come with a 'steps to reproduce' list and maybe screenshots or system information.

I would also like to mention lessons learned as part of integrating documentation.  Lessons learned is one of those things that gets forgotten a lot at the end of a project.  This is partially due to the prevalent yet false idea that lessons learned should be administered only at the end, and not integrated into, a project or phase.  Agile surprisingly, has made some gains in integration by using daily standup reporting and sprint retrospective, essentially daily, and sprint lessons learned.  Whatever the method used, integration of lessons learned into the natural process of carrying out the project is likely the most helpful improvement of all of these. 

These are some great practices that will help your organization have a smoother process for documentation.  This should also foster ideas and discussion on tools and techniques.  One thing that we have not addressed yet is best practices and rules, which should certainly be developed at some time.  Please feel free to share your ideas and stay tuned.

To document or not to document; that is the question

Whether 'tis nobler in the company to suffer 

The numbers and writing of outrageously large reports

Or to take arms against the sea of papers

And by opposing, go Agile?

    I'm sure most people have been confronted with the problem of lengthy documentation that no one reads versus cutting out documentation so that no one knows what's really going on.  To give my readers some background though, there is bit of a struggle on how to move on a project with the two forces (they sometimes oppose each other but they don't need to) being:

1.  Traditional Waterfall Methodology, which encompasses lengthy planning, execution, documentation, reporting, and structure.  Generally the tasks will look like a waterfall on the Gantt chart, hence the name.

2.  Agile methodology, captured in programs such as Extreme Programming, Scrum and Kanban.  This takes a much more adaptive approach and responds much quicker to change.  It is generally characterized by much less documentation, much shorter iterations, and much more focus on team collaboration and accountability.

So, documentation.  This is the rough breakdown of the arguments put forward

1.  Supporting Waterfall

• Requirements are VERY traceable
• Planning is documented with all factors kept in mind
• Covering your back is easier
• Progress is much more straightforward and easier to measure
• The entire process is under much more control and can quickly detect problems in scope, time , cost, etc.

2.  Supporting Agile

• The documentation is short enough that people will actually READ it and know what's inside
• Conversations and working code are emphasized more than comprehensive documentation(see Agile Manifesto)
• Having less documentation makes it easier to respond to change, saves time

Notice first that these arguments may have some flaws in them and second that they both address needs of different bodies.  Medical and legal implementations obviously need to be better documented than other fields.

Let me share you some experiences from both worlds.  In a course in project management, we were asked to split into teams and create bids for improving an airport.  Each team turned in a recommendation and supporting documentation of some 20 - 50 pages.  Who won?  My team won for the very simple reason that I summed up all 20 -50 pages of work into one summary paragraph at the top.  The instructor read the paragraph, skimmed the rest to see it was there, and gave it to us.  He didn't even read all the proofs and whatnots!
Second experience, in agile this time:  I was assigned in a company to work on some schedule management stories.  They looked like they had been dropped for a year and were coming up again.  I looked at the progress that was done.  All I could find were some mocks.  I asked an agile leader.  Blank stare.  No one even remembered what the tasks were about!

I'm not going to get into a lengthy solution today, but I do want to show that while both have good points, improperly managed, they can both be disastrous.

What I do want to propose is this.  Documentation should be integrated in, not apart from, tasks as much as possible.  QA is probably the most advanced in this sort of thinking.  For example, they will automate tests and use the automation itself to document and show what the code is supposed to do.  Simple idea but it combines a host of ideas together such as agile, TDD, management of technical debt, etc into a simple way of documenting and maintaining what the code actually does.
While this is the golden example, and may not be possible to this extent in many situations, the idea should be that by the very structure of the program you are presenting you should have documentation built in.  Just like a design artist wants the user to not even be aware of the work of navigating, a Project manager should, to his fullest extent, have stakeholders not even aware of the work of digging up numbers, reports, etc.

We'll talk more on this later and suggest some practical implementations.