Agile project management
Four goals of Agile documentation: Part one
This is the second article in a two-part series looking at agile documentation. Read Part One here. In this article, I’ll touch on the probably most frequently asked question with regards to documentation: How do you keep it up to date?
Before that, let’s dive into our third and fourth goals for documentation.
I finished off Part One of the article with an example of how I used a paper widget kit to efficiently explain a data model. That particular kit had a purpose beyond the basic communication of information. The database technology had been quite an important strategic decision in this company, but it was causing a lot of effort for our team. It wasn’t the best fit for the part of the system we were building.
The kit and a step-by-step approach gave us an effective and repeatable way to bring across our requirements and create empathy and understanding for our challenges quickly. It helped to keep emotional factors out of the repeating discussions and keep them efficient and fact-based.
In her article about "Empathy Driven Development", Duretti Hirpa describes the notion of an “empathetic codebase”. Creating a codebase like that means taking advantage of “any tool that will provide context”, including documentation. “Docs or it didn’t happen”.
By creating documentation, we show empathy for our fellow teammates and create an environment where people feel guided and safe when changing the software, especially the newer members of the team. This goal applies particularly to the category of documentation for onboarding and troubleshooting, like “readme” files, or checklists for things that are not automated yet.
Here is an example: When building a PDF document generator, the product owner stopped by our desks almost every day to report yet another case where they didn’t like how a particular “page break” was placed by the layouting algorithm. Sometimes a fix we did today would bring back a flaw we tackled the day before. To facilitate those discussions, we created a poster outlining the logic of the “page break” algorithm. It helped us show how introducing new rules could invalidate existing rules, and we could then discuss priorities on equal footing. Making the logic transparent helped the users empathize with us because we showed them the complexity in a more understandable way, and they also understood the scope of the implementation better. It made us feel like we were creating this feature as a team, not as two opposing parties of “feature requesters” and “feature implementers”.
Slack’s overview of how the software determines the need to send a notification is another nice example of this.
Identifying candidates for this type of documentation
Creating an overview that demonstrates the complexity of your solution, as just described, also helps create empathy with other technologists. It helps them understand why the problem you’re solving is hard, which is a good foundation for a constructive discussion.
But what if you can’t understand why the problem is hard, because nobody knows what the problem was in the first place?
In his 2011 blog post about architecture decision records, Michael Nygard writes: “One of the hardest things to track during the life of a project is the motivation behind certain decisions.” But, he adds, unless you understand that rationale, you can only either accept the decision blindly or reject it blindly. Writing down the reasoning, context, and consequences of a decision right after you take it can hugely improve future decision-making.
You can record decisions with many tools, from a Word document to a Wiki, to simple text files. I’ve successfully used this little tool to create markdown files that were then version controlled along with the code. However, the tool isn’t so much the challenge here. The difficulty is to get into a routine of creating this really valuable form of documentation. We have to overcome the urge to just get on with the execution once we’ve come to a decision, and pause for a bit of documentation while the details are fresh on our minds.
Identifying candidates for decision records
Once you get better at identifying these moments, remember not only to describe the solution you went for, but also the problem and the context. For example, if you write down that you chose technology X because it’s scalable and supports Docker, then you’re actually listing characteristics of your solution, but not really why you need them. Why do you need scalability, why do you need Docker support? That reasoning will help your future self decide if circumstances have changed in a way that allows for a less scalable solution, or for one without Docker. The Nygard-recommended structure of “Context – Decision – Consequences” helps to focus on a problem description.
The following are a few principles I use to try and keep the more high-level, non-executable documentation reasonably up to date.
Having as little documentation as possible is the only long-term protection for outdatedness. And the more details you add, the higher the probability that you will have to update them soon. Again, think about the value a piece of documentation or detail brings, and remember that you will have to maintain it, just like every line of code you’re writing.
Also, don’t be afraid to throw things away! Some of the examples described earlier are actually only useful temporarily, it’s okay to throw them out when they aren’t helpful anymore.
Find ways to include the grooming of your documentation in your team rituals. I find a weekly “developer huddle” useful for this. This is a meeting in which the team discusses technical topics and questions that have come up during the week.
You should also make good use of team member rotations to check the current usefulness and understandability of your documentation.
Make sure the creation of documentation is a collaborative activity, to create collective ownership. This will increase the probability that all team members have an eye on how up to date the docs are, and correct them when necessary. If only one person on the team is creating documentation, they won’t keep up in the long term.
I hope I could show with these examples that the process of creating and grooming documentation can be a great catalyst for some of the other agile principles, like “Simplicity”, “Business people and developers work together daily”, “Attention to technical excellence and good design”, or “The team regularly reflects on how to become more effective”.
If you’re somebody who tends to create a lot of documentation, and you want to reduce some waste: A focus on the value documentation will help you prioritise.
If you’re somebody who creates very little documentation, reflect on if you’re missing some of the values mentioned, and what types of documentation would improve your team’s effectiveness.
Before that, let’s dive into our third and fourth goals for documentation.
3. Create empathy
Documentation creates understanding - not only an individual’s understanding of the code or the architecture but also understanding in the sense of empathy between the people involved in building it.
The empathy between tech decision makers and developers
I finished off Part One of the article with an example of how I used a paper widget kit to efficiently explain a data model. That particular kit had a purpose beyond the basic communication of information. The database technology had been quite an important strategic decision in this company, but it was causing a lot of effort for our team. It wasn’t the best fit for the part of the system we were building.
The kit and a step-by-step approach gave us an effective and repeatable way to bring across our requirements and create empathy and understanding for our challenges quickly. It helped to keep emotional factors out of the repeating discussions and keep them efficient and fact-based.
Empathy among developers
Working on software without guidance, without documentation, is anxiety-producing.
In her article about "Empathy Driven Development", Duretti Hirpa describes the notion of an “empathetic codebase”. Creating a codebase like that means taking advantage of “any tool that will provide context”, including documentation. “Docs or it didn’t happen”.
By creating documentation, we show empathy for our fellow teammates and create an environment where people feel guided and safe when changing the software, especially the newer members of the team. This goal applies particularly to the category of documentation for onboarding and troubleshooting, like “readme” files, or checklists for things that are not automated yet.
The empathy between developers and non-developers
It’s not uncommon for developers to complain about their product managers, saying they “don’t really understand how the software actually works”. Documentation can help developers bridge that gap.Here is an example: When building a PDF document generator, the product owner stopped by our desks almost every day to report yet another case where they didn’t like how a particular “page break” was placed by the layouting algorithm. Sometimes a fix we did today would bring back a flaw we tackled the day before. To facilitate those discussions, we created a poster outlining the logic of the “page break” algorithm. It helped us show how introducing new rules could invalidate existing rules, and we could then discuss priorities on equal footing. Making the logic transparent helped the users empathize with us because we showed them the complexity in a more understandable way, and they also understood the scope of the implementation better. It made us feel like we were creating this feature as a team, not as two opposing parties of “feature requesters” and “feature implementers”.
Slack’s overview of how the software determines the need to send a notification is another nice example of this.
Identifying candidates for this type of documentation
- When a lot of bugs are reported, that turns out to be caused by the misunderstanding of a particular feature.
- Features that are going through a phase of change and consist of an elaborate system of rules and logic. Everybody involved needs a good understanding of that logic to work on the change together.
4. Help our future selves make informed decisions
“If you look at another engineer's work and think, ‘That's dumb. Why don't you just…’ Take a breath. Find out why the problem is hard.” Adrienne Porter FeltCreating an overview that demonstrates the complexity of your solution, as just described, also helps create empathy with other technologists. It helps them understand why the problem you’re solving is hard, which is a good foundation for a constructive discussion.
But what if you can’t understand why the problem is hard, because nobody knows what the problem was in the first place?
Architecture decision records
In his 2011 blog post about architecture decision records, Michael Nygard writes: “One of the hardest things to track during the life of a project is the motivation behind certain decisions.” But, he adds, unless you understand that rationale, you can only either accept the decision blindly or reject it blindly. Writing down the reasoning, context, and consequences of a decision right after you take it can hugely improve future decision-making.
You can record decisions with many tools, from a Word document to a Wiki, to simple text files. I’ve successfully used this little tool to create markdown files that were then version controlled along with the code. However, the tool isn’t so much the challenge here. The difficulty is to get into a routine of creating this really valuable form of documentation. We have to overcome the urge to just get on with the execution once we’ve come to a decision, and pause for a bit of documentation while the details are fresh on our minds.
Identifying candidates for decision records
- Decisions that needed a lot of discussions to get to
- Decisions that are harder to change
- Imagine somebody joining your team in the future: Do you think they would challenge this decision without having more context?
- “Whack-a-mole” decisions: Every two or three months a team member brings them up and wants to change them, only to find out after a day of revisiting that there were actually good reasons for the decision. Having a clear record of the original decision and requirements makes these regular challenges more efficient. It then usually boils down to checking the documented list of requirements in the record and asking “Has the problem or the context changed in any way?”
Once you get better at identifying these moments, remember not only to describe the solution you went for, but also the problem and the context. For example, if you write down that you chose technology X because it’s scalable and supports Docker, then you’re actually listing characteristics of your solution, but not really why you need them. Why do you need scalability, why do you need Docker support? That reasoning will help your future self decide if circumstances have changed in a way that allows for a less scalable solution, or for one without Docker. The Nygard-recommended structure of “Context – Decision – Consequences” helps to focus on a problem description.
How to keep it up to date?
One of the most frequently asked questions with regards to documentation is: How do I keep it up to date? The short answer is: You probably don’t - at least not 100%. In the end, anything that is not necessary to keep the software running will ultimately be out-of-date to some degree, so executable documentation forms like readable code, tests and scripts are important foundations. The techniques described here are all just complementary to that.The following are a few principles I use to try and keep the more high-level, non-executable documentation reasonably up to date.
Create as little as possible
Having as little documentation as possible is the only long-term protection for outdatedness. And the more details you add, the higher the probability that you will have to update them soon. Again, think about the value a piece of documentation or detail brings, and remember that you will have to maintain it, just like every line of code you’re writing.Also, don’t be afraid to throw things away! Some of the examples described earlier are actually only useful temporarily, it’s okay to throw them out when they aren’t helpful anymore.
Include documentation grooming in team rituals
Find ways to include the grooming of your documentation in your team rituals. I find a weekly “developer huddle” useful for this. This is a meeting in which the team discusses technical topics and questions that have come up during the week.
You should also make good use of team member rotations to check the current usefulness and understandability of your documentation.
Make it visible
If a piece of documentation is buried in some corner of your wiki that you rarely go to, then it will most certainly wither away. If it’s up on the wall on the other hand, for everybody to see, or in a “readme” file that every new team member comes across, outdatedness will be noticed and corrected more frequently.
Create ownership through collaboration
Make sure the creation of documentation is a collaborative activity, to create collective ownership. This will increase the probability that all team members have an eye on how up to date the docs are, and correct them when necessary. If only one person on the team is creating documentation, they won’t keep up in the long term.
Agile + Documentation = <3
Ultimately, the code is the only truth describing the current state of your systems. High-level documentation serves as maps to find entry points and your way around. And you need to put special emphasis on documenting the things that code cannot tell you: History and context. The self-documenting code does not help you challenge past decisions, or understand why things were done a certain way. Therefore, at the minimum, consider writing decision records, and take good care of your commit messages.I hope I could show with these examples that the process of creating and grooming documentation can be a great catalyst for some of the other agile principles, like “Simplicity”, “Business people and developers work together daily”, “Attention to technical excellence and good design”, or “The team regularly reflects on how to become more effective”.
If you’re somebody who tends to create a lot of documentation, and you want to reduce some waste: A focus on the value documentation will help you prioritise.
If you’re somebody who creates very little documentation, reflect on if you’re missing some of the values mentioned, and what types of documentation would improve your team’s effectiveness.
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.
