Knowledge

What Should Documentation Be?

The problems Business Intelligence organizations solve in organizations are generally the same. Pull some data out of somewhere, synthesize the data, analyze it, then create a picture of what is happening, what is going to happen, or what has happened. Working in small and large organizations, I’ve had the pleasure of seeing a variety of different processes used to deliver these insights. These range from the overbearing, and associated documentation that crush people’s productivity, to the lightweight that creates quicksand beneath teams feet through the lack of knowledge transfer.

Seeing the overbearing and the extremely light weight, there’s one conclusion I’ve arrived at concerning documentation…

Document as Little as Possible

 

documentation_joke

Relevant literature

Don’t commit time to things that aren’t creating revenue or helping the business. Looking at IT projects, there is no doubt that the more documentation that there is, the less value there is. The perfect example came about when having a beer with a former co-worker.

 

It was brought up that the process at the company we both previously worked at had documentation that took longer to create then coding, testing, and implementing the change. Additionally, this painstakingly crafted documentation that the engineer had to spend time tracking down information for didn’t result in documents that would be useful to the team doing the work going forward. The process decreed that you must document X, Y, and Z in order to deploy the change/implementation so that’s what was done. The fundamental truth is that the “…benefit of having documentation must be greater than the cost of creating and maintaining it.”

Some people believe in the exact opposite of over documentation. Nothing should be documented. The code/implementation should speak for itself. This may work when you have a small size IT application the will always be managed by the same group of individuals (which likely won’t happen). Once you reach an application spanning multiple servers, teams, and databases the expectation for the code/implementation to “speak for itself” in a timely manner to those who have to report and get analytics out is unreasonable.

So, what’s being proposed in this rant? The only useful documentation that I’ve seen documents the “Why” and the “How”. Everything else doesn’t create value for the organization, as the cost to maintain and develop the documentation is too high.

Why

Creating a BI Product entails connecting the business process to an application(s) or database(s). Depending on the environment that you’re working in, Inmon, Kimball, or something else entirely, you need to know the answer to why things in your system exist. The “Why” is important not only from a high leadership level, but also at a low technical implementation level. The “Why” statement done at the low level helps to ensure that a team is using previously created tools and implementations as designed. And if a change is made that goes against the original “Why”, it is intentional and by design.

As an example, working on the Vehicle Profitability by VIN project, the Data Architect created both Inmon (3-NF) and Kimball (dimensional reporting) structures on the project. The “Why” was made extremely apparent through documentation, so the teams knew how to use the current implementation to achieve their goal in the best way possible.

Are you importing new invoice data? That should go into the wholesale invoice structure so that it flows up in the existing fact that contains the revenue information for vehicles which our reports feed from. Why? Because we want a single source of truth for vehicle revenue.

When documentation providing the “Why” for technical implementations exists, it makes adding on and changing the existing processes and assets easier. As opposed to re-inventing the wheel over and over.

How

Payment-Data-Flow-Diagram

basic data flow diagram

So after we know why something exists, the other piece that is useful for documentation is the “How”. The “How” shouldn’t be step by step instructions, it should function like a high level map. Data Flow Diagrams are a great example of “How” documentation that I’ve found useful for Business Intelligence products. Armed with the Data Flow Diagram and the “Why” of the design, team members who need to report on, extend, maintain, or refactor a system will be able to make informed decisions.

 

 

Make It Useful

At the end of the day, documentation gets in the way of creating code/analysis/direct business value. So the argument for spending time creating documentation is hard to make when someone hasn’t experienced the pains associated with lack of documentation. Lack of architecture that makes sense, misreported numbers, time wasted building processes that do exactly what existing processes already do.

Without documentation, maintaining and using a system or process as intended is impossible. With documentation that is accessible, searchable, and focuses on the “How” and “Why”, organizations can make smart and informed decisions of where to spend time, how to tweak things, and how to get value from their assets.

 

Why Storytelling is Required

Storytelling and marketing is something that seems to be undervalued by technical individuals in the information technology field. The reason why I’m talking about this? Recently at SXSW in Austin, Contently hosted a talk where Shane Snow discussed the power of storytelling. While the audience attendance was a definitely skewed towards the marketing industry, the concepts that were presented can be applied to any idea or presentation that technical people are trying to sell to customers, managers, or co-workers.

Story Continuation

Shane, in his talk, brought up some interesting statistics that prove a powerful point. People tend to gravitate towards stories that build on existing lore and story lines. The area that was pointed to as proving his point? Movies. Shane mentioned a metric that can be used to demonstrate this. Movie revenue. The question is, does Shane’s theory prove true?Spiderman Movie Layout

If you look at the above, grabbed from The Numbers, it clearly shows a relative trend of decreasing sales revenue for Spider-Man movies. On close examination though, the biggest drop in revenue (~15%) when comparing a movie to its predecessor occurred between Spider-Man 3 and The Amazing Spider-Man…When the continuous story line from the first  three Spider-Man movies was broken.

Jurrassic park - Revenue

But…doing some spot checking, also reveals the opposite to be true. Looking at Jurassic Park’s history of revenues, it appears that sequels where the story line is broken can make just as much (or much more). In order to prove out this theory, it appears that more analysis would be needed to prove this point objectively…Maybe this is an oddity with re-boots of classic series?

Regardless, in SOME cases, when movies break from a continuous narrative there appears to be increased risk of people abandoning interest in the movie/idea.

Familiarity

Additionally, Shane mentioned the power of familiarity. When traveling abroad and being around unfamiliar scents, sounds, and tastes, people tend to gravitate towards the known. The perfect example that many can relate to? Beer. Heineken is sold in over 170 countries. When someone is given the choice between a familiar brand that may even be disliked and an unfamiliar brand, people generally choose the known brand that has familiarity. Thinking about the odd concoctions one might encounter when travelling abroad, what would you rather have?

Complexity of Content

The last point that was presented? The easier that content is to read and understand, the more popular it will be. Mark Twain has books that come up at around a 5th grade reading level according to Scholastic’s system. Even the more modern classics, depending on your point of view, come up at around the same reading level. The lesson? It may make us feel good communicating with big words, but it is not the most effective way to communicate.

So What?

At the end of the day, while these ideas are interesting, what can we learn? Everyone is trying to sell stories every day. In technology/knowledge work, it’s a new design or approach to solve a problem. These strategies can be used to communicate an idea effectively and gain the support of others when combined with logical arguments. Establish a narrative that creates a vision and compelling continuous story line. Do it in a way that anyone could understand, from developers to directors, technical to non-technical. Establish a brand, identifier, or name that people can familiarize themselves with. If technical people peddled ideas that have been implemented half as well as they implement them, it would be to everyone’s advantage. Playing to people’s logic works usually…playing to logic and human nature? Couldn’t hurt.