On solving problems

As developers, our job is finding solutions to problems. Finding a good solution to a problem that has not yet been solved will give the business a tremendous advantage over its competitors. This is where the money comes in. As software developers we play a key role in this process. It has indeed become apparent that Software is eating the World.

We will look at a few tools that can be used to think about the problems we need to solve. An integrated approach like this (and these are certainly not the only tools you should use) is what I call ‘Real Enterprise Architecture’, regardless of who actually performs the necessary analyses.

How to find solutions

Although the fact that problems need solutions is fairly obvious, it is not at all clear how we can find the right solutions. However, there are some tools to help us. The Cynefin framework, created by Dave Snowden guides us to what type of solutions or explanations might apply. Definitely watch the video, if only to find out how to pronounce the word Cynefin. There are five different regions where a problem can live in.

Diagram of the Cynefin framework
Diagram of the Cynefin framework (from https://en.wikipedia.org/wiki/Cynefin_Framework)

These regions are defined by the relationship between cause and effect. The first four regions are:

Obvious
Here the relationship between cause and effect is obvious to everybody. The approach through which we find solutions is: Sense – Categorise – Respond. Best practices can be applied to solve the problem.

Complicated
The relationship between cause and effect in this region is not obvious, but can be found by careful analysis, investigation or with the help of expert knowledge. The approach through which we solve problems here is: Sense – Analyze – Respond. We apply good practice here.

Complex
In the complex region the relationship between cause and effect can only be discovered in retrospect. Therefore we need to solve problems with the approach: Probe – Sense – Respond. By continuous probing we will discover emergent practice.

Chaotic
Here there is no relationship between cause and effect. The only approach is: Act (fast) – Sense – Respond. There is the possibility to discover novel practice.

The fifth region is called Disorder where people are not aware in which of the other four regions they operate. Therefore they will operate on gut feeling.

I find the Cynefin framework a very useful tool to think about the type of problems I am addressing. The Cynefin framework can be combined with other tools to make the right choices and investments.

Focus on the right problems

A lot of components are needed to fulfill the needs of a customer. If you would run a webshop, for example, you need also to manage the content on that website, you need hardware to run the site on, the hardware needs power, etc. One tool to make this visual is a ‘value chain map‘. It is a visualization technique by Simon Wardley. Follow the link to see a detailed description of it.

It works basically like this:

  1. Find out who your customer is
  2. Find out what the needs of that customer are
  3. Find out what components are needed to fulfill the needs of customer
  4. Find out what components are needed for the components in the last step
  5. Repeat step 4 until you can go no further

You can plot all these components in an evolution phase. These phases describe the maturity of the component. Here is an example of such a map of components mapped on their evolution phases.

Example of a 'value chain map'.
Example of a ‘value chain map’. (from http://blog.gardeviance.org)

Now you can compare your situation with the market (your competitors). When you have a custom built component that is actually a commodity, your competitors might be in a better shape than you are. However, you might have some components in genesis or custom built phase that competitors do not have. That makes you special .

Drawing such a map makes visible what kind of problems you might want to solve. Maybe you want to replace a custom built, hard to maintain component with an off-the-shelf alternative. Maybe you see what components are crucial and special to fulfill the needs of your customers with respect to the market.

Solve the problems in the value chain

If we examine the different evolution phases in the value chain map, we can immediately see that a different type of solution strategy is required for each of these phases. Let’s compare the evolution phases of the Wardley map with the Cynefin framework.

Commodity components
If we want to use a commodity component, everybody knows immediately what to do. For example the need for power can best be solved by an energy company.

Product components
When we need to buy or rent a specific product, for example an emergency power unit, we need some analysis and an expert to find out what the best solution in our case is. Another example is the need to buy or rent computing capacity. Do you need a datacenter or are you going cloud? This question can be answered by some analysis with some expert knowledge.

Custom build components
These products do not yet exist in the market. It is unclear what features will actually be needed. Nowadays, the way this is solved for software is by using an Agile approach. The Agile approach is all about the apply – inspect – adapt cycle. This is really the same as the Probe – Sense – Respond approach from Cynefin.

Genesis products
In uncharted territory there is no way to know if you move the right way. Just act and see what happens.

By now, it is probably clear to you that the four evolution phases of the Wardley maps map exactly to the regions in the Cynefin framework:

Cynefin Value Chain Map
Simple Commodity
Complicated Product / Rental
Complex Custom build
Chaos Genesis

In case you might wonder why I don’t mention the Disorder region. When you are in that region, first get out of it.

This is Enterprise Architecture

Now that we have covered all the above topics, it is clear what the main challenge for us software developers is. As you all know, every developer performs a bit of software architecture. But to get an overview of the whole system, to clearly see where investments and innovation are needed, or to know whether you need an Agile process or not, an organization needs to apply Enterprise Architecture.

Let’s add some extra columns to that table I’ve shown before.

Cynefin Value chain map Domain Driven Design Example solving strategy
Simple Commodity Use a library
Complicated Product / Rental Generic and Support domains Buy or outsource
Complex Custom build Core domain Innovate and invest
Chaos Genesis Just try stuff

Complex problems are the ones that provide real value. These need custom build solutions and either give a direct competitive advantage or enable your company to challenge the market leader with a better implementation. These parts of your domain need the most attention and will provide value. And this is what Enterprise Architecture is all about. Applying strategic Domain driven Design is valuable where the problems have some cause-effect relationship, even if it is not obvious, and where it is important to be better than your competitor. When things are really unknown, just experiment. Don’t reinvent wheels that are already a commodity.

Make sure that you review your situation regularly. In time, when competitors also invent your wheels, the complex problems of today will become complicated and eventually simple. You will need to find new complex problems to stay in front of the pack. This is why you might want to reinvent your own wheels from time to time.