Frameworks for problem solving in complex environments

Banner photo hands holding lightbulb.

Critical thinking and analysis are not easy skills to pick up.  Universities used to teach them as pre-requisites, but political and commercial pressures have largely driven them out of more vocationally-focused curricula.

However, there are some models in management theory that can be quite useful in guiding problem solving, especially if the problem solvers suspect they may lack direct subject matter expertise.

We’ll look at three of these models here—

  1. A theory of knowledge checking and development. Pretty useful as a rock-solid method for evaluating whether a claim, policy, or argument is soundly based.
  2. A framework for dealing with ‘wicked’ problems which highlights that many problems cannot be addressed with the engineering-oriented ‘solutioneering’ favoured in science/technology/engineering/mathematics (STEM) practices.
  3. A reworking of the Cynefin model to help manage the fear of change and overcome apparently chaotic conditions.

Knowledge management

Diagram of DIKW information to wisdom hierarchy.
FIGURE 1: A hierarchy of data moving to knowledge and wisdom.

In the 1970s, academics Chris Argyris and David Schön came up with the notion of single and double loop learning, which was later developed further to embrace the idea of triple loop learning.  The model appears to be influenced initially on the data-information-knowledge-wisdom (DIKW) hierarchy.

The DIKW scale proposes that human activities transform raw data into information, and then knowledge and wisdom.  Wisdom in this context is probably a high level of informed and considered professional and social competence in pursuing self-directed goals and outcomes.  The origin of the model is uncertain, except it seems to emulate Maslow’s hierarchy of needs, developed sometime in the 1940s.

Figure 2 below shows what Argyris and Schön did with the DIKW hierarchy, in a model that is relatively easy to adapt for personal, technical, business, and policy purposes.

Diagram of original and adapted triple loop learning model.
FIGURE 2: Single, double, and triple loop learning.

What the diagram shows is that—

  1. Raw data is just unordered noise.  Like static on a radio.
  2. We can impose order on raw data to make it information. Like selecting sequences of raw data that are dated and themed to imply relationships, as in historical events, or astronomical observations.  It is still neutral information in that it does not yet carry within it any distinct human purpose, just an arbitrary order.
  3. We can string together and/or contextualize information to use it for practical purposes, like rules of grammar, principles of philosophy, or formulae for rocket fuel.  This is knowledge with explicit embedded purposes, though it may remain neutral about why it should be used.  At this stage of information or knowledge management there can be re-checking with foundational data (single loop learning), or with both data and knowledge (double loop learning) to check, amend, or update received information.
  4. We can integrate or further develop knowledge into a wider understanding of one or more domains of knowledge, to propose an ‘expert’ theory, or describe principles of professional practice, possibly combining various pieces of knowledge to create entirely new knowledge, and adding the context of why knowledge should be used for certain purposes.  At this level the problem solver double and triple checks all that underpins the knowledge at hand, and perhaps also innovates by highlighting newly important patterns in the process of revisiting the entire chain of data-information-knowledge.
  5. Transposed onto everyday experiences, raw data includes ‘facts’ we may seek out, but also the messages we are bombarded with through social and conventional media as well as social and professional contact.
  6. In all cultures there are gatekeepers or sources of arbitration about what information means: academics, professionals, journalists, government, and so on.  Problem solvers accessing such information should be aware that gatekeepers’ interpretations usually serve particular interests.
  7. Cultural and social conventions dictate how ‘consensus’ meanings are arrived at for most types of knowledge in a variety of theoretical, practical, and policy contexts.  So, for example, we have unwritten agreements about the credibility of peer review and orthodox professional practices in various disciplines.
  8. Analysts are the people who come to the whole process for a specific problem solving purpose, and to question and adapt existing or conventional wisdom as filtered through the previous steps.

In the Argyris and Schön version at the top of the diagram, the triple loop process is just checking that data and information actually supports the received meanings of knowledge.  In our adaptation, the triple loop must also include the constant question about continuing relevance and who benefits from structuring data and information into their conventional forms and meanings.  The questions at this stage must include whether conventional wisdom is capable of addressing the problem we’re trying to solve without creating new complications.

With this tool in mind, let’s move on to a significant caution about reaching for overly simplistic solutions to complex problems.

Wicked problems

Problems relating to human purposes, motivations, or a sense of self are always ‘wicked’, not because they misbehave, but because they often have no fixed solution, no single method for their resolution, no repeatable process for addressing similar circumstances, and no answers that may not be regarded as both a success or failure at different times, and sometimes even both at the same time.

Such problems are quite common in politics, where problems are always conceived of in terms of human perspectives, solutions are almost always compromises that may not please everyone (or even anyone), and chosen solutions may be more stop-gap than permanent fix, with everyone knowing the issue will have to be revisited.  In rapidly changing social and business environments, the same ‘wickedness’ occurs in personal and professional contexts.  Solutions become a balancing act between what is desired and what is possible within given constraints, including especially timelines and resources.

Diagram of ten features of Rittel and Webber's wicked problems framework.
FIGURE 3: Complexities of wicked problems.

One very important point is to recognize that a reliance on a binary, oppositional paradigm can be unhelpful.  The premiss that there are two sides to every story is more often false than useful.  In lived reality, sometimes there is only one side, but mostly there are many.  It is the reflection of cultural and social multilateralism that many different ideas can coexist comfortably, and that instead of choosing one or another solution, we can choose parts of many different options in an intellectual eclecticism aligned with what’s best, rather than a reflexive ‘either/or’ determinism that limits choices to only two, often opposing options.

The power in Rittel and Webber’s formulation of wicked problems is its recognition that STEM ‘solutioneering’ simply does not apply to managing human wants and needs.  Not only do the STEM disciplines lack the tools to fathom human motivations, but they simply cannot deal with problems that have no defined parameters at all, including especially no indication of where to start or finish.

Yet, almost all problems have a strong human dimension, especially in public policy, but also in business, which must focus on satisfying human wants or needs to drive human desires for profitability, within a framework of laws and regulations designed to protect agreed public benefit standards.

Using the wicked problems paradigm, problem solvers can focus on the kind of practical but informal methods we can observe every day in workplaces, legislatures, and even families: it is a process of discovery, negotiation, compromise, and temporary accommodations we all know may need to be revisited or supplanted.

It is possible that significant numbers of people are far more competent in informal problem solving than they suspect, and adopting the wicked problem framework may help them to discuss their ideas and persuade their audiences more successfully than relying only on references to intuition or commonplace examples.

The importance of persuasion should not be underestimated.  People fear change and may oppose it for that reason alone, while some others may be committed to particular views that are challenged by innovative new approaches.  This can be as true in domestic circumstances as in social and professional settings.

If we were to examine a number of high profile failures in IT and business projects, or public policy, it is likely we would find excessive reliance on one-sided problem solving frameworks as the root cause.

But how do we know when we should chance departure from orthodox models and practices?  To answer that, we’ll look at an adaptation of the Cynefin model.


Diagram of David Snowden's Cynefin model.
FIGURE 4: The original framework.

Developed in the later 1990s by IBM’s David Snowden, this management model was probably influenced by the rising tide of alarmism about what might happen when two digit date fields in embedded computer systems were forced to tick over from 99 to 00.  There was quite a bit of anxiety about the thought of scores of simple systems ceasing to function at the turn of the century.

Snowden conceived of disorder as a default state, to be managed by adopting an approach to suit specific problem solving contexts, proposing four archetypes.

  1. Simple environments can be managed with uncomplicated and repeatable solutions that are readily learnt and used by most people.
  2. Complicated environments are manageable by applying expertise, as acquired through formal education, training, professional practice, and experience.
  3. Complex environments are transitional states in which new expertise and skills must be developed to deal with new kinds of problems.
  4. Chaotic circumstances require reactive measures like rapid response disaster management, possibly requiring immediate ad hoc or band-aid solutions, like putting out fires.

Underlying Snowden’s Cynefin framework is an engineer’s terror of circumstances in which the orderly progression of STEM logics prove inadequate to deal with the problems faced.

That phobia of disorder highlights one of the greatest tensions in the modern era: between the sciences and humanities.  In recent decades the rise of IT-related economic drivers has eclipsed the humanities and often devalued their importance in education.

Diagram of adapted Cynefin framework for 'wicked management'.
FIGURE 5: Cynefin adapted.

It could be argued that this ‘tilt’ of Enlightenment values too far towards only the technical rationalities has created a generation of private and public sector leaders and professionals who are unable or unwilling to think creatively and independently, relying too much on the determinism of STEM logics.

The Cynefin framework has been developed into a detailed leadership framework since the 2000s, but we only need to tweak the original a little to offer us a guide for ‘thinking outside the box’ when addressing problems that appear to be impossibly complicated, defying solution with conventional methods.

In our adapted version, we move away from fears about chaos, and embrace instead the notion of managing wicked problems in a state of permanent disruption.

  1. Simple problems can be addressed using elementary processes in widespread use that are readily observed and emulated.
  2. Complicated problems require more formal methods we might find in academic and professional practices, or other sources of subject matter expertise.
  3. Complex problems are the kind faced by trained professionals in advanced industrial, economic, and social systems.  Subject matter experts draw on diverse bodies of knowledge and practice to develop solutions
  4. Transformational environments require responding to unexpected new influences, needs, and circumstances with creativity and innovation, quite possibly by challenging all underlying assumptions, the way discussed in triple loop learning above, and as applied to the wicked problem paradigm.

Putting it all together

The principal utility of the triple loop learning idea is to remind us to check all of our assumptions when we are called on to problem solve, and to help us visualize how we might extend existing knowledge, or even develop entirely new insights.

The wicked problem model is a reminder that all problems address human needs or wants, not some abstract engineering conception of ‘start here, end there’.  That means thorough analysis of what the clients and end users want, what is possible to deliver within the existing constraints, and then how to sell the less than ideal ‘solution’ to stakeholders, even if it is always only going to be a stop-gap.

This approach to problem solving has sometimes been called ‘agile’ in IT disciplines, but can devolve into deliberately mediocre solutions if it becomes a mundane method in which problem solvers become used to delivering only sub-par solutions, indefinitely postponing any attempt to seriously grapple with underlying complexity.

The Cynefin model reminds us that we should adapt to the distinct environments in which we solve problems.  So, for example, it could be that we are in the transformational environment in which new technologies render our business model irrelevant, or the one in which we, as parents, are suddenly confronted by children grown into unpredictable teenagers.  In either case we might have to address such problems by discarding preconceived notions about a pre-existing ‘normality’, and focus on putting out fires first.  Only when the fires are out can we move on to consider what a transitional or new normality might look like.

This approach to problem solving has conspicuous parallels with the situational leadership paradigm.

Most importantly, we should recognize in all frameworks and models that they are no substitute for independent critical thought.  They are only a start to an intellectual process that questions everything, not to be contrary, but to make sure we aren’t just solutioneering.

Even then, it may be that the ‘clients’ for problem solving activities reject the best options, opting for more conventional solutions.  Nevertheless, original thinking will never be ignored in professional settings.

People capable of developing creative and diverse options are in much shorter supply than we might imagine.

Be the first to comment

Leave a Reply

Your email address will not be published.