Showing Our Work: Design Economics

Showing Our Work
Photo by Irving Rusinow [Public Domain], via Wikimedia Commons
You start a new project and receive a heap of content from stakeholders. Your shiny new stack of content arrives with an expectation from the project owner that you deliver that content (every ounce of it) to the mental doormats of their audience. In an attempt to do the right thing, you dutifully conduct an analysis of the performance & skills that the program you’re helping needs to address. You get to know the audience. You frame a set of performance objectives around a reasonably rigorous analysis.  Despite this effort, you’re still stuck with big pile of content.

The tail wags the dog. You dive into the rabbit hole head first. You search for aesthetically pleasing layouts and graphics that will grab attention. You look for ways to make an awful experience less awful. Soon you’ve invested a ton of time trying to engineer treatments for content and you’ve lost sight of the purpose of the solution (if ever there was a purpose).

Sound familiar? You bet. When you enter into a battle with content treatments, it’s easy to get lost in the noise. That noise makes it easy to lose sight of the things we can do to really make the biggest difference. Unfortunately, this unenviable situation is far too common. It seems unfair to give information this much of our attention, doesn’t it?


In the performance solutions field (ISD, learning experiences, whatever you want to call what we do), we often look at other fields of discipline for guidance and inspiration. We often examine other design disciplines or areas of media production, looking for better ways to position and polish our communication. As helpful as these fields might be, we may be overlooking an important, albeit less sexy, discipline. The field of economics may hold one key to making solutions worth the effort we put into them.

Economics is a process that converts inputs that have economic value into outputs that also have economic value.

In design, we deal with many factors and artifacts (currencies). Ideally we’ll weigh all of the inputs we can to make decisions that provide the best trade-offs and consequently the best outputs and value for all stakeholders, given the inputs we have to work with. Content is just one of these currencies. Other currencies include:

  • Target accomplishments
  • Tasks that enable the accomplishments
  • Skills that enable the tasks
  • Components of the skills that enable the tasks
  • Audience factors
  • Environment factors
  • Technology supports / constraints
  • The client’s wants
  • The client’s needs

An experience itself is defined by economics. Good experiences consist of moments that drive towards the design intent or the goal of the solution. That’s a deep one, I’d rather go down that route in another post.

Good economics supply greater value in the outputs than the inputs, making the outcome worth the effort. While it’s not as sexy as User Experience (UX) or game design, doesn’t economics sound like a field worth looking into?

Make the outcome worth the effort.

Often, maybe too often, we make decisions that seem to draw magic lines between content and solution without consideration of the intent of the aggregate of those decisions. Starting with purpose in mind is a sensible guideline. It’s easy and intuitive to point to this and say “Yeah, that’s the right way to do it” but it’s challenging to follow in practice. I guess that’s why we call it a discipline.

Here are two suggestions you can use to battle the magic leap and increase your effort to outcome ratio.

  1. Look at your solutions as economic problems. Map the structure of your problem so you can know exactly how the structure of your answers matter in the big picture. Understand the structure of the task down to the skills and all of the factors that shape the execution of the skill. Know what concepts drive the skill. Know common interpretations of a rule that influences the execution of the skill. Use this structure to validate your decisions. If something doesn’t stick, dump it.
  2. Don’t start with content. Don’t get hung up on content. While content is often an important artifact, it’s the wrong starting point. And in many cases it hadn’t earned the effort we often invest in it. Getting lost in the (often horrible) noise of treating content is a great way to lose sight of the big picture and is a terrible way to deliver value. It’s hard to win the performance war if we tie all of our energy up in a content battle. Content is a means to an end. Starting with a focus on content will make it a slog for you and a slog for your participants. Pick another starting point.

Everything we do and every choice we make has a cost. Every cost stacks debt. That debt can manifest as time, money, disappointment, or simply things you need to fix to make it work (costing time, money, and disappointment). This debt adds up. Design is an economic problem. How do you account for your design debt?

In follow-up posts, I’ll talk a little about a process I’ve been exploring to expose a deeper look at the currencies we use in design and a way to change the tendencies we have that give so much weight to content.

How are you showing your work?

Tech, People and Systems

The article below was written for an internal audience to illustrate the potential of the Experience API (Tin Can API). This brief overview contains narrative descriptions of four use-cases.


The use-cases described below illustrate projections and opportunities that could help to resolve some organizational challenges. The use cases described below do not represent plans. This article doesn’t necessarily represent the viewpoint of the U.S. Coast Guard or the Department of Homeland Security.

The DoD ADL Initiative is working on a set of technology interoperability standards to enable training and learning systems. This new set of standards is called the Training and Learning Architecture (TLA). The first technology standard to emerge from the TLA is the Experience API (XAPI) also known as the Tin Can API. Differing from current systems that focus on connecting content with systems, this standard creates a language and technology framework that focuses on connecting people with systems. More specifically, the Experience API enables systems to capture the actions, activities, experiences, and accomplishments of people. Continue reading Tech, People and Systems

Artificial Competence

This post will mix a kitchen remodel with some video game nostalgia. Believe it or not, these have something powerful in common. Artificial competence isn’t a bad thing if it gets the job done.

Chances are, you’ve played Super Mario Brothers at some point in your life. Super Mario Brothers is a side-scrolling platform game where you, the player, control Mario. Mario can walk, run and jump through puzzles, defeating enemies using permanent or temporary abilities. The designers of the game strategically placed performance support power-ups throughout the game to provide options for defeating puzzles and enemies. Without these power-ups, tasks within the game are more difficult to accomplish.


A kitchen remodel

We’ve been planning to remodel our kitchen since moving into our townhome 8 years ago. The old pine cabinets and light blue countertop had each overstayed their welcome. This year we took the plunge and decided to do it ourselves. I have some construction skills but had never tackled anything as complex as a kitchen. I was going to need a special kind of magic to get this done without help. Continue reading Artificial Competence

Defining Competence, Proficiency, Expertise, and Mastery

Think about how you’d answer these questions:

  • How do we as performers make the journey from not able to ready to perform and beyond?
  • How can Human Performance Technology (HPT) practices help remove friction and improve the environments where this journey happens?
  • If we don’t all have the same idea for what the pathway looks like, how can we work together to efficiently help the performer ascend to the levels they need to reach and heights that they aspire to?

While I’m not sure it’s always necessary, I think it’s important to create some sense of consistent gravity around critical anchors in the language of our discipline. Skill, proficiency, and expertise are among the concepts I would consider to be critical in a field that exists to support, facilitate, and improve the things that these terms represent. In this case the mechanisms by which we help folks climb the ladder, navigate pathways, and make connections seems to be more more important than the meaning of the terms (levels) themselves.

The post is still going to be about the lexicon for competence, proficiency, expertise, and mastery but it’s going to cover a bit of ground in the process.

U.S. Coast Guard Commandant Robert J. Papp

I had been planning to read an article written by ADM Robert J. Papp , Jr. titled Proficiency: The Essence of Discipline for a few weeks. I’ve known ADM Papp for a few years and he’s one of the many people I admire. His views on experience and proficiency have informed my own. As an organizational leader (he’s the four star Admiral in charge of the USCG and a genuinely cool dude), the Admiral walks the talk and the steps he’s outlined are indeed in motion. The article did not disappoint.  I recommend giving it a read.

This excerpt from ADM Papp’s article encapsulates the problem nicely.

I began speaking of proficiency in my first “State of the Coast Guard” address in early 2011, and it generated a flood of questions. During all-hands meetings last year, I frequently was asked to describe “proficiency.” I would reply by recounting how during a visit to a Coast Guard boat station I had asked the crew, “Who is the best boat coxswain?” Of course half a dozen boatswain’s mates immediately raised their hands. So I rephrased the question. “If the search-and-rescue alarm sounded and you had to go out in a severe storm, who would you want to be the coxswain of the motor lifeboat?” Everyone turned and pointed to the commander, a chief warrant officer boatswain (BOSN4) and surfman with more than 30 years of experience. Clearly, we all know proficiency when we see it. But how do we become proficient? And proficient at what?

~ ADM Robert J. Papp, Commandant USCG

Do we have a common language for the steps in skill acquisition?

As I read ADM Papp’s article, it struck me that while we often talk about proficiency, skill, and expertise in my field, we might not have a common idea for what these terms represent. If we don’t have a common concept for the meaning, it could be difficult for us to agree on the mechanisms we use to facilitate (or know when to get out of the way and trust in the strength of the network or the individual). When do we mediate and intervene? When do we let go? Tough questions if we don’t agree on the model for what skill progression looks like.

I talked about my views on the place “skills” live in the great network of being in Make the Structure Visible. For the purposes of this discussion, let’s assume that the outcome is the goal and the skill is one of the means to reach the goal. Let’s also assume that most, if not all, skills can be categorized with multiple different levels of mastery.

To start, let’s riff off of an inspiring post by Craig Wiggins with some martial arts flavor. These are probably some of the oldest mastery development models in existence. Ancient disciplines seem like a good place to start. The definition of levels is one facet, but looking at the model, the real value of the definition is in the mechanisms that bridge from one level to the next.

Shuhari (foundations, breaking away, transcendence)

Shuhari comes from a Japanese martial art concept that illustrates progression for attaining mastery through disciplined learning.

 shu (守) translates as protect or obey. In the context of Japanese martial arts, this is where learning progression begins. These foundations represent learning fundamental techniques, heuristics, and proverbs. During this stage, the subject of the discipline does not deviate from the forms presented by a single instructor.

 ha (破) translates as detach or digress. This is where learning progression expands beyond the foundations and the subject is encouraged to innovate and break free of the rigid foundations. During this stage, the subject explores the application of the foundational forms, making some forms their own and discarding others.

 ri (離) translates as leave or separate. In this stage, the subject completely departs from the discipline of the forms and opens themselves to creative techniques that align with the desires of the heart and mind within the bounds of laws, rules, and values. The subject is encouraged to use what they have acquired during shu and ha to transcend teachings and acquire mastery, making their own connections and relationships to the discipline.

Chinese martial arts such as Wushu also offer a three-phase mastery concept.

  • Earth – Basics.
  • Human – Ready to learn (equated to 1st level black belt, so I suppose you’re not really ready to learn in this context until you’ve reached performance competence.)
  • Sky – No conscious thought.

It looks like the concepts of defining states or levels along the path to mastery have been around for a while. These relatively simple models represent a disciplined progression and transformation from novice to apprentice through to journeyman and master.

The Flowers Model of Capacity

I’ve long held my own categorization for levels of performer across three dimensions: Selection, interpretation, and execution.  Following novice through master, each level is matched with a more colloquial label.

Novice is matched with the label Burden since it can be a challenge to find the right resources to grow a novice.  This isn’t meant to be an insult to the novice. Everyone has to start somewhere. This merely implies that the development of the novice can be a larger draw on an organization’s resources than the development of the higher levels. That doesn’t prevent a senior performer from regressing or not growing beyond the novice level. We don’t always do the Novice relationship (opportunity) justice. The burden level is about exchange of value. When considering a person that’s new to an organization, training and patience are an investment in future returns.

It’s purely coincidental, but this is really similar to the Shuhari model of progression through mastery. The chart describing this model represents three main activity areas: selection, interpretation, and execution. I still like this model for some types of work but I don’t think it’s good enough to map into a common lexicon.

Dreyfus’ Model of Skill Acquisition

This model of skill acquisition comes from Hubert Dreyfus, a philosopher and educator. The original model proposed that people pass through five stages in pursuit of skills: novice, advanced beginner, competent, proficient, and expert. This was later revised to encompass seven distinct stages.

Novice A novice is just learning the basics of a subject, unable to exercise discretionary judgment and has rigid adherence to taught rules or plans.

Advanced Beginner The advanced beginner is beginning to connect relevant contexts to the rules and facts they are learning. Folks at this level may have no sense of practical priority. All aspects of work may be treated separately and will likely have equal importance.

Competent A competent performer is able to select rules or perspectives appropriate to the situation, taking responsibility for approach.

Proficient A proficient performer has experience making situational discriminations that enables recognition of problems and best approaches for solving the problems. At this stage, intuitive reactions replace reasoned responses.

Expert  The expert performer is able to see what needs to be achieved and how to achieve it. This level of performer is able to make more refined and subtle discriminations than a proficient performer, tailoring approach and method to each situation based on this level of skill.

Master The mastery performer has developed their own style, extending expertise within a domain with their own synthesis of tools and methods.

Practical Wisdom This level was tacked on later at the behest of a colleague. This describes the assimilation of the master’s creations within the culture of a work unit or organization. In my interpretation, this is the closure of the cycle and describes the giving back from the master to the domain, enhancing the domain body of knowledge itself.

Let’s focus for a moment on the four levels in the middle: competence, proficiency, expertise, and mastery. These represent the progression from confidently able (decreased burden on the system – returns are beginning to emerge), intuitively able (solver), to crazy able (synthesizer), and beyond.

While I like this model, it doesn’t seem to offer a solid definition of the behavioral components of proficient, expert, or master. If it matters that these levels are attained, what signifies each milestone for a particular skill? How are these levels distinct?

James Atherton offers an extension that might be useful in narrowing the definition expertise and may be helpful in quantifying the meaning of and mechanisms for attaining proficiency, expertise, and mastery. He adds that an expert might be defined by the demonstration of:

  • Competence: The ability to perform a requisite range of skills.
  • Contextualization: Knowing when to do what.
  • Contingency: The flexibility to cope, adapt, and respond when things go wrong.
  • Creativity: The capacity to solve novel problems.

I’m not sure these round it out for me completely, but these do clarify the borders of meaning between these terms. To be useful, the definitions should ultimately define a clear distinction among proficiency, expertise, and mastery that sets these levels apart from the threshold of competency.

One question that seems to surface often, one that I’ve brought up as well, is the importance of skill mastery beyond performance competence. How important is mastery beyond performance competence? I think the answer is “it depends” but it seems like there’s a consistent human drive that requires a goal beyond status quo for satisfaction in life (including life at work). If we don’t define what those goals could look like and don’t set our expectations higher, we risk performance competence becoming the high water mark in organizational performance expectations. Shouldn’t performance competence be the minimum standard?

Wrapping this up

I think holding a common understanding of what each of these levels mean could help to better communicate relationships. The value of this common thread transcends the relationships between the levels. A common understanding could help us better map the what, when, and why of how we help folks navigate the journey from no skill to skill mastery or any stop in between.

Let’s continue the conversation. Does Dreyfus’ model resonate with you? What have you found? Does this matter as much as I think it does?

The Case for a Clear Lexicon Part 1: Orientations

I’ve been thinking about this post for bit. Dr. Clark Quinn has been thinking along similar lines, from a different angle at his Learnlets blog (

This post is about lexicons. What we call things, what these labels mean, and how these labels lose their meaning when folks develop a tendency to apply the same label to everything. Yes, courses, I’m picking on you again. Or am I?

Earlier this week, a co-worker and I discussed some recent feedback we received on a “course” the organization deployed earlier in the year. I had a really hard time seeing these products as a valuable solution when this set of courses were originally contracted for design and production last year. I still think we paid far more for these than the value they deliver, but there’s something this class of products appear to provide that I didn’t account for. From the feedback we’ve received, a few potential and (to me) unexpected value propositions seem have emerged:

  1.  These junior teammates were often involved in tasks away from operations and didn’t have the opportunity to orient to equipment and platforms while qualifying in other areas or performing entry level tasks. The orientations helped these members maintain a sense of connection with meaningful operational tasks and the bigger picture.
  2. An independent opportunity to study and orient to the basics of a platform stokes confidence to ask better questions when interacting with experienced teammates.

I used quotes around “course” above because I think the classification of solutions described above are not courses at all. This might seem like a petty labeling nitpick.  But it seems important, if not critical, to reconcile the performance solutions lexicon around fit, purpose, and characteristics so we don’t pollute one stream with the meaning provided by another. This intersects with solution selection processes as well.

When you think about fit, purpose, and characteristics of a course what attributes come to mind? Do any of these characteristics resonate? Courses…

  • Are designed for a resulting net positive change in capability or behavior for tasks that must be committed to memory (measured in outcomes, not quiz scores)
  • Provide examples and demonstrations that show, illustrate, and elaborate concepts, procedures, and processes.
  • Provide opportunities for authentic and contextual practice and feedback.
  • Frame structures of clear progression toward skill goals.

How many packages have you seen that take on the course label don’t carry all (or any) of these features? Many? Most? Almost all of the compliance “courses” I’ve had the “pleasure” to participate in didn’t offer any of these characteristics. And this isn’t isolated to online offerings. Many physical classrooms suffer from the same deficiencies.

These qualities aren’t elements I’ve made up. Dr. Dave Merrill, Dr. Ruth Colvin Clark and countless others express these qualities as the core principles of effective instruction. Reams of books and studies have been published to support these characteristics as central to success.

It’s not uncommon for packaged content to take on the label of a course. To me, a mislabeling can seriously dilute the value and distract from the deliberate services provided by a course of instruction. A lexical hierarchy of solutions that includes courses and other classifications of solution could work together to better define what solutions are, what they do, and how they are selected to match with problem contexts. Heck, this common lexicon could already exist. If you know of a systems lexicon for performance solutions, I’d love to hear about it.

I suggest adding a category of solution at the level of awareness called orientation. Orientations have value but they are not courses. Orientations provide foundations for more complex learning structures and support tools. Solutions that aren’t courses have value to the big picture. Orientations, for example, can provide a nudge or boost for task performance when added to performance support. Orientations can also fit well when intentionally situated within course structures. Orientations don’t need to be fancy or expensive to get the job done.

We should try to avoid dressing solutions up as objects that they’re not and confusing ourselves as well as the people we provide services for.

I’ll be thinking and writing about this more in the future and welcome feedback and advice. In the end, maybe a more clearly and consistently defined set of characteristic definitions and labels for performance solutions wouldn’t be a bad thing.

What do you think?

Life should be defined by what we are for, not what we are against.


Life should be defined by what we are for, not what we are against.  ~ Don Robison, CDR, USCG Retired

A good friend of mine just published a book of stories. Each of these stories hold a special meaning and a little bit of “inside joke” to someone who has spent some time on the seas. You might wonder, what does this have to do with the theme of the site? Two things. First, I think Don is a great storyteller and you’ll actually enjoy this. Second, he’s finishing his post-graduate stuff in Instructional Design at Old Dominion University and is doing some really cool research into the connections between aesthetic and motivation. So, there’s definitely a connection.

He’s made a sample of three stories available for download. You can also buy the book here (Amazon) or here (Create Space).  My lovely wife made the cover and all of the illustrations for the book.

This is the first story from the book and the sample. I love the conclusion and the motto that follows the end of the story. Not only is it an expression of a perfect motto. It also exposes the heart and character of the author – a man of passion, compassion, and the just right perspective.

In these times where we regularly see one citizen pitted against another in a verbal battle of ideals, I think this message could bring a little bit of sanity to the debate.

I hope you enjoy these as much as I did.


I suspected the interview would be tough from the beginning. LT Sellar grinned at me, cast an amused glance at his two colleagues, and asked the first question: “Now, Mr. Robison, why do you want to be a Coast Guard officer besides…” and here he looked down at my application essay and read from it; “…feeling the wind in your hair and salt spray on your face?” Continue reading Life should be defined by what we are for, not what we are against.

Narrowing the Solution Field – Part 2

In the first part of this series, I introduced an approach for making selection of interventions a little less arbitrary. In this installation, I’ll refine process funnel a bit to round out and clarify definitions of the output levels.

When using processes like this, I think it’s important to keep in mind:

  1. There is no magical process or algorithm to make design decisions automatically. Because of all the factors at play, this stuff is work. Doing it right takes rigor and  discipline. Anything short of a disciplined approach is rolling the dice.
  2. Design is a complex business of weighed options and trade-offs. The answers are almost never easy and there is rarely only one answer.
  3. This process of refinement isn’t linear. It IS iterative and involves making and validating assumptions.
  4. Design is a process of illumination. You can’t solve a problem that you can’t see.

That said, models and tools for thinking through problems can be really helpful to shake things loose and help you to show your work.

Continue reading Narrowing the Solution Field – Part 2