Sunday, June 07, 2009

The Agile Elearning Design Manual: Problems with existing approaches

In recent months I've met over 50 elearning practitioners and evaluated at least 6 elearning firms. My inspiration for this series of posts has been what I've learnt about their style of learning design and how approaches we've tried internally at ThoughtWorks compare as a stark contrast. In today's post I want to outline the current state of elearning content development in the industry. Understanding the pains of this current state will help us explore the virtues of Agile Thinking in future posts. In order to describe my thinking better, I've gone ahead an adopted Dan Roam's Six Ways of Seeing from The Back of the Napkin. The topics I want to outline today are:
  • The composition of the teams that deliver the learning modules.
  • The standard process followed by many internal teams and design firms.
  • The need for a change
So without further ado, lets get going.

The people behind the scenes on a typical elearning module

On most elearning projects (mind you a project could mean multiple modules and a rather complicated delivery), the key roles I've observed are the ones I mention in the picture above. These roles are:
  • The Project Manager
  • The Instructional Designer
  • The Builder
  • The Tester
  • and the Freelance/ Multi-departmental Graphic Artist

Those roles are pretty representative of what happens on a software development team as well, except when you consider the following problems in their implementation:
  • This is not really a team in the first place -- people depicted in this group generally work on multiple things one of which is the client's elearning project. So, they aren't really dedicated to ONE purpose. They're doing their jobs in a factory atmosphere where every machine has a transactional purpose.
  • The only people in contact with the customer are the Instructional Designer and the Project Manager
  • The only people that understand the customer's problem and the subject matter of learning are the Instructional Designers
  • The builder delivers the most value in the process of elearning development, but is the least respected. Her role is reduced to executing instructions from the Instructional Designers
  • The Graphic Artist plays a very transactional role in the project. He works with descriptions provided by the Instructional Designer, without any considerations of whether the final output fits the image of the client or the learning material in the module.


If you look a little deeper into the skills that this team has, you will notice that there is very little overlap between the skills of various people in the team. While yes, there's a small overlap between Builders and Testers, the skills of Graphic Design and Instructional Writing seem to be completely detached from others. Which is to say that the Graphic Designer has no idea how to design an elearning module, the Instructional Designer has no idea what goes into building an online course (even with rapid tools) and the Builder can't even do basic instructional design! I've written earlier about the need for Learning Generalists. The risk on these teams is extremely high, considering that they're groups of extremely uni-dimensional individuals, working with an un-shared purpose.

More about this when we come to the process.

The current process - a rather rigid one


The above picture depicts the method of content creation at most internal teams and design firms. The approach is pretty simple and will remind a lot of us of a modified "Waterfall" model of development. Here's an explanation of the pain points I've indicated:
  • The process relies very heavily on the Instructional Designer to write on an MS Word Document (called a script) the details of:
    • What goes into the module
    • What text appears on screen
    • The interactions that'll get used
    • The pictures that'll get used.
    • The choice of media and activities

  • Outwardly there's nothing wrong with this, until you consider that:
    • material is evolved from exisiting Powerpoint presentations than from a business discussion with the customer. Nothing wrong with Powerpoints except that they assume the SME's familiarity with the topic; are written in non-ubiquitous language; could be meant for a completely different audience and are generally poor instructional documents
    • the format used to determine if the customer/SME is happy with what they will SEE is a document that has NO VISUALS. The customer doesn't even see a prototype to know the impact of saying OK or not OK.
    • the approach relies on getting all of the training module nailed down up front which is practically impossible until the customer see's something in action - a problem commonly experienced on Waterfall projects.
  • As you will notice, there's no safety net to account for "incorrect" analysis since the Builders, Testers and Graphic Artists have no idea of what the subject matter is.
  • As a result if what the customer sees during the showcase is not what he thought he would get, its a whole new cycle of amendments which could go on forever.
Now if you look at the time taken for development and consider a 3 week development period for a training module, the "Script" takes the longest time to create. The actual value adding process of building takes very little time. In the ideal world, all the "Scripting" would be perfect and everyone would be happy. In the real world, people make mistakes and the Script doesn't necessarily reflect the customer's vision and this often leads to an endless cycle of amendments at the end of development which can span multiples of the actual development time required.

The process that you see above, doesn't allow for fast feedback. It doesn't allow for builders to collaboratively develop the module with Instructional Designers. Instead the Builders are expected to follow the script. Many designers believe they would be doing a bad job, if the builder had to come back to them with questions about their script. I remember a team where the builder and the developer were sitting at the same table and the Instructional designer was into her 6th day scripting. I asked her, what if the builder has questions about her script. Her face almost turned pale in horror and she said to me, "Well if I've done my job well and written a good script, there should be no reason for the builder to come back to me for anything!" Its something to think about. 10 days of documentation for 4 days of development work to be passed on to a colleague that sits at the same table! What ever happened to good old conversations? The customer doesn't see working elearning until very late. As a consequence, the showcase is a surprise -- good or bad, depending on the end product!

Time vs Cost vs Effectiveness


The question to ask ourselves however, is why do training organizations and design firms choose such a heavyweight method? The answer is simple -- because the previous method didn't work! The problem is that the previous method was a haphazard, non-scientific, cowboy practice of "Hey lets get going with development!", long before identifying the training need, breaking it down into actions and then learning activities. The "lets develop" attitude was obviously marred by the absence of any planning or visioning, led to different strands of unexpected work emerging, context switches, involved but confused customers, ineffective learning, a huge time to market and a high cost.

So, what did the experts decide? To compensate for not planning, they overplanned. To compensate for not analysing, they over-analysed. To compensate for customers changing their mind all the time, they added levels of documentation for different people to say "I do!". And to stop builders from going all nuts about a new module, they regimented them into following instructions from a Word Document. Sounds like a typical response and I don't fault anyone for this. To be fair, the current solution definitely works well compared to the problem state it tried to address. Is it ideal for the customers though? Does it help create a shared sense of purpose in the team? Does it respect the ability of builders and graphic designers? Does it foster communication within the team or reduce the risk of misrepresentation of customer requirements or plain miscommunication? Does it give the team the ability to fail fast, learn from its mistakes and delight the customer? The answer I'm afraid, is "NO!".

So what's the approach to follow then? That's what the Agile approach is all about. In the next post, I will outline some of the basic principles of Agile as applied in Software Development and how they apply to this particular context. I will also pick the most important practices and substitute them with usable practices for elearning development. If this post interested you, stay tuned for next Monday's article, and I hope you'll find that useful too. Till then, think about your approaches towards online learning and feel free to drop your comments if you'd like. I'll be really happy to hear from you.

2 comments:

Cathy Moore said...

These are great graphics and a spot-on description of what I've seen in elearning development. You've captured the common waterfall approach perfectly.

At one point, you say that the builder delivers the most value. I'd say that the person who identifies the actual performance problem and how it could be solved online provides the best value.

To me, elearning development is too focused on content. "I'd like a course on X," the client says, and the elearning team says, "OK, send us the content and we'll make it look good."

There's often very little discussion of *why* the client wants the course. What problem are they trying to solve? Why do they believe that a "course" (usually an information dump) will solve it?

So to me the most useful people are the ones who clearly identify the problem and identify what types of online activities (if any!) could help solve it. Maybe those people are the project manager, ID, and builder working together with the client, and ideally the discussion includes quick prototypes.

Of course, this is the analysis part of the process and increasingly seems to be cut. Maybe it's cut because some designers spend too much time alone, creating super-detailed matrices and flowcharts when a quick sketch would have been enough. But mostly I think we skimp on analysis because it forces us to examine our culture and procedures, something that can reveal messy truths that require a more complex solution. So we put on blinders and focus on the much easier task of delivering content.

Sorry for the rant. I'm looking forward to seeing your agile workflow!

Sumeet Moghe said...

Hi Cathy,
Thanks for your comments. You're not ranting at all. I completely agree with the points you make. When I said that the builder delivers the most value, I meant it quite literally (the delivery bit). So yes, its a team effort to identify the performance problem and define the solution. Its finally upto someone to actually "deliver" that value to the customer and I meant that the person who does that is the builder.

Hope that clarifies what I meant!

In general I'm disappointed with the customers lack of involvement at the build stage and I see that while sometimes its just the customer acting busy, at most times, its the team not involving the customer to provide fast and furious feedback.

BTW, I'm a great follower of your action mapping approach. "My" Agile approach really rests on your action maps to be successful. But more on that later.

Related Posts with Thumbnails