Always fresh new new things!

ROI of learning in Organizations :
business context analysis

without comments

One of the big nagging issues till date in organizational learning, including eLearning is ROI. In a recent interview with Karl Kapp for, Cisco’s Tom Kelly mentioned –

No one ever calculated the ROI on fax machines or email or anything like that. They seem to think the learning industry should focus more on benefits like improved quality or responsiveness and less on some of the harder business methods. In fact that’s what we did here for the first three or four years, we said the same thing. We said no one did ROI on email or voicemail because it is better to be connected than not connected. We asked, so why are you having this problem with learning? Isn’t the only reason to communicate to either teach or learn? And mostly the answer is “yes” even if you’re yelling. So then email, voicemail are two tools for e Learning. And if that’s true then we should say that all this stuff, even if its video based, is content that drives both the formal and the informal parts of learning. And yes, in some cases, it is expensive, but ignorance costs a whole lot more.

I would personally agree that we shouldn’t have to do ROI on things that make people smarter and better at their job, and more connected to their companies, their goals, and their missions but I don’t write the rules.

Jay cross firmly believes that (and explained in detail in his e Book- Metrics)

  • The metrics of training should be business metrics
  • ROI is in the eye of the beholder
  • Intangibles and services make old accounting models obsolete
  • If you don’t have your sponsor’s support, you are toast

Chip Cleary recently wrote a great article at CLO magazine on this subject – Measuring Business Results Using Business Impact Analysis.
First he stated the problem -

Most learning executives find it frustratingly hard to determine how much a learning event benefits their organization. After all, the direct results sit inside employees’ heads, where you can’t see them. The business benefits then unfold over time, in the midst of many other forces that also influence end results. Because of this indirectness, the business impact of training is rarely evaluated.


Why is evaluation (in business context) so difficult?

  • The training is only one of many factors that drive results
  • It is not possible to run controlled studies
  • Existing metrics do not adequately measure performance

How can we do better?
The approach underlying BIA is to open up the black box by creating a simple causal analysis of how a given training course is expected to create value. With this analysis, we can then create an efficient method of quantifying the impact of a piece of training and measure its economic benefit.
here are many factors that cause peak performers to achieve better results than average performers. It is wasteful to try to target all of them with training. Some do not much matter to the business. Others are not amenable to training. So, instead, we focus on a set of the “critical mistakes” that separate peak from typical performance and that matter the most to the business.
To maximize and measure the benefit produced by training:

  • Identify a set of potential critical mistakes.
  • Estimate their cost.
  • Provide a solution that eliminates the most costly mistakes.
  • Measure the resulting reduction in the frequency of those mistakes.

The process reminds me of a post by Dave Pollard – The Cost Of Not Knowing. It’s like a practical and technical application of the business context analysis of the business context analysis of learning in organisations.

Written by anol

June 20th, 2005 at 5:49 am

Leave a Reply