Learning Evaluation Network

When is a social network not a social network?  This issue was raised in the testing phase of the Learning Evaluation Network (LEN), just launched on the Ning platform.

Ning describes all the networks it supports as “social networks” but this conveys all sorts of assumptions, largely based on the most widely-recognised Facebook model. However, to take just one example, the Learning Evaluation Network is closed to the public, and open only to paying subscribers.

And so the founders of LEN (of whom I am one) don’t use the term “social network” – instead it is based on Etienne Wenger’s theory of Communities of Practice, and on the more recent concept of the personal learning network (derived in part from the idea of personal learning environments). In fact the term used by LEN is a “shared learning network”, which encapsulates some of all of the above – let’s see if that one catches on!

Evaluation of learning and development is pretty niche. Some would say that about L&D, and concern about its impact, and relationship with performance improvement, is even more niche, so evaluation of L&D bounds on the extremes of esoterica…

Except that it is rather important. Someone told me recently I should forget about trying to sell clients what they need and focus instead on what they want – at the risk of perpetuating a form of condescending paternalism, I’m going to carry on ignoring that advice. L&D practitioners need the learning evaluation network.

All of which amounts to some philosophical musings about what is really a very concrete proposition – members who subscribe to LEN connect to experts and practitioners in learning evaluation from all over the world, get free information, advice and resources, and the scope to accomplish much more. I think it’s a great concept, and I commend it to you.

Followers of this blog can join the Learning Evaluation Network at a discount of 10% from the published annual subscription by quoting promotional code TL420 when they register at http://www.airthrey.com/network/
 

Hot off the press!

The Learnforever Book is now published.

You can find it on amazon, and further links will follow.

Any follower of this blog, or commenter, may request a copy directly from the author - I'll be happy to post a personally inscribed and signed copy with no added charge for postage.

I'll blog again with any reviews or other feedback.

And of course, comments are (always) welcome.

The Learnforever Book

I’m pleased to announce that my fifth book will be published by Paragon in time for Xmas.

The Learnforever Book is a compendium of selected posts from this blog, compiled by theme, and featuring a new introduction, editorial comments and bibliography.  If you enjoy this blog, you should like the book too.

The book’s themes include e-learning and blended learning, leadership and management development, and evaluation of learning and development, among others.  The foreword is by Nigel Paine.

All of the posts included are from before this post, and nothing that follows will be in the book.

More information once the book is available in December.

Explaining Total Value Add

One or two people who have read the e-book I published last year with Alasdair Rutherford – Total Value Add, a new approach to evaluating learning and development – have asked for more information about how Total Value Add works.

Total Value Add includes two related ideas.  The first is that organisations derive a lot of value from learning and development, and need to capture more of it, as cost-effectively as possible.  The second is that different evaluation methods and tools lend themselves to different kinds of capture, and so a range of tools and techniques need to be applied in different situations.

Let’s take the first idea first.  Opportunities for development are increasingly seen as part of the benefits package of working for an organisation – at least by the more enlightened and ambitious employees – and so it makes sense to check that learners enjoy the learning opportunities.  Reaction sheets do this well, but do you really need to check the reactions of every employee to every learning experience?  Yet that’s what most organisations actually do, and it’s not cost-effective – judicious sampling would serve the same end, for a fraction of the effort expended.

That’s one example.  Another is that learning interventions do not necessarily provide the sole means to effect the kinds of behaviour change organisations need – but they may provide a spur.  Acting in concert with line managers dedicated to managing performance, learning interventions can help bring about improvement, perhaps by training line managers to be better coaches, or by providing performance support to employees on-the-job.  This sort of intervention requires analysis of the performance desired, a measure of the current level of performance, and means to record progress in improving performance.

Turning to the second idea, our first example should direct you away from universal distribution of happy sheets.  These are often poorly designed, usually poorly distributed, and rarely properly analysed or acted upon.  A better use of resources, and a better way of capturing value, would be to select representative samples, and interrogate them in more detail, perhaps using interviews rather than surveys.  The barriers here are that L&D practitioners lack survey design skills, question writing skills, and typically don’t know how to select a truly representative sample.

The second example calls for an evaluation method that tracks employee performance, and gauges the impact of L&D in improving that performance.  Possible methods include Business Impact Modelling, Dave Basarab’s Predictive Evaluation, or Ed Holton’s sixteen learning transfer indicators.  Most L&D practitioners – in the UK at least – will have scarcely heard of these methods, far less have the skills to implement them.  Yet neither of the more commonly recognised evaluation models – Kirkpatrick or ROI – features the means to tackle this sort of issue.

Part of the problem is that many L&D practitioners are not even conscious that they don’t know enough. We need to raise awareness of what’s involved in evaluating L&D, spread knowledge of a wider range of tools, and clarify what extra skills are needed.  More information can be found at www.airthrey.com

 

E-learning: better than face-to-face?

blended learning > e-learning > face-to-face learning!

I recently discovered a three year old research report from the Department of Education in the United States.  A meta-analysis, encompassing studies covering a 12 year period from 1996 to 2008, it compares the effectiveness of e-learning and blended learning to face-to-face instruction.

Advocates of e-learning have argued for some time that it is at least as good as face-to-face methods, but this research appears to provide substantial supporting evidence.  In fact, the research shows blended learning (a mix of online and face-to-face) works best, purely online learning is second best, and face-to-face the least effective option of the three.

Barbara Means, an educational psychologist and the report’s lead author, has said “the study’s major significance lies in demonstrating that online learning today is not just better than nothing - it actually tends to be better than conventional instruction”.

Before we get too carried away, we should note that the scope of this research was schools, where the objective was to evaluate the effectiveness of learning.  In the world of work, we need also to take account of the transfer of learning to the workplace, and subsequent impacts on performance and business results.  But this still amounts to an emphatic endorsement of the use of digital technology in learning.

 

Report here.

Don’t waste your money on training!

Like most L&D practitioners, I regard learning as A Good Thing, something to be encouraged at all times, and a good habit to get into.  Learning, in its broadest sense, helps people get better at everything they do, and it helps organisations succeed.

But that doesn’t mean all learning is always good for organisations.  This is basic economics.  Organisations have scarce resources, and need to make difficult decisions about what to spend their money on – what activities will add most value?

Ignoring this basic economic principle is the reason why expenditure on training is often wasted.  I’m reminded of the marketing manager who knows half of his advertising budget is wasted – he just doesn’t know which half.  Organisations often undertake training as little more than a matter of faith – they know it ought to be good for them, but they’re not sure exactly why, or to what extent.  This means the organisation can’t tell whether they’re best spending money on training, or better spending it on something else that might impact more on performance and results.

This is one of the reasons why Dr Alasdair Rutherford and I set up Airthrey, the learning evaluation solutions business, two years ago.  We wanted to help organisations work out whether training is working for them, and what parts of their training budget might be wasted.  Thus the message not to waste money on training, but to find ways to measure (or otherwise accurately judge) the contribution training makes, and so target scarce resources for training more effectively.

Airthrey’s flagship programme, Learning Evaluation Action Development (LEAD), is a means for organisations to work this out, using the best techniques available, with support from their peers, with the best resources on the subject, and with tuition from Alasdair and me.  It’s not another training course, but a form of supported consultancy, by the participating HR/L&D professionals themselves.  It works really well, and I commend it to you.

The Seven Pillars of the Corporate University

Thanks to Business Annual for re-publishing my 2009 article, The Seven Pillars of the Corporate University, in July 2013, as part of a feature on corporate universities.

You can read the article here: http://businessannual.net/