Learning and evaluating-Success Case Method’ of Brinkerhoff

Learning and evaluating: ‘Success Case Method’ of Brinkerhoff

Learning never stops! The new fad is outcome based learning! In times of recession and bad economy, training is luxury and anti-national in sentiments and anti-economy when jobs are vanishing and we have a threat to the whole of new generation not being able to experience the work-place? Ever thought over, the US and European nations have suffered job losses to existing work force to the levels of 40% and some countries no job creation for un-skilled or un-experienced job seekers. In times of uncertain economy focus shifts from building long-term strategic advantages to short term process efficiencies, lean and mean organisations! Are we getting back to the 70’s when most companies were managed by bean counters?

 Well, till the time, you have not decided to close down and still believe, market trends are cyclical and after every fall, there is a rise. The rise of the rise and the fall of the fall will still remain unpredictable and inevitable.

With every recession, we learn a few lessons to be quickly forgotten when we see the upswing! The problem is not recession or volatility or unpredictability or things and events beyond our control or comprehension, it is not willing to learn and not being able to build own survival and growth systems that can survive turbulence in economy!

Learning from each event (stories) of success and failure, identifying the factors that led to the results and testing and validating the impact of such factors against corroborating facts and evidences before we analyse, present and build our own model to see the success you visualize!

 Alternative’s to Kirkpatrick: Brinkerhoff’s Success Case Method

When it comes to learning solutions leveraged in organizations, Kirkpatrick’s evaluation model is often looked to as the assessment method, despite its focus on training events rather than on learning processes.  In this post, I’ll take a look at a more flexible model, Brinkerhoff’s Success Case Method.

Brinkerhoff: A focus on systems

Brinkerhoff (2005) squarely addresses one of the critiques of Kirkpatrick’s model: “Performance results can’t be achieved by training alone; therefore training should not be the object of evaluation”. According to Brinkerhoff, this is like saying that the success of a marriage depends on the quality of the wedding ceremony. Multiple variables contribute to the impact of a learning opportunity (be it training, performance support, or another solution) and multiple stakeholders own these results—from senior executives to managers to HR professionals/L&D team members to employees.

Given that the factors that influence performance operate at the level of systems, Brinkerhoff urges that the proper focus of evaluation is also at the system level. Evaluation should address the following questions:

·         How well is an organization using learning to improve performance? (STRATEGY/STRUCTURE/SYSTEMS/SHARED VALUES/STAFF?)

·         What organizational processes/resources are in place to support performance improvement? What needs to be improved? (STRUCTURE/SYSTEMS/STAFF?)

·         What organizational barriers stand in the way of performance improvement? (SYSTEMS/STRUCTURE/STAFF?)

The Success Case Method: Evidence- and narrative-based

The Success Case Method (SCM) “combines the ancient craft of storytelling with more current evaluation approaches of naturalistic inquiry and case study”.

 Essentially, when you apply SCM, you ask:

·         What groups/individuals have been successful in applying a learning opportunity to achieve a business result? Why have they been successful?

·         What groups have been unsuccessful? Why have they been unsuccessful?

There are a number of steps involved in SCM.

·         Develop an impact model: Identify the goals of the learning opportunity and determine how these goals are connected to business needs. The impact model defines what success should look like.

·         Survey participants to identify best cases and worst cases. (For example, a survey question might ask: How have you applied what you learned to achieve a business result?)

·         Obtain corroborating evidence that would “stand up in court” (e.g., using interviews, document reviews or other methods).

·         Analyze the data.

·         Communicate findings: Share what successes have occurred and what organizational resources have supported these successes. As important, share examples of non-successes. What barriers kept people from applying what they learned?

A better model?

There are some appealing aspects to Brinkerhoff’s SCM.

·         It’s do-able: You do need to collect data but you can do this in a cost-effective manner.

·         Because SCM utilizes purposeful, not random sampling, it collects information that would be missed if you only looked at averages or central tendencies.

·         SCM allows for the discovery of emergent success factors: Although you do create an impact model in advance, your surveys and interviews may uncover additional, unexpected business results.

·         The focus is on systems and leveraging learning resources into continuously improved performance.

·         Outputs include relatable stories you can share.

Although there are the typical caveats associated with surveys (potential biases and halo effects), these can be mitigated through the step of collecting corroborating evidence. I think that the focus on successes and case stories is extremely attractive because it provides a way to involve stakeholders in a meaningful dialog about continued process improvement.

Kirkpatrick 4 Levels

The four levels of Kirkpatrick's evaluation model essentially measure:

1.       Reaction of trainee - what they thought and felt about the training

2.       Learning - the resulting increase in knowledge or capability

3.       Behaviour - extent of behaviour and capability improvement and Implementation/application

4.       Results - the effects on the business or environment resulting from the trainee's performance

All these levels are recommended for full and meaningful evaluation of learning in organizations.

Jack Phillips' Five Level ROI Model

Building upon the Kirkpatrick model, Jack Phillips added the fifth level the Return On Investment (ROI) produced by a training course using the financial formula:

ROI (%) = (Net Program Benefits/Program Costs) x 100

Reference

Brinkerhoff, R. O. (2005).  The Success Case Method: A strategic evaluation approach to increasing the value and effect of training.