Measuring Learning Impact
For a Learning and Development professional to truly become a Trusted Business Advisor, it is critical to demonstrate business impact through learning interventions.
I will now talk about a model of measuring learning impact that cuts across learning formats-the Kirkpatrick's model.
The Kirkpatrick model has 4 levels of evaluating learning impact: -
-Level 1-Reaction/Satisfaction Levels
-Level 2-Change in the level of Knowledge
-Level 3-Change in Behaviour
-Level 4-Results/Business Impact
Let me elaborate on each level a little further.
·Level 1-Reaction/Satisfaction Levels
This level involves assessing the immediate reaction of the learners after the training/the e-learning course. Feedback is captured from learners about how engaging and useful they found the content and the methodology. Inputs may also be captured about the learning platform (for an e-learning) or the trainer in case of an Instructor Led Session. However, this is just the first level of measurement and does not give us a complete insight into the effectiveness of the training.
·Level 2-Change in the level of Knowledge
This level measures the knowledge level after completing an e-learning course/a classroom training. This is done through an assessment/review either through the course of the lesson or at the end of it. The assessments maybe in the form of a quiz or a case study. In some cases, there is also a pre and post test done to compare the knowledge level before and after the training
·Level 3-Change in Behaviour
This level measures if the training(e-learning/classroom) leads to a change in behaviour. This can be assessed through a 360 degree (Self-Manager-Peer-Client) assessment on competencies. A pre and post training assessment over a period of time can measure a shift in proficiency and also how the skills learnt are being applied on the job.
·Level 4-Results/Business Impact
This is the ultimate level of impact every Learning and Development Professional wants to achieve. However, neither is this easy to accomplish and nor is it easy to measure. Over here, the new skills/behaviour acquired through training lead to a change in performance/improvement in metrics and this results in a dollar impact. In fact, the ROI of training can be calculated at this level by considering how much was spent on the training vis a vis the value of the impact delivered.
Let's deep dive into this with an example.
Business Need:-There is a Business Process Outsourcing firm who provides Customer Service Support through voice and non-voice processes. Essentially, these are business processes that have been outsourced by their clients. The quality of customer service is measured by Customer Satisfaction(CSAT scores). The Business wants to improve these scores from 3.9/5 to 4.5/5. They approach the Learning and Development team for a solution.
Solution Design, Development and Deployment:- The Learning and Development team after a ‘Needs assessment’ drills down that there are keys competencies that can be developed in both the Advisors who actually take customer calls and their Team Leads who monitor their calls and give them performance coaching. For the purpose of this article will only focus on the upskilling of the Team Leads. The Learning and Development team designs a blended learning solution to address the issue- Classroom training on Performance Coaching, followed by eLearning course on People Development and Coaching, a 30-day Action Learning Plan and Group Learning Session to discuss learnings and progress.
Measuring Impact:- Now how can learning impact be measured with the Kirkpatrick model in this case?
-L1- The first level would be to check students’ reaction/level of satisfaction for each initiative-classroom, e-learning, etc.
-L2- The second step would be to assess knowledge. This could be done with a pre and post test to check for knowledge on People Development and Coaching using scenarios, a quiz, etc.
-L3- The third would be to observe coaching sessions of Team Leads and give them feedback against specified competencies and criteria. Again, having a pre-training observation and 360-degree assessment helps in comparing if there has been a movement in key competencies
-L4-The fourth one is business impact and that can be evaluated by seeing if there is an improvement in CSAT scores over a period of 3 months. Of course, if the Advisors are also being trained then that also needs to be considered as a variable. It is also possible to measure the ROI in Level 4. It will be covered in a subsequent blog. Watch this space for more!
So, the next time you’re designing a Learning Solution, keep in mind how you can demonstrate the value of learning and convey to your stakeholders that the buck stops with you!
