Evaluating Safety Training Effectiveness (Based on ANSI Z490.1, Section 6)

Hello.

This post focuses on best practices for evaluating environmental, health, and safety (EHS) training. That means a couple things: evaluating how well your employees learned from their EHS training and, of course, evaluating the EHS training itself. Plus, it means using that evaluation information during continuous improvement efforts.

This article, like others in the series, is based on some of the guidelines in ANSI Z490.1, the standard that lists "Criteria for Accepted Practices in Safety, Health, and Environmental Training. In particular, this article is based on Section 6 of the standard.

If you want to download our free 42-page Guide to Effective EHS Training, click that link you just whizzed past or scroll down to the bottom of this article and click the download button.

Otherwise, let's get learning about evaluating EHS training.

Evaluating Environmental, Health, and Safety (EHS) Training

As you probably guessed, the purpose of evaluating EHS training is to see if it the training was effective. Are your employees learning from the training? Does the training lead to the desired change in their behaviors on the job? Have your EHS metrics improved? Does one or more individual employee need additional help after the training? Do you need to modify the EHS training?

We'll look at three aspects of this below:

• Criteria for evaluating EHS training
• Ways to evaluate EHS training
• Continuous improvement efforts

Criteria for Evaluating EHS Training

The standard provides some general criteria for evaluating EHS training, including the following.

Begin Planning for Evaluation Back During Development

Before you deliver the training to employees, you've got to design it and develop it. We explained this earlier in our article How to Develop Effective EHS Training.

Even that early, you should be thinking about your post-training evaluation of the learners and evaluation of the training materials themselves. Outcomes that can be evaluated include (but are not limited to):

• Did the employee acquire the desired knowledge, skill, ability, and/or attitude after training?
• Did the trainer who led the training session effectively help employees acquire the desired knowledge, skills, abilities, and/or attitudes during the training?
• Are the employees satisfied with the training experience? What were their reactions?
• Is the EHS training contributing to organizational goals?

The Post-Training Employee Evaluation: Can The Employees Satisfy/Meet the Learning Objectives?

After the training is over, one thing to evaluate is whether or not each employee in the training successfully satisfies each learning objective of the training.

We explained what learning objectives are in our article on how to develop EHS training, but you can also click here or here to read more about learning objectives.

Identification of Employees Being Assessed/Evaluated

You're going to need to correctly identify every employee who's being tested. Be sure to create a mechanism for this so you'll know who's who and can confirm each employee is who he/she says he/she is and of correctly matching the employee taking the test with his or her grades/scores/test answers.

Pre-Testing and the Possibility of "Testing Out"

During EHS training design and development, give thought to the possibility of allowing employees to take some form of pre-test and "test out" of the training if they demonstrate that they already have the desired knowledge, skills, and/or attitudes the training is intended to teach.

Typically, if you DO allow for the possibility of testing out, the employee would test out by successfully completing the assessment that's provided to employees after the training.

Be sure to check regulatory guidelines to make sure testing-out is acceptable.

Criteria for Successful Completion

During the design and development phase, you'll also have to determine the standard an employee must meet in order to complete the post-training evaluation successfully.

The standard for successful completion that you choose will vary on a number of things, including the type of training evaluation you use. However, some possibilities may include:

• If the employee has to take a test, successful completion may mean answering a specific number of questions correctly or answering a specific percentage of questions correctly
• If the employee has to demonstrate that he or she can perform a task correctly, successful completion may require performing all steps in a specific order and manner. You'll want to specify the correct steps of the procedure in advance
• If the employee must complete a project or some sort, successful completion may require that the employee include all key elements in the project

Evaluation Must Comply With All Industry Standards and Regulations

Also remember that the evaluation must comply with all industry standards and regulations. For example, regulations often specify a minimum acceptable level of training (although it's always OK to exceed those requirements).

Reliability and Validity

When you create an assessment to evaluate if the employees can meet the learning objectives, it's important to know the evaluation must be both reliable and valid. But what does that mean? Here's what the standard says:

• Reliable: Gives consistent results over time
• Valid measure: Reflects the knowledge, skills, abilities, or attitudes specified in the learning objective

Click to read more about reliability and validity in testing.

Providing Results to Employees

Another thing to think of in advance is to create a manner in each employee who took the training will be provided with the results of the evaluation (whether it is the scored test or the observed skill demonstration).

Trainees can use this as part of the feedback to help guide them as they look for additional information or practice and also to help them create a plan for future training.

Periodic Re-Evaluation of Employees

The EHS training program should include some form of periodic re-evaluation of the trainee's ability to satisfy the learning objectives. This helps to evaluate the training as a whole as well as the need for retraining or refresher training.

Training Evaluation Approaches

Now that we've covered some general criteria for environmental, health, and safety training evaluation, let's look more closely at some approaches or methods for evaluating training.

Section 6 of the standard suggests a method that's based on what is commonly known as Kirkpatrick's Four Levels of Evaluation.

In this method, training is evaluated at as many as four different levels to determine if it's effective. Those four levels are:

• Employee reaction
• Employee learning
• On-the-job behaviors after training
• Business results

Next, let's turn our attention to the type of evaluation to use. The evaluation should include one, and may include all, of the following approaches:

Level 1: Employee Reaction

After the training is complete, have the employees give their opinion of the training. This can be delivered as a survey. Be sure to include space for written comments, too.

Survey questions can cover topics such as the trainer's presentation skills, the suitability of the training environment, the pace and difficulty of the training, and the usefulness of the training material.

These surveys can help the trainees assess their own level of learning. In addition, they can be used by trainers and training providers to assess and improve the training materials and training delivery.

A good source of information about creating useful level-1 survey sheets, often called "smile sheets," is the book Performance-Focused Smile Sheets: A Radical Rethinking of a Dangerous Art Form by Dr. Will Thalheimer. You can see our article on Thalheimer's "Smile Sheets" book here.

Level 2: Employee learning

This is an assessment or test, during or immediately after training, to determine if the employees can satisfy the learning objectives.

This assessment may take several different forms, including:

• A written or online test
• A performance assessment in which the worker demonstrates a skill while being observed; this can happen in a simulated work environment or the real work environment
• An oral examination
• The completion of an assigned project

You may find these articles helpful while planning your own post-training assessment:

• Testing employees
• Writing multiple-choice questions

Level 3: Observation of On-the-Job Performance

Because training is intended to change how people perform on the job, it's important to observe the workers in the field after the training to see if they can demonstrate the skills/abilities/knowledge/attitude from the training while actually working.

You might want to consider doing this both before and after training to get a better sense of any improvement the training may have caused.

On-the-job observation can come from direct observation, but also from information collected by supervisors, coworkers, or customers, or evem from indirect measures in product records or safety reports.

If observation detects a gap between the actual performance and the desired performance, try to find the cause for that gap. In some cases, this can be tracked back to problems with the EHS training and/or training delivery, but in other cases, other factors that are un-related to training may exist. This article on identifying and closing skill gaps explains this in more detail and provides a method for investigating this issue.

Level 4: Effect of Training on Organization/Business Goals

Finally, you may try to evaluate the effect that the training had on goals of the organization or business.

This can include tracking things like:

• An increase in observed safe behaviors
• An increase in preventive measures and controls implemented
• A reduction in near misses, injuries, and/or illnesses
• A reduction in insurance claims
• Improved environment compliance
• A return on ROI of training

You may have noticed that to do this, you'll have to select particular key performance indicators (KPIs) of the business that you'll track over time. The following articles provide more information on this topic:

• Business Goals, KPIs, and Job Training
• How to Measure the Impact of Training on Business Goals and KPIs
• Level 4 Training Evaluations
• Isolating the Effects of Your Training Program
• EHS Leading Indicators
• Tracking Training-Related EHS Leading Indicators

Continuous Improvement of the EHS Training Program

As we mentioned earlier, the evaluation phase of EHS training has two primary purposes:

• Evaluating each employee to see if he or she can satisfy the learning objectives of the training and has acquired the desired knowledge, skills, abilities, and/or attitudes and is applying them on the job
• Evaluating the EHS training program/trainer/material itself to determine if it's effective and how it can be improved

That second bullet, above, is what "continuous improvement" is all about.

Use all information from your training evaluation to improve court content, delivery methods, materials, and learning environment.

EHS training developers and trainers should also use the results of the learner evaluations and all other training evaluations to periodically review the effectiveness of the training materials and training presentation themselves. Evaluation should include course content, training delivery methods, additional training materials, trainer performance, learning environment, and more.

Remember to also use information from incident investigations, job-site observations, safety audits, and inspection data to make improvements to the training as well.

Conclusion: The Importance of Evaluating EHS Training

No doubt you recognized the importance of evaluation the effectiveness of your EHS training and the learning of your employees even before you read this article. With luck, this article has given you some best practices you can use to evaluation your EHS training more effectively.

Of course, we encourage you to go get the ANSI Z490.1 standard as well.