Measurements of control

Measurements of control are used to evaluate how well standards set for activities to manage the risk control system are being met.  Control refers to the function of professional management.  

These measurements are based on the accepted professional premise that the work to produce desired results in a safety or risk control system can be clearly identified and measured, and that these results can be evaluated in an accepted quantifiable language that enables management at all levels to correct performance deficiencies to the level of their established standard.  (Refer to the table on the next page.)  

Such measurements emphasise proactive risk control management activities.  They do not depend on losses having already occurred.  Examples of measurements of control include:

• Number of inspections conducted.
• The quality of completed investigation reports.
• Percentages of required investigations made.
• Rules reviewed.
• Percentage of task procedures completed.
• Number and effectiveness of group meetings conducted.
• Number of personal contacts made.
• Protective equipment compliance.
• Engineering prints reviewed.
• Task analyses completed.
• The quality of written task procedures.

There are several advantages of measurements of control.

1. They are easily understood. There is nothing vague about the fact that 50% of accidents reported are being investigated, or that 75% of required general or critical parts inspections are being conducted.
2. They give inference to control. An extremely high frequency rate may have great motivational value but all it can tell management is that “something must be done.”  On the other hand, 50% coverage of critical jobs by Standard Job Procedures gives influence to why people may be making mistakes.
3. They are not dependent on losses. They completely involve measuring compliance to system standards that produce results that control  In this context, they are proactive (before-the-fact).
4. They are suitable for inter-group comparisons. They could be used within a multiple-location organisation (with common standards such as the investigation of all reported accidents, the conducting of a proper job indoctrination with all new employees, the presentation of a group communication to all employees weekly, etc.).  These measurements can be utilised more equitably than measurements greatly influenced by variable exposures.
5. They minimise bias and can be statistically valid. A high degree of quantification can be built into a system that can be administered by trained safety or risk control personnel.  Measurement tools already accepted by management as valid barometers of system effectiveness can be utilised.

Control measurements can be both comprehensive measurements (audits) and representative measurements.

Once the S-H-E-Q manager decides to establish control through professional management techniques, he or she must decide how to best utilise time and resources to accomplish the job in the most efficient manner.  It would be utterly impossible for the average system manager to accomplish the measurement of performance levels for each management group in the organisation each month in all work activities.   

It is far more practical to devise a system that measures management performance in representative work areas to detect substandard performance on a timely basis in order to maintain continuous control.  Selecting six to ten work activity areas for measurement and evaluation on a bi-monthly or quarterly basis would be a manageable practical goal.

Representative measurements focus on compliance with established standards in selected activities, i.e.:

• “Are we doing what our process says we should be doing?”
• “Are we on track?”

These are more frequent measures than audits and are usually done at regular intervals. 

Monthly, quarterly, or biannual measurements of the quantity and quality of planned general inspections conducted, investigations done, group meetings held, etc.

Measuring compliance with standards set for these ongoing activities helps the organisation evaluate performance on a timely basis, constructively correct substandard performance and reinforce excellent performance.

Comprehensive measurements usually take the form of audits, which are conducted on a periodic basis (such as annually).  They are “snapshots” of the level of activity going on at the site to control risk.  

The questions asked during an audit specifically inquire whether the site is engaged in distinct activities which are proven to reduce loss if implemented and operated properly.  The audit measures the adequacy of the risk control system and its standards to control the loss exposures at that site.  The audit asks:

• “Does our system include these activities?”
• “Do we have the proper standards for doing them and are we in compliance with the standards?”

The auditor gauges the functional effectiveness of an activity by reviewing records, interviewing people and evaluating physical conditions.  

A comprehensive measurement of the entire programme (all major work areas) could be made on a less frequent basis than representative measurements.  A minimum goal should be annually; quarterly would be considered ideal.  (Refer to the table, “Risk control performance evaluation” on page 14.)

As experience is gained in administering the management process involved, measurements become increasingly objective as well as efficient.  A manager could elect to measure five or six work activities that are considered the critical few from a basic need standpoint (i.e., Incident Investigation, Planned Inspections, Protective Equipment, Group Communications, and Personal Communications).  

The manager should select four or five different areas each measurement period and maintain a desired level of performance at all times in all areas.  This minimises the possibility of concerted effort on only a few areas receiving maximum attention continuously.  

Further minimisation of overall effort can be maintained by keeping score of year-to-date performance levels as well as the performance at the time of the measurement.  These measurements can become the basis of recognition for management personnel at the department, division, and organisation levels.  

A similar measurement system can be utilised to measure the performance level of the individual front-line supervisor and can be utilised as an important part of the annual job performance review.  

Again, Figure 2 (page 9) shows where control measurements fit into the causation model.

While a variety of methods can be employed to measure management performance levels for the various work activities in the programme, the three briefly discussed below will probably be utilised more than others.

 1. Random Sampling

This accepted tool of management (suggested earlier to determine the composition of the whole by examining a large enough representative part) can be used as an effective measurement and time saver.  Since the particular statistical populations being measured in most of the areas involving measurements of control do not change rapidly, fewer observations are needed when sampling such items as safe behaviour, which has a fast, constantly changing population.

Random sampling is especially helpful in measuring personal communications of various types, as well as compliance with protective equipment requirements.

2. Actual Count

This simple, objective approach of actually counting the items involved and comparing results of the number prescribed by the standard can be used in such work areas as inspections, job procedures, job observations, investigations, rules, and skill training.  Measurement by actual count is probably the most widely used technique; it may also be used in conjunction with professional judgement when quality factors are included as presented in the example under professional judgement.

3. Professional Judgement

 
When a measurement system is initially introduced to determine the degree of management control, the stress is usually placed on quantity rather than quality (i.e. holding group meetings 100% of the time is usually considered more important than having great concern about the degree of their planning and content).  As time and experience create confidence in the system, the manager will desire to encourage quality in communications, job procedures, inspections, observations, etc.  She/he can then employ a value factor method of scoring and utilise a judgement system to measure quality.

Experience has shown that whenever judgement is employed, accuracy can be increased by applying these steps for measuring the quality of control: 

1. Break the item down into its components.
2. Place a value factor on the parts.
3. Ask enough questions about each part to increase certainty of judgement.
4. Record findings accurately.

In other words, accuracy is increased when a method is utilised to facilitate a systematic evaluation of whatever is being judged.

A value factor could be defined as the worth of a component as compared to the worth of the whole. 

Standards for Standard Job Procedures could include a specific amount of prescribed formal training, a specific objective number of procedures to complete, and a specific requirement for updating procedures. 

These three component areas of work involved, with related standards, could be broken down and assigned value factors:

Similarly in order to determine a total loss control performance rating, each work area would have to be given a value factor in order to combine scores into one figure:

While the establishment of value factors is subjective, based on the opinion of the professional staff, a difference in opinion is offset by the overall average of the system.  The biggest benefit of value factors is that this system prevents an insignificant component factor from disproportionately influencing the evaluation of the whole.

Before selecting any measurement that will provide management with useful tools for the control of losses, it is necessary to determine what it is one wants to measure and what purpose the measurement will serve in the achievement of desired goals.  

The nature of the interactions of multiple causal factors that lead to various types of loss is of such complexity that no single measurement can possibly fulfil all needs of management.  It has been suggested that management’s needs can best be served by providing measurements of consequence, measurements of cause, and measurements of control, each providing a different type of information and serving a separate useful purpose. 

While all three measurements are important, safety leaders concentrate on the measurement of management compliance with established standards in the local safety or risk control system (measurement of control).

The timely measurement and evaluation of work performed to standards provides management (at all levels) with information needed to correct performance and produce desired results.

This measurement, more than any other, applies the principles and skills of professional management.  With diligent application, we can attain the sure results in risk control which have been achieved in quality control, production and other important areas of management concern.