Cost vs. Accuracy of Talent Reviews: Managerial Ratings & Talent Assessments

With the last quarter usually come end-of-year talent reviews. The process isn’t particularly loved by management or employees, but it does serve an important function – it’s an opportunity for the business to collect data about employees, such as their performance and potential for a bigger role. It is this data that informs how organizations manage their single largest investment – their people. Who is promoted, developed, identified as high-potential, flagged as a successor for a key position, and what training the company offers are some of the decisions made based on this data. And, in many companies, these decisions impact up to 70% of their total business costs.   

Cost of Talent Reviews

The review process itself is quite laborious. On average, managers spend 210 hours per year on performance reviews. Since each manager would have about 10 direct reports, he or she would be expected to spend 21 hours per employee. After multiplying hours spent on performance reviews by their average hourly rate ($28.15), the cost of managers’ time is $591.15 per review of a single employee. In addition to managers’ time, each employee also spends about 40 hours per year on the process. At the average hourly rate of $11.33, that equals to an extra $453.20. Taken together, the average cost of review per employee is $1,044.35 per year, and a company with 1,000 employees spends about $1,044,350 every year on talent reviews. 

Accuracy of Talent Reviews

The investment in talent reviews is certainly justified when we realize that the collected data inform decisions about the people in the company. Since labor represents up to 70% of total business costs, these are arguably the most important business decisions for every company. And, of course, the more accurate the data, the better the decisions management can make. So how accurate are managerial ratings collected during talent reviews?

We can answer the question from the perspective of reliability, or how much error there is in the measurement. A ruler is a perfectly reliable tool with no error: A letter-sized sheet of paper will always measure 8.5×11 inches, no matter which ruler I use or who does the measurement. But measuring employee performance and potential is more complicated. If I asked two managers familiar with the same employee to rate her, I would most likely get two different perspectives and the two managers would not agree perfectly. In fact, multiple research studies showed that the overlap is only 52%, which leaves the other 48% to error. That is a relatively high margin of error considering the significance of the decisions made.

A different way to study the accuracy of managerial ratings is by analyzing how well they predict outcomes of interest. We conducted a study (n = 290) at a large technology company and found that neither ratings of performance nor potential predicted how well the employees would perform as leaders – how well they actually retained talent on their teams and how engaged their teams were. The correlations were near zero as seen in the table below. This shouldn’t come as a surprise knowing the large error rate of managerial ratings.  

What’s the Alternative?

Despite the high cost of talent reviews, the accuracy of collected data in the form of managerial ratings is disappointingly low: measurement error represents about 48% and they generally don’t predict important outcomes like how the employee would function as a leader. It’s a paradox for management to make decisions that impact almost 70% of the total business costs on data with such a high error rate and low predictive power.

Only better quality data can lead to better decisions. If the goal of talent reviews is to collect the most accurate data about employees and at a lower cost, perhaps we need to search beyond managerial ratings and look at talent assessments. Comprehensive assessments such as Pinsight rely on well-researched tools and processes to arrive at more accurate measures of employees’ performance and potential. They provide more accurate data to management for better business decision-making and at lower cost.

Conclusion

As seen in the table above, the error rate of a Pinsight assessment is about 23%. Unlike managerial ratings, Pinsight assessments significantly predict how employees would perform as leaders – how well they retain staff on their teams (r = .28) and much their teams are engaged (r = .27). Given that such assessments tend to be 43% less expensive per employee and provide more accurate and predictive data, organizations should consider them as a viable option with the potential for significantly higher ROI. Replacing the traditional talent review process with an annual assessment program in a company of 1,000 employees would result in $444,350 of savings and exponentially better business decisions coming from more accurate data.