HBOMax’s Persona Couldn’t Have Come at a Better Time!

No matter how upsetting you found the recent HBO Max and CNN Films documentary, Persona: The Dark Truth Behind Personality Tests, it couldn’t have come at a better time. The business of talent assessments is exploding as venture capitalists pour billions into new HR technology. And we should all feel wary that, perhaps in some companies, chasing profits may win over good science. 

Yes, from the perspective of many Industrial/Organizational Psychologists, the documentary is controversial, one-sided, skewed, and taken out of context. But it carries a very important and timely message: All of us need to become more informed consumers of people assessments.

Whether you’re purchasing assessments, using them to make decisions, or being asked to take them, their rise in popularity means that you’ll encounter them more often and in places you perhaps didn’t even expect:

Looking for love on dating apps? A personality test can help narrow down your search to your perfect matches.

Investing in a new startup? A leadership assessment can help you understand the founder’s knack for business.

But picking the right assessment and knowing how much weight to give it in your decision can be challenging. So how do you make sense of assessments and ensure responsible use?  These three steps can help you get started:

1. Ask to see the technical manual and read it.

Most legitimate assessments come with a technical manual – a summary of the research findings supporting its use. In it, you’ll find information about what the assessment does and does not predict. For instance, assessments designed to inform hiring and promotion decisions will show sufficiently high correlations with important business outcomes, such as job performance, employee engagement, retention rate, organizational tenure, and career advancement. For the assessment to be useful in your decision, you should typically find correlation coefficients (commonly reported as the letter “r”) in the .21 to .35 range.  

Pay attention to these predicted outcomes and make sure they align with how you plan to use the assessment. For example, if the assessment only predicts sales performance, don’t use it to hire sales managers as it won’t predict their ability to lead. Similarly, if the assessment only predicts engagement, don’t expect it to identify top performers. Careful examination of what outcomes the assessment has been validated to predict will ensure responsible use.

2. Check the technical manual for group differences.

As you’re reviewing the technical manual, zoom in particularly on group differences between men and women, racial and age groups. Be wary of assessments that don’t report any analyses of group differences. It could mean that such analyses were not performed and so the potential for bias in the assessment is unknown. Using assessments with unknown bias means it might be an unfair assessment and could expose your organization to legal risk.

Some types of assessments show larger group differences than others. For instance, ability tests tend to show a larger black-white difference than personality tests or simulations. This may not be necessarily an inherent bias in the test, it could very well be that the test is accurately picking up on the inequalities inherent in the society.

The magnitude of group differences is commonly reported as effect size and the letter “d”. A d value around 0.2 is considered small and unlikely to result in unintentional discrimination. As the value increases to 0.5 and above, so does the risk of unintentional discrimination. Assessments that show large group differences should be used with caution and you should always supplement them with another assessment than shows smaller differences. 

3. For best results, combine different types of assessments.

People are complex to measure, and no single assessment will ever do them justice. But if you combine multiple different assessment types (ability, personality, simulation, interview), you can get closer to the truth. Research has shown that combining an ability and personality test with assessment center simulations can lead to very high predictive power. This approach can also balance the higher group differences in ability tests with smaller differences in personality and simulations, thus decreasing the potential for unintentional discrimination.  In other words, using multiple methods provides organizations insights to make unbiased hiring, development, and succession talent decisions.

We’re living in the age of data. As more data becomes available to us, we need to become better consumers of that data, especially when it informs important decisions about people. We could slow down the adoption of talent assessments and risk going back to unscientific, haphazard decision-making rooted in opinions and gut feelings. But good assessments that are used responsibly can lead us to better decisions that are fair and rooted in science.