Structured interviews now outrank cognitive ability tests - what that changes for your programme

In 2022, Paul Sackett's research team published a correction in the Journal of Applied Psychology to a statistical treatment that had been used in personnel selection for over two decades. The result was a rewritten ranking of predictive validity, with structured interviews moving above cognitive ability tests. VCO Group summarises the design implications for recruitment programmes in Vietnam.

What changed and why it matters

For more than twenty years, almost every hiring process design decision rested on a ranking published in 1998, in which cognitive ability was treated as the strongest predictor of job performance. That table appeared in textbooks, in test vendors' sales material, and in the reasoning behind most proposals to change a hiring process.

The research team - Paul Sackett, Charlene Zhang, Christopher Berry and Filip Lievens - identified a technical problem in how earlier meta-analyses corrected for range restriction. In short, the correction had been applied too aggressively and unevenly across tools, causing some to be rated higher than warranted.

After recalibration, the conclusions shifted in two ways. First, most common selection tools have lower predictive validity than previously reported. Second, and more important for anyone designing a process, the relative order of tools changed materially. Structured interviews moved to the top, while cognitive ability is no longer the standout instrument it was taken to be.

Sackett himself has described this as the most important work of his career, calling it a correction to the field's accumulated body of knowledge.

Four direct implications for a running programme

The table below translates the research conclusions into specific decisions a recruiting team can act on next season.

From research conclusion to process design decision

This table summarises for design decisions. Specific validity coefficients should be referenced directly from the original paper, as each carries substantial variation.

What "structured" actually means in practice

The term structured is used loosely in recruitment practice. Many companies believe that having a list of suggested questions already constitutes a structured interview. In the research literature the concept is considerably tighter.

An interview round counts as structured when several conditions hold simultaneously: every candidate receives the same question set in the same order; questions are derived from job requirement analysis rather than personal instinct; each question has a rating scale describing the behaviour matching each score level; interviewers score each question immediately after hearing the answer rather than forming an overall impression at the end; and multiple assessors score independently before conferring.

Dropping any one condition reduces validity. This is why two companies that both say they use structured interviews can get very different results.

Worth noting for programmes in Vietnam: structuring is the lowest-cost improvement available anywhere in the process. It requires no software purchase, no increase in attraction budget, and can be implemented within a single season. What it requires is operational discipline and time spent training interviewers.

Why asynchronous interviewing connects directly to this finding

There is a bridge between the research conclusion above and a technology being deployed widely in Vietnam, and that bridge is often missed.

In the professional literature on asynchronous interviewing, one argument is made plainly: no interview format has higher structural potential than the asynchronous format. The reason is mechanical. Every candidate receives exactly the same questions in the same order, with no improvised follow-ups, no information carried over from a previous candidate, and the same thinking and answering time for everyone.

In other words, conditions that a live interview must work hard and stay disciplined to achieve are achieved by default in the asynchronous format.

This does not mean every automated interviewing tool is good. Assessment quality still determines outcomes, and how the rating scale is built, how reviewers are deployed, and how edge cases are handled all remain serious design work. But it does mean that when a programme considers putting asynchronous interviewing into the first round, the supporting argument is not only time saved - it is structure.

Three things to start next season

For Young Talent Program and Management Trainee schemes in Vietnam, the three items below return the most per unit of effort.

First, rewrite the interview question set around competencies and build behaviourally anchored rating scales for each score level. This has the largest impact and the lowest cost. It is also the easiest to defend to a panel, because results become consistent and explainable.

Second, separate scoring from discussion. Interviewers score each question immediately after hearing the answer and record it before conferring with colleagues. This step costs almost nothing and removes most of the influence of whoever has the strongest voice in the room.

Third, start capturing data for local validity research. Link scores from each selection round to performance review results at 6 to 12 months. After two seasons, the company has evidence from its own candidate pool - evidence far more persuasive than any international meta-analysis.

Three things to do

1. Rewrite the interview question set around competencies and build behaviourally anchored rating scales, as this is the lowest-cost improvement available anywhere in the process.
2. Require interviewers to score each question immediately after hearing the answer and record it before conferring, rather than forming an overall impression at the end.
3. Start linking scores from each selection round to performance review data at 6 to 12 months, so that after two seasons you have validity research on your own data.

Frequently asked questions

What is a structured interview?

An interview round in which every candidate receives the same questions in the same order, questions are derived from job requirement analysis, each question has a rating scale describing the behaviour matching each score level, interviewers score each question immediately after hearing the answer, and multiple assessors score independently before conferring.

Are cognitive ability tests still necessary?

Yes, but their role needs repositioning. The same authors' 2023 research shows tool combinations with and without a cognitive test can achieve near-equivalent validity, meaning the weighting between components matters more than the presence of any single component.

Should you invest in a new test bank or in structuring the interview round?

For most programmes, structuring the interview round returns more. It requires no software purchase or increase in attraction budget, can be implemented within one season, and under the recalibrated 2022 validity ranking it is the leading instrument.

About VCO Group

VCO Group operates the assessment infrastructure behind Young Talent Programmes and Management Trainee schemes in Vietnam, covering application intake, aptitude testing, automated interviewing and Assessment Centres. VCO Group serves young talent and candidates with under three years of experience, working with large enterprises and multinationals in Vietnam. Its assessment and candidate management platform is called Testalents. Head office in Ho Chi Minh City.

If you would like to benchmark your own selection data against the market, or discuss the Young Talent Program Insight Report, contact Mr. Dung on 0334 889 068 or visit vcogroup.com.vn.