Our colleague Carl-Johan Holmberg, researcher at Refapp, has co-authored a new scientific article together with Kathryn J. L. Jacobson and Ryan P. Jacobson on how overly positive ratings can be reduced in structured online reference checks.
Published in the International Journal of Selection and Assessment, the study examines leniency bias – the tendency to evaluate a candidate more positively than may be objectively warranted.
Across a series of experimental studies involving more than 1,700 full-time managers in the US and UK, the researchers found that managers tended to provide more positive assessments when completing a structured online reference check than when assessing employees privately.
The researchers also explored why this happens. Their findings suggest that one explanation is vicarious self-promotion: managers may see an employee’s performance as a reflection of their own competence as a manager, which can contribute to more positive assessments.
Importantly, the study also identified a way to reduce this bias. When reference providers were instructed to take the perspective of the hiring manager when completing the reference check, ratings of performance and counterproductive work behavior became less lenient.
Scientific findings like these are an important part of how we develop and improve Refapp. By applying insights from research to our product development, we can continue refining how reference checks are designed to capture relevant and useful information for hiring decisions.
One example is our module designed to capture counterproductive work behaviors (CWB) – information that can be particularly valuable when assessing potential risks and a candidate’s suitability for a role.
Refapp data shows that 9% of candidates assessed through this module receive a remark indicating that they have displayed counterproductive work behaviors. This could include frequently arriving late to work, using organizational resources carelessly, or other behaviors that have resulted in disciplinary action.
This type of information can be difficult to uncover through other selection methods. Capturing it in a structured reference check can therefore add valuable insights to the overall risk and suitability assessment of a candidate.
For us, this is an important part of evidence-based product development: using research to better understand what affects the quality of reference data and how our methods can continue to improve.
“We” Are High Performers: Vicarious Self-Promotion Underlies Managers’ Leniency in Online Reference Checks by Kathryn J. L. Jacobson, Ryan P. Jacobson and Carl-Johan Holmberg.