Using Digital Tools to Assess Talent

Introduction

The moderator welcomes the audience and introduces the speaker, Tomas Romero. He also explains that the event will be recorded and available to attendees in a few days.

Submitting Questions

The moderator explains how attendees can submit questions for the speaker during the webinar.

Setting Context for Talent Identification

The speaker introduces himself and discusses how he will examine technological developments in talent identification. He also mentions the importance of transitioning from older tools to newer innovations.

Audience Survey Results

The speaker shares the results of an audience survey on how well organizations identify and attract talent. He notes that most respondents rated their organization as "about average."

War for Talent Concept

The speaker discusses the concept of the "war for talent," introduced by McKinsey & Company 20 years ago. He notes that despite its widespread acceptance, there are still indicators that suggest progress needs to be made in identifying and attracting talent.

Normal Distribution of Talent Identification

The speaker comments on the normal distribution of responses from the audience survey, noting that it mirrors what has been found in wider research on talent identification.

Progress Needed in War for Talent

The speaker discusses various indicators that suggest progress is needed in identifying and attracting talent, including low employee engagement rates, high numbers of passive job seekers, increasing self-employment rates, and more people becoming entrepreneurs.

HR's Role in Talent Identification and Development

The speaker discusses the importance of HR departments contributing strategically to the growth of businesses by improving careers and work experience for employees. However, despite the potential for HR to help organizations understand their talent, they still have a poor reputation with both employees and leaders.

War for Talent vs. War on Talent

• The speaker argues that we should be talking about a "war for talent" rather than a "war on talent."

• Monies are being spent mostly at the top in leadership identification and development interventions, but the average experience that employees get from their managers or leaders is not congruent with all the money being spent.

• Only 20-29% of executives think their company is doing a good job at attracting top talent.

Importance of Understanding Employees

• Solving the crisis of understanding where organizations don't really understand their people is important for mutual benefit.

• Technology and innovation can add value by helping organizations understand their employees while simultaneously providing feedback that helps employees understand themselves.

Overcoming Intuition with Data Analytics

• The main reason why established or new tools haven't helped solve this problem is because we still operate on intuition more than we should.

• Most managers rely on intuition when judging potential or talent, even though data-driven management is becoming more prevalent.

• We need to find ways to overcome our intuition, and AI will help us do that by getting better data on potential and performance.

Improving Talent Identification Methodologies

The speaker discusses how current methodologies used to identify talent are flawed due to subjectivity and bias. They suggest finding more predictive signals that can help quantify an individual's future performance and potential.

Problems with Current Methodologies

• It's almost impossible to get any type of job without going through an interview, which is mostly unstructured and not very reliable or predictive of future performance.

• Performance is mostly judged through subjective ratings from an employee's direct line manager, which are contaminated by politics and subjectivity.

• Tools like the MBTI have almost cult-like followings but don't predict much of value in terms of performance or future employee behavior.

Overcoming Bias with Better Data

• We need to find more prevalent and predictive signals that can help us quantify an individual's future performance and potential.

• The speaker recommends the book "Prediction Machines" for more information on this subject.

The Importance of Data in Talent Identification

In this section, the speaker discusses how data is becoming a key differentiator in talent identification and how new tools are helping organizations translate human behavior into data.

New Signals for Talent Identification

• New signals for talent identification have three components - people, data, and prediction.

• Datafying people or translating human behavior into data presents the biggest opportunity for overcoming intuition and improving talent identification.

• Innovative talent tools provide new signals or indicators of an individual's potential and talent.

• Digital interview platforms such as HireVue enable standardized administration of questions to all candidates, increasing reliability.

Overcoming Bias in Talent Identification

• Developing algorithms that predict the preferences of human interviewers can help eliminate bias in hiring decisions.

• Targeting candidates' actual future performance instead of subjective impressions can overcome biases that exist in current analog interviews.

• Speech or voice data captured by platforms like Harvey can be linked to perceptions of performance and business unit performance.

Translating Text Data

• Linguistic natural language processing can translate text data from digital communication into personality data, competencies, and ability indicators.

• Research has shown that self-referential pronouns are indicative of narcissism levels while using more complex words indicates curiosity and openness to new experiences.

Conclusion

In this section, the speaker concludes by summarizing the importance of data in talent identification and how new tools can help organizations overcome biases and improve hiring decisions.

• Data is becoming a key differentiator in talent identification, and new tools are helping organizations translate human behavior into data.

• Developing algorithms that predict future performance instead of subjective impressions can help eliminate bias in hiring decisions.

• Speech or voice data captured by platforms like Harvey can be linked to perceptions of performance and business unit performance.

• Linguistic natural language processing can translate text data from digital communication into personality data, competencies, and ability indicators.

Personality, Intelligence and Gamification in Talent Identification

In this section, the speaker discusses how social sensing monitoring can predict future performance and work done by individuals. The speaker also talks about gamification attempts to improve the candidate experience.

Social Sensing Monitoring

• Social sensing monitoring can indicate group level dynamics and social networking.

• It can also tell us something about an individual's degree of entrepreneurship, curiosity, pro or anti-diversity stance, inclusivity etc.

• Digital reputation scattered around the web can be translated into valuable elements or components of an individual's reputation including their potential or talent.

Gamification in Talent Identification

• Gamification attempts to increase or improve the candidate experience by either shortening the assessment process or making it more fun.

• Most gamified tools are situational judgment tests that give people different scenarios to choose from.

• There are examples of behavioral or simulation activities like mind x gamifying IQ tests and biometrics gamifying impulsivity or risk-taking tasks.

• Evaluation criteria for judging whether tools work include accuracy, speed, cost, candidate experience moderated by ethics.

Transition from IO Psychology to AI-based Talent Identification Tools

• Historically IO psychology cared mostly about accuracy but didn't care much about time taken to put a candidate through it, costs and candidate experience.

• New tools need to think seriously about ethics while balancing accuracy with quickness and cheapness.

• Traditional science-based tools have had a low penetration in the market.

• There is a tension between accuracy, speed, cost, candidate experience moderated by ethics.

The Ethics of Talent Identification and Predictive Analytics

In this section, the speaker discusses the ethical implications of using data in talent identification and predictive analytics.

Data Collection and Privacy Concerns

• HR professionals need to be aware of all available data that could potentially be used for talent identification.

• There is a big difference between what we could and should know about people, and what the ethical implications are.

• It is important to consider whether it is ethical to withhold feedback from candidates if their data is being scraped without their knowledge.

Biases in Predictive Analytics

• Using tools to predict biased outcomes can often augment or maximize existing biases.

• It is important to move beyond black box models that are just predictive without being explanatory.

New Technologies in Talent Identification

• Face recognition technology can provide information about a person's preferences, attitudes, personality, sexual identity, religious beliefs, etc.

• DNA profiling companies like 23andme can provide information on genetic makeup that may be predictive of future performance including leadership potential.

Implications for the Future

• Short-term advances in talent analytics will come from organizations using internal data because it gets past legal constraints and some ethical concerns.

• The use of rating systems in social networking apps raises questions about how acceptable it is to use that data for identifying the right person for the right job.

This transcript was originally spoken in English.

The Potential of Talent Identification Tools

In this section, the speaker discusses how talent identification tools can make the job market more efficient and bridge the gap between supply and demand. However, there is a risk that these tools may increase inequality.

Efficiency of Labor Market

• Most people don't need to be assessed because they have already been assessed or their data can be easily translated into an estimate of potential.

• Better assessment or talent identification tools can make the labor market more efficient by bridging the gap between supply and demand.

• Identifying talent is not about finding the best people but finding the best jobs that people could do.

Risk of Inequality

• The use of talent identification tools may increase inequality and decrease social mobility.

• Despite this risk, it is feasible to expect that these tools will still help improve opportunities for those in lower socioeconomic classes.

Apocalyptic vs Optimistic Views

• Some believe that increasing AI and automation in the workplace will create a class of useless individuals, while others see great promise in merging science and technology to deploy assessment at scale.

• The speaker has written about this topic in his book "The Talent Delusion."

Q&A Session

This section features a Q&A session where questions are asked about bias in talent identification tools and how to properly vet new technologies.

Bias in Talent Identification Tools

• There is a risk of bias in talent identification tools, and it is important to mitigate this risk.

• As we look at new AI-driven solutions for talent evaluation, we need to know which ones have been properly vetted.

• A governing body or regulation may be necessary to ensure that these tools meet certain standards.

Mitigating Bias

• Large organizations can benchmark existing internal employees or leaders to minimize or eliminate adverse impact bias when evaluating new technologies.

• It is important to remember that bias also exists in human assessments, and we should strive for the same level of scrutiny when evaluating them.

Overcoming Bias in AI Tools

In this section, the speakers discuss how humans tend to prefer their own intuition over algorithmic results when it comes to evaluating other humans. They also explore the potential for bias in new AI tools and how they can be used to isolate individual characteristics too much.

Humans vs. Algorithms

• Humans tend to prefer their own intuition over algorithmic results when it comes to evaluating other humans.

• Technology needs to prove with results that it can do a better job than human intuition.

• People are more self-critical and humble when vetting products, but less so when judging or evaluating other humans.

Potential for Bias in New AI Tools

• The existence of new AI tools presents a problem of bias.

• Do these tools inherently give an advantage to people who are comfortable with them?

• High scores on these games or tools may be confounded with certain demographic characteristics like race, gender, or age.

• If performance on the game correlates with future performance on the job, then the game is tapping into relevant competencies that are desirable.

Isolating Individual Characteristics Too Much

• These tools may isolate individual characteristics and performance too much.

• It's hard enough to predict individual performance, let alone group-level performance and interactions between people.

• Benchmarking existing high-performance teams or cultures can help understand collective interactions or synergy.

Changes in Hiring Practices

In this section, the speaker discusses how changes in hiring practices have led to a bigger focus on team composition rather than just hiring high performers who are clones of each other. The speaker also addresses the risk of new tools being gamed and individuals creating profiles with the intent of being attractive to employers.

Team Composition

• Companies now focus more on team composition rather than just hiring high performers who are clones of each other.

Risk of Gaming New Tools

• There is a risk that new tools can be gamed, such as data mining and social profiles. Individuals may create profiles with the pure intent of being attractive to employers.

• However, if gaming these algorithms is indicative of positive qualities such as being smart, astute, and politically skilled, then it's not a problem. Companies want to hire people who can game the system.

Faking Good in Assessments

In this section, the speaker talks about faking good in assessments and how it correlates positively with certain positive qualities.

Faking Good Correlates Positively with Positive Qualities

• The ability to fake good correlates positively with the ability to pretend to work well with others in the future.

• The issue is not whether people fake or lie but whether they stop lying in the future.

• If there are crazy outliers in an organization who don't fit perfectly into the culture but provide unique value, they should not be selected out if companies have a flexible model for what they're looking for at a team level composition.

Gig Economy and Hiring Tools

In this section, the speaker discusses the gig economy and whether it makes hiring tools less important.

Gig Economy

• The gig economy is much smaller than the media suggests.

Importance of Hiring Tools in the Gig Economy

• On the contrary, if relationships between organizations and talent are becoming more transactional, then hiring tools become even more important.

The Importance of Trustworthy Scores

In this section, the speaker discusses the importance of trustworthy scores or indicators in determining whether an individual can be trusted. He emphasizes the need for more ubiquitous assessment tools and data to predict an individual's performance.

Trustworthy Scores

• Trustworthy scores are important in determining whether an individual can be trusted.

• More ubiquitous assessment tools and data are needed to predict an individual's performance.

• Potential is a data point that enables us to make a bet on an individual based on our estimate of their future performance.

Capabilities Needed for Effective Use of New Tools

In this section, the speaker answers a question about the capabilities organizations need to acquire to make effective use of new tools. He explains that as these tools become more user-friendly, the level of expertise required goes down but HR professionals still need to be somewhat data-driven and tech-savvy.

Required Capabilities

• As new tools become more user-friendly, less expertise is required.

• HR professionals need to be somewhat data-driven and tech-savvy.

• Organizations tend to think they are unique but there are core indicators of talent and potential that will always matter.

Conclusion

In this section, the speaker concludes the Q&A session by thanking the audience for their questions and reminding them about a feedback survey. He also announces that a recording of the program will be available within three to four business days.

Final Remarks

• A feedback survey will be sent by email in the next few days.

• A recording of the program will be available within three to four business days.