What are some of the existing challenges in hiring?
Michael Campion: Some of the existing challenges to the hiring process include the fact that they are costly and time consuming. This includes for candidates who must complete assessments, as well as for hiring officials and recruiters who must score them. Plus job openings that remain open cause lost productivity and, as well, good candidates can get jobs elsewhere if they are not hired in a short period of time. Moreover, human judgment has many limitations. There is subjectivity. There's biases. There's differences between people that lead to unreliability and the possibility of discrimination through unconscious bias and direct bias. Also, candidates impression manage in hiring. Meaning if they are evaluating themselves, they can give you impressions of them that are not accurate. Artificial intelligence may allow us to score candidate information in a manner that cannot be faked. Finally, there is a neglect of job related information in the hiring process that artificial intelligence can help with, such as narrative information in applications, in resumes, and often even in interviews.
How does Talent Select AI solve these existing challenges in hiring?
MC: Talent Select’s AI products, solve some of the existing problems in the hiring context in the following ways. First, the tools allow us to measure skills and personality traits, as well as the content of applications and interviews through automated text analysis and other advanced artificial intelligence procedures. This allows us to measure competencies, personality traits and motivation in a manner that is improved over prior systems. It may also allow us to examine structured interviewing questions, which is a process in development. This does not require additional assessments on the part of candidates or additional effort on the part of the employer to score those applications. The scoring is automatic and the assessments are conducted on existing material submitted by the organization, such as their application, and answers to questions that might be part of that application. Because these are scored in an automated way, they cannot have subjectivity or unconscious bias, so should be fairer to candidates and more objective. Finally, there are no development costs. These are products that are fully developed and that can be used in an organization immediately.
Why are so many personality tests and assessments unreliable?
Emily Campion: The reason personality tests and other self-report measures of human attributes can be unreliable is because of fakeability. Now, when we talk about fakeability, we're not necessarily saying that people are attempting to hugely distort the truth or lie about who they are. But we live in a society where there are socially desirable responses to things and context clues of how we should be responding behaviorally or attitudinally to things. So, for example, in a hiring context, if a recruiter or a hiring official or an interviewer asks you, you know, are you dependable, are you organized? You know, to say “yes.” If it's in what we call a Likert type scale, where it’s sort of "Strongly Disagree" to "Strongly Agree" according to statements and the statement is “I'm a highly reliable or highly organized person,” you know that in a hiring context, you need to say “strongly agree,” even though maybe you're not a very organized person, because we know in that context we should be organized. We should be presenting ourselves as a highly organized and reliable person. And that would be, of course, the Conscientiousness factor of the Big Five. And social desirability also plays a big role.
So let's pretend your, uh, one of the other parts of the Big Five, is a facet called Agreeableness. Let's pretend you're a really disagreeable person. You maybe are sort of critical, maybe even combative. You don't just sort of like go with the flow sometimes. You're not maybe very compassionate when you're speaking with people, but you know that the socially desirable thing to do is to say that you are maybe a generally agreeable person because society makes us feel this way. That social pressure and those social forces can influence how we respond to things. And this is a really big problem in self-report measures. When I say self-report measures, I mean close-ended items where you are responding on a scale 1 to 5, that agreement scale I mentioned before, and we have a sense of how we should be responding. And that's this fakeability and this motivation to say something that's socially desirable, something that will be socially accepted, plays a role in how we respond to things. Particularly as it relates to personality and also work related attitudes to things that we might be able that we might be able to fake for at least a little while in the beginning of a job or during an interview. Because it's easy for for these test takers to indicate what they think others want them to say.
What is the biggest misconception about AI in hiring?
EC: So I think there's a few misconceptions about AI and hiring. And I think the first one is really potent and something maybe we don't fully appreciate. The job search process is already, I think, fraught with misunderstanding. Job seekers may not always have a good idea of what organizations are looking for. They might not always know why they don't get a job. They may not always know why they get, you know, they get the job. And and so because there's so much, I think, confusion and there's this sort of black box in hiring where, you know, candidates don't always know what's being looked for by organizations. I think adding technology, especially something like artificial intelligence that we don't that we feel like we don't fully understand or we feel like we can't fully understand makes it all the more confusing. So there's this affective or emotional response to a situation that's already high stakes.
Let's think about hiring and the role it plays more broadly. Employers want to hire really good people who they can retain because turnover is costly and employees or candidates need to work to make money to live. And so hiring decisions are not just these low stakes decisions that we can sort of do poorly and without consequence. I mean, they have very significant consequences on the economic realities of people's lives. And so amping up the stakes like that makes it all the more important that people feel like they understand why these decisions are being made. And so adding something that they feel like they can't understand, even though I would argue that they can with enough time and explanation. So adding that complication to it makes it all the more confusing and again, evoking this more affective reaction. So I think that's one of the first misconceptions about AI.
MC: Some of the biggest misconceptions about AI in hiring stem from the fact that it's the fear of the unknown that most people have because they don't fully understand how artificial intelligence works. In fact, most people really don't understand it at all. The result is that they imagine the worst, for example, that the artificial intelligence, because its model based on previous employees, will only produce acceptable candidates who look demographically just like their past employees. Which couldn't be further from the truth because the artificial intelligence doesn't look at any of those factors and only looks at skills and abilities, at least in our case, that is what we measure, which cannot possibly reflect demographics intentionally. So it reflects past employees to the extent that it's looking for skills and abilities, but not at all looking for demographic similarity or any characteristics that could be linked to demographic similarity. In addition, the negative public press that has resulted from high profile lawsuits in some states that have proposed laws such as New York City are also creating negative press that people read and assume that it is true. And that probably contributes to the problem.
EC: Humans, unlike machines, are subject to, you know, mood effects or environmental factors that might influence how we perceive and evaluate information. And so, you know, it really takes us a substantial amount of personal resources to consistently evaluate and weigh the same variables similarly, across candidates over time. Computer algorithms do this so much better than we do because they don't tire out. They don't have other thoughts that are entering their mind. They don't have distractions. They don't have previous, you know, preconceived notions that influence how they look at a resume or how they look at someone's job experience or where they've worked or, you know, their career trajectories or anything like that. They they don't. We train these models ourselves and that can offer an improvement over sort of the subjectivity that really does come in when it comes to human evaluations. And so for this reason, that can be difficult to understand how and why humans arrive at certain decisions, particularly in selection, unless they're highly structured, highly trained, highly structured selection processes with highly trained individuals. Whereas even then sometimes, you know, some of our subjective perceptions find a way in.
But when we evaluate, you know, a candidate, we evaluate candidate applications using algorithms. We can typically try and understand fairly well how and why we arrived at a score, because we know what's being scored and we know how it's being weighted. And so in this way, I think that when we talk about misconceptions about AI, a big misconception is potentially the value it brings to reducing some of the that some of those subjective perceptions that come into the into hiring practices just because it's done by a human. And I think for many of us we feel like things are maybe fairer when a human is evaluating us or we feel like we might feel better or, you know, less vulnerable when we're being evaluated by a human rather than a machine because we feel like we can't understand what the machine is doing. But in reality, there are a lot of tools that researchers have to be able to better understand why a decision is arrived at or how a score is estimated when it comes to using an algorithm versus a human at times. So I think that's one of the biggest misconceptions we have about AI.
MC: The reality is the artificial intelligence is fairly simple. All it is is an automated way to objectively and consistently score job related information, and it does so very effectively. Ironically, it's fairer than humans making those judgments because it cannot possibly be subjective. I look forward to the day when the criticism will be not “Why did you use artificial intelligence?” but “Why didn't you?” Because then there would be no possibility of subjectivity. I bet that happens in my lifetime.
I believe the solution to it all is probably to make sure people understand what artificial intelligence measures. Take the time and the care to do so. This includes managers, for sure, but it also includes the general public, and to do so by helping them see exactly what is measured and how it does it. In the case of the Talent Select products, they are very easy to interpret in an intuitive way where the information scored can be examined by anyone and judged to be job related by most of them, I believe. And there's no hidden technology that cannot be examined.
How can AI positively impact the hiring process?
MC: Artificial intelligence can positively influence the hiring process in at least several ways. First is the obvious efficiency advantages. This includes efficiency in administration, in scoring, and even more efficiency for the candidate, as we'll explain in a bit. In addition, there is the benefit of increased objectivity, consistency and objectivity that is enabled by the fact that the computer itself scores in a very consistent way and it is incapable of any biases. The scoring of the information by artificial intelligence also importantly, allows us to measure the things that we have historically not measured very well, such as the text information in applications and in resumes, as well as answers to questions that may occur there or answers to interview questions. Historically, we've not scored that information very well, if at all, and AI allows us to do it and do so very effectively. The result of all this should be a more accurate and complete assessment of candidates that should improve outcomes such as the job performance of various employees, reduction in adverse impact, more perceptions of fairness on the part of the candidates, and better reactions from managers who won't be burdened with having to do excessive scoring.
How can Talent Select AI identify the best candidate beyond who is "most qualified?"
MC: Talent Select hopes to be able to identify the best candidate beyond who would be considered the most qualified based on traditional measures in at least a couple of ways. First, we hope to measure attributes that complement the existing attributes measured at time of hire. These include personality traits, competencies and the motivation of candidates. We did not want to produce yet another job related test of skills or knowledge, or yet another personality test of which there are many, many on the market. We wanted to instead evaluate constructs that are historically not well measured to add to the total information.
In addition, we wanted to measure information that is historically not very well used and that contains that kind of information, and that is the text information provided by candidates as part of their applications and interview answers. And this becomes now currently enabled by artificial intelligence. So it's a great opportunity. We hope to be able to predict beyond current assessments. That means tests of mental ability and other traditional hiring procedures by measuring these core capabilities in the areas of competencies, personality traits and motivation. We hope that this will give us a more holistic view of the candidate that includes a broader range of job related skills and other characteristics.
How can text analysis capture personality and other non-cognitive traits?
MC: Automated text analysis can capture skills, personality and other traits, primarily by using natural language processing, which has a long history of research in linguistics and in psychology. What is really only new about it is not necessarily the core technology, although there are some things that are new there, but it's the application to employment decisions, which is an obvious application because there's so much narrative information. At its simplest level, it counts words and phrases that reflect the skills and personality traits. This is very similar to how an interviewer listening to a candidate will likewise pick up on words that reflect personality and skills in the same manner, but do so subjectively. The advantage of the computer is it doesn't miss anything, the candidate says, and it scores all candidates in exactly the same way, which are, of course, you know, two major advantages that people could never match.
EC: Unlike a lot of assessments that seek to capture personality, attitudes, or competencies through these self-report measures, Talent Select is using word dictionaries to do that. And it's important to remember that candidates produce at times an incredible amount of text data when they're applying for jobs. And it can be so time consuming for hiring officials to read and score those resumes or essays or objectives or letters of recommendation. And so text analysis can really help reduce time spent on materials.
MC: Another way to think about this is that the words and phrases that people use to describe in their answers reflect past behaviors, past tasks, and past activities that reflected these skills and abilities. So it's really very logical. For example, in the case of, let's say, teamwork skills, we might measure “cooperation,” “assist others,” “resolve conflict.” All of those words reflect the underlying skill of teamwork. Or let's consider perhaps the personality trait, conscientiousness. Use of words like “act,” “complete,” “dependable,” “take responsibility” would all reflect conscientiousness, and we measure many hundreds of these words. And that's how we attain a reliable measure that's fairly complete.
Why are the Great 8 and Big 5 frameworks used to evaluate workers?
MC: Talent Select decided to measure the Big Five personality traits and the eight Great Eight competencies because they are so commonly required on jobs by virtue of the way they are developed or their name. So the Great Eight competencies grew out of research on competency use across a wide range of organizations and first was published in... about 2005, I believe, in Journal of Applied Psychology that asserted that these great eight are so commonly used that they are universal and needed by virtually all jobs.
EC: Using word dictionaries to capture a candidate's either big five personality or great eight competencies such as leading and deciding or adapting and coping, I think makes great sense in the hiring context for a couple of reasons. First, we know from the literature that these attributes contribute to our ability to perform. So we know that the Big Five personality traits, particularly conscientiousness, can be predictive of our performance. Conscientiousness can be predictive across jobs.
MC: The Big Five personality traits came from could be potentially eighty years of research that had produced hundreds of different personality traits. And finally, in about 1990, we converged on the Big Five as the best way to describe the range of possible personality traits. And since that time, they've been used for hiring, because most jobs will require those traits or some subset of them. For example, we have found that conscientiousness and agreeableness are highly predictive of job performance in many, if not most, samples where we've studied them. And in addition openness to experience, and sometimes extroversion, can be also predictive. And even, sometimes, emotional maturity, which we have really labeled “emotional intensity,” because we feel that's a more accurate description.
EC: Some of these other parts of the Big five can be particularly predictive in certain occupations. For example, you can imagine that a sales person who is extroverted is perhaps much, much more successful than a salesperson who's maybe an introvert, although there are exceptions.
How do some of Talent Select AI's additional traits, such as "Grit," compliment those frameworks?
MC: Talent Select’s measures complement the Big five by measuring other non-cognitive traits that we thought were important to the hiring context, but heretofore unmeasured. They include enthusiasm, proactivity, empathy and grit, which I’ll define in a moment. These are, I think, best described as motivationally related constructs that are not cognitive skills but reflect the candidate's overall amount of motivation. So they are different than personality, but they are still non-cognitive.
Taking first enthusiasm is a characteristic that virtually all hiring officials value, especially in the context of interviews that may be one of the most noticeable features and usually something everybody views positively.
The second proactivity is a direct measure of motivation and one that reflects a proclivity to be very active and to take action, which again has got to be related to performance in about every job.
Grit is a complement to those because it measures motivation, but more the long term, the ability to be sort of continuingly passionate about a topic, putting effort over a great period of time, which is highly relevant to most jobs. It measures things like resilience, self-determination and the ability to delay gratification. So really long term motivation. And finally, there's empathy.
And the empathy measures are a complement, we think, to the others that is more focused on the human side and one's ability to relate to other human beings. We think that's especially important to supervisory and management personnel.
How can Talent Select AI promote DEI initiatives?
MC: Talent Select can help promote DEI initiatives in at least three ways. First, the algorithms can have no subjectivity or unconscious bias. That is obvious first. Second of all, the method of identifying skills and abilities from candidates, narrative information and other background means that we may be able to identify candidates who do not have the classic background in terms of the jobs and the education that has characterized employees hired in the past by the organization. Instead, we measure the skills and personality traits of the candidates from the things they have done, and that may identify candidates who would not have been identified before. And then thirdly, because these assessments are evaluating existing information that the candidate submits and not require them to complete a new assessment, especially a test, they avoid the test anxiety and the stereotype threat that might be possible with other types of assessments. And by stereotype threat, I mean the self-imposed limitations people often have through test anxiety when they know that people like themselves may not perform as well in those tests. And this would be a way to avoid any possibility of that threat.
What about different demographics?
So it's important to recognize that the artificial intelligence products that we are promoting here do not in any way measure demographic characteristics such as where you grew up, the name of the school you went to, or necessarily even your academic major. It instead measures words that reflect underlying skills and abilities, and they can be represented by any race, any gender, any age, by people who have had those those experiences and who can describe them in words that we can then detect. But it's not something that we are looking for a particular profile, as in went to the right schools, had the right majors, did the right internships. It's instead focusing on the demonstration of those skills and abilities.
EC: One conclusion we can draw from developing these text dictionaries is that the traits, competencies and motivations we're measuring should not show notable demographic subgroup differences because they're capturing concepts that most individuals have had the opportunity to develop through any work and school experiences.
What is your relationship with Talent Select AI?
MC: I've helped Talent Select AI validate its technology by serving as a consultant in the development and validation of these selection procedures. It's important to note that I am a consultant. I am not an owner of the company and I get no payment for sales or based on profitability. I'm just strictly on an hourly basis. Talent Select wanted somebody outside the company to do the validation in order to add the objectivity that that would provide, and I can provide that.
What excites you most about Talent Select AI?
MC: What excites me most about Talent Select AI is the possibility of new discovery. I think my reputation in the field is I'm someone who looks for improvements in practice and very importantly always shares that knowledge with others. And so I wrote the first article on using Artificial intelligence in personnel selection in a 2016 article in the Journal of Applied Psychology. And I have really become an advocate of the topic ever since. Recently, we edited a special issue in Personnel Psychology that's in press that presents 11 more studies from a range of authors. And we also have several ourselves, myself and my research team, several articles under review. I guess what impresses me most is how shockingly, it’s the only way I can put it, shockingly effective these techniques are. I believe that they're going to be a game changer in the field and and you can bet that I'll be publishing the findings as soon as they're discovered.
EC: The speed at which candidate materials can be scored and assessed has incredible benefits for both the employers and the candidates. So for employers, they want to quicken their time to hire. So to candidates. But so I think it has this great payoff to organizations that instead of waiting days or weeks or months to sift through a pile of resumes or application materials on your desk, that you're able to maybe do it in just a couple hours using an algorithm that gives the, you know, an equal amount of time to each candidate as it evaluates it.
You know, searching for a job is so tough. I don't know if anyone truly enjoys it. So anything that we can do to help improve candidate job experience I think is a fruitful and worthwhile effort. And I think using text analysis to quicken the process for candidates is so important. A lot of these individuals and I like I mentioned, I mostly work with, you know, undergraduate students who are seeking their first professional job. And, you know, the waiting is some of the most uncomfortable is one of the most uncomfortable pieces, I think, for them. And so if there's a way that we can quicken the process for them to be able to then shift or adjust their focus to the next job instead of maybe waiting to hear, I think is absolutely worthwhile.
MC: In addition, one of the things that excites me so much about Talent Select AI is how the application of these fairly basic artificial intelligence tools to this new context is so very effective. The statistical prediction of job, excuse me, of interview ratings and of the decisions of hiring organizations based on these measures is shocking, is the only way I can describe it. The correlations are huge, surprising in every way, and that's very appealing. And it's probably due in part to the measurement. It's due to in part to the constructs. But I think it's largely due in part to using information that was heretofore not evaluated very well in the hiring process. And when it comes right down to it, more and better data trumps fancy analysis. And so I think that even though these are very basic tools, they are able to measure things we didn't measure very well before. And those things add greatly to prediction.
